Proposition: We assume the following.
1) The force exerted by the air on a surface is pure pressure thus normal to the surface without friction. The pressure increases with respect to the magnitude of the surface normal component of the incident air flow velocity and is zero when the surface normal component becomes negative.
2) The surface of the capsule is axially symmetric. Label the intersection of the symmetric axis and the surface (bottom) facing the incoming airflow $B$. The inward normal vector $\vec n$ of any infinitesimal surface patch either intersects the axis at point $N$ some finite distance from $B$ or $\vec n$ parallels the axis. The center of mass of the capsule $C$ locates between $B$ and $N$.
The capsule achieves aerodynamic stability.
Before presenting the proof of this proposition, I give a plausible toy model of this air flow pressure function. The realistic function will surely be more complicated.
However, interestingly, two and a half months after I posted this answer, I happened upon the theory of hypersonic aerodynamics that surprisingly endorsed fully the following derivation as the correct computation for the pressure of hypersonic (Mach 3-5) airflow on an largely axial symmetric body with blunt surface geometry. c.f. equations (11-2) and (11-3) of chapter 11 on the hypersonic aerodynamics of W. H. Mason's lecture on configuration aerodynamics. Search for "Newtonian Impact Theory" in this accompanying PPT to that chapter.
Suppose an air column of an infinitesimal cross section area $dA$ collide with a facet with its normal vector forming an angle $\theta\in\big[0,\frac\pi2\big]$ with the air flow direction vector. The air bounces off the facet completely elastically. The momentum change (all in the normal direction of the facet) per unit time is then $2\rho v^2\cos\theta dA$, where $\rho$ is the density of the air flow and $v$ the speed of it. The area upon which this momentum change occurs is $\frac{dA}{\cos\theta}$. Divide the first quantity by the second, we get the pressure $p(\theta):=2\rho v^2\cos^2\theta$. Now the early arriving particles bounce off of the surface normally and collide completely elastically with the late arriving particles and bounce back towards the surface again. By symmetry, the average particle velocity near the surface vanishes in the surface normal direction but its component tangent to the surface remains. Macroscopically, the fluid on average as a whole moves along the tangent of the surface. Alternatively we can assume the complete inelastic collision of the air molecule with the surface, so that the momentum normal to the surface completely dissipates only the tangential component is unmolested so the air molecules after the collision move parallel along the surface. In this case, it is clear $p(\theta):=\rho v^2\cos^2\theta$ which is half of the previous value as the surface normal momentum transferred is half of that in the elastic case. In the case of fractional elastic collision, the $p(\theta):=(1+\alpha)\rho v^2\cos^2\theta$ where $\alpha\in[0,1]$ is the coefficient of collision elasticity.
Moreover, the part of the object surface that is in the "shadow" of the incoming airflow will remain untouched by the airflow and thus experience no pressure.
Proof:
1) 2-dimension.
Let us formulate the problem formally. Let $s\in[-s_0,s_0],\,s_0>0$ measure the distance, with sign, from the intersection of the symmetry axis with the surface. Denote the unit inward normal vector at $s$ by $\hat n(s)$. Let $\theta(s)$ be the angle from $\hat n(0)$ to $\hat n(s)$ with counterclockwise direction as the positive direction for the angle. $\theta(-s)=-\theta(s)$ by the axial symmetry. Let the angle from $\hat n(s=0)$ to the incoming airflow direction be $\theta_a$ also with counterclockwise direction as the positive direction. Place the curve $(x(s),y(s))$ in the Cartesian coordinate such that $(x(s=0)=0,y(s=0)=0)$ and the center of mass be located at $(x=0,y=y_c)$. We have $(x(-s),y(-s))=(-x(s),y(s))$. Let $p(\beta)$ be the pressure as a function of the angle $\beta$ with respect to the incoming air flow. The torque at each curve with respect to $(0,y_c)$ is $l(s)p(\theta_a-\theta(s))$ where $l(s)\hat z = \big((x(s),y(s))-(0,y_c)\big)\times \hat n(s)$.
Without loss of generality we assume $\theta_a>0$. Otherwise we can just reflect the coordinate with respect to the $y$ axis and get back the same problem because of the axial symmetry.
The total torque is, needing to account for only the surface facing the incoming airflow,
\begin{align}
T&:=\int_{-s_0}^{s_0}l(s)p(\theta_a-\theta(s))ds \\
&=\int_0^{s_0}l(s)\big(p(\theta_a-\theta(s))-p(\theta_a+\theta(s))\big)\,ds
\end{align}
as $l(-s)=-l(s)$ by the axial symmetry of the curve. Stability is achieved if $T>0$. We have $l(s)>0,\,\forall s>0$ since, by Assumption 2), the center of mass $C$ located at $(0,y_c)$ is between $N$ (at the origin of the coordinate $(0,0)$) and $B$. $p(\theta_a-\theta(s))>p(\theta_a+\theta(s))$, since $|\theta_a-\theta(s)|<\theta_a+\theta(s),\ \forall \theta_a>0,\, \theta(s)>0,\, s>0$, and the fact that $p(u)>p(v),\,\forall |u|<|v|$. Therefore $T>0$.
QED
