How are the coefficients in the EGM96 model normalized?

Question

For my own interest I'm trying to implement the EGM96 gravitational model. I found the coefficients here. The readme file says that they are "fully normalized", but I'm not 100% sure what this means.

For instance MATLAB defines fully normalized associated Legendre polynomials as: $$ N_\ell^m = (-1)^m\sqrt{\frac{(\ell+0.5)(\ell-m)!}{(\ell+m)!}}P_\ell^m$$

whereas the Wikipedia article on spherical harmonics says that geodesy often uses the normalization: $$ Y_\ell^m = \sqrt{\frac{(2\ell + 1)(\ell-m)}{(\ell +m)!}}P_\ell^m $$

The readme file uses the notation Ynm, which leads me to believe I should be using the second normalization. Is that correct?

Answer 1

The normalization factor conventionally used in satellite geodesy appears in [Kaula's Eq. 1.34]{https://store.doverpublications.com/0486414655.html}. Here I call it $N_{n,m}$: \begin{align*} N_{n,m} = \sqrt {(2-\delta_{0,m})(2n+1) \dfrac {(n-m)!} {(n+m)!}} = \begin{cases} \sqrt {(2n+1) \dfrac {(n-m)!} {(n+m)!}}, & \quad m = 0 \\ \sqrt {2 (2n+1) \dfrac {(n-m)!} {(n+m)!}}, & \quad m > 0 \end{cases} \end{align*}

TL;DR

Why normalize?

One compact formula for gravitational potential is \begin{align*} U & = \frac {G M}{r} \left[ 1 + \sum_{n=1}^{\infty} \left(\frac {a_e} {r}\right)^n \sum_{m=0}^{n} P_{n,m}(\cos\theta) \big[ C_{n,m} \cos(m \lambda) + S_{n,m} \sin (m \lambda) \big] \right] , \end{align*} where coordinates $(r,\theta,\lambda)$ are respectively the radial distance, colatitude, and longitude of the point where potential is to be evaluated, all expressed in a coordinate system fixed in the body and rotating with it. Product $G M$ is the body's gravitational parameter. Coefficients $C_{n,m}$, $S_{n,m}$ encode the body's gravitational field. The $P_{n,m}(\cos\theta)$ are Associated Legendre Functions (ALFs). Their argument $\cos\theta$ is the cosine of colatitude $\theta$ measured southward from the north pole, or equivalently the sine of latitude measured north and south from the equator. Degree $n$ and order $m$ are non-negative integers $0, 1, 2, \dots$ with $0 \le m \le n.$ Reference distance $a_e$ and body mass $M$ make $C_{n,m}$, $S_{n,m}$ dimensionless pure numbers.

For large $n$ or $m$, ALFs can become very large while coefficients $C_{n,m}$, $S_{n,m}$ become very small. To compensate, a normalization factor (denoted above as $N_{n,m}$) is introduced which becomes small with large $n,m$. The factor multiplies $P_{n,m}(\cos\theta)$ and divides $C_{n,m}$ and $S_{n,m}$ so that products $C_{n,m} P_{n,m}(\cos \theta)$ and $S_{n,m} P_{n,m}(\cos \theta)$ are unchanged within the potential formula. An overline above $P$, $C$, and $S$ distinguishes "fully normalized" quantities from their unnormalized counterparts: \begin{align*} C_{n,m} \, P_{n,m}(\cos\theta) & = (C_{n,m} / N_{n,m}) (N_{n,m} P_{n,m}(\cos\theta)) \equiv \overline{C}_{n,m} \, \overline{P}_{n,m}(\cos\theta) , \\ S_{n,m} \, P_{n,m}(\cos\theta) & = (S_{n,m} / N_{n,m}) (N_{n,m} P_{n,m}(\cos\theta)) \equiv \overline{S}_{n,m} \, \overline{P}_{n,m}(\cos\theta) . \end{align*} The only change to the potential formula is the overlines: \begin{align*} U & = \frac {G M}{r} \left[ 1 + \sum_{n=1}^{\infty} \left(\frac {a_e} {r}\right)^n \sum_{m=0}^{n} \overline{P}_{n,m}(\cos\theta) \big[ \overline{C}_{n,m} \cos(m \lambda) + \overline{S}_{n,m} \sin (m \lambda) \big] \right] . \end{align*}