Angular distance between two satellites given azimuth and elevation angles for each?

Question

If the look angles for a pair of satellites are $Az_1, El_1$ and $Az_2, El_2$, what would be the angular separation between them? An associated problem is how do we determine whether and when the two satellites, in slightly different orbits, i.e. slightly different COE (Clasical Orbital Elements), are in radio frequency interference with respect to a given ground antenna. Again a way to determine their angular separation is through the angle of their respective topocentric vectors

Answer 1

From "Practical Astronomy with your Calculator or Spreadsheet" by Peter Duffett-Smith, Jonathan Zwart:

Sometimes it is of interest to know what is the angle between two objects in the sky, and this can be calculated very easily provided their equatorial coordinates ($\alpha$,$\delta$) or ecliptic coordinates ($\lambda, \beta$) are known. The formulae are:
cos $d$ = sin $\delta_1$ sin $\delta_2$ + cos $\delta_1$ cos $\delta_2$ cos ($\alpha_1 - \alpha_2$),
or
cos $d$ = sin $\beta_1$ sin $\beta_2$ + cos $\beta_1$ cos $\beta_2$ cos ($\lambda_1 - \lambda_2$),
where $d$ is the angle between the objects whose coordinates are $\alpha_1, \delta_1$ (or $\lambda_1, \beta_1$) and $\alpha_2, \delta_2$ (or $\lambda_2, \beta_2$). Those formulae are exact and mathematically correct for any values of $\alpha, \delta$ or $\lambda, \beta$. However, when $d$ becomes either very small, or close to 180 degrees, your calculator may not have enough precision to return the correct answer, in which case better expressions are
$ d = \sqrt{cos^{2}\delta * \Delta \alpha^{2} + \Delta\delta^2} $
or
$ d = \sqrt{cos^{2}\beta* \Delta \lambda^{2} + \Delta\beta^2} $
where $\Delta\alpha, \Delta\delta$ (or $\Delta\lambda, \Delta\beta$) are the differences in the two coordinates (i.e. $\Delta\alpha = \alpha_1 - \alpha_2$, etc.). These expressions may be used for values of $d$ within about 10 arcminutes of 0 degrees or 180 degrees. Both $\Delta\alpha$ $(\Delta\lambda) $ and $\Delta\delta$ $(\Delta\beta)$ must be expressed in the same units (e.g. arcseconds) and $d$ will then be returned in those units.

The $\beta$ in the $cos^{2}\beta$ above can be either $\beta_1$ or $\beta_2$. It makes little difference, since this formula applies only when d is nearly 0, or nearly 180. Eg, the two points are either very close together, or almost antipodal.

This method should work for RA/Declination, Elev/Azimuth, Lat/Lon, or any similar coordinate system with north and south "poles." (unsure of the general term for these)

I believe this is what you are asking for, as it's generally the most useful in astronomy, attitude determination, etc.

Answer 2

Indeed, the solution to the problem is found in solving a spherical triangle, as @uhoh states.

Consider this diagram for a general spherical triangle:

In this problem, $C$ is the zenith, and the two points are $A$ and $B$; the co-altitudes of the two points are the arc-lengths $A-C$ and $B-C$. The difference in the two azimuths is the angle $ACB$, and the desired value is the arc-length $A-B$. The sphere portrayed is the celestial sphere; the observer is on the tiny, invisible terrestrial sphere, located at the centre of the celestial sphere, down where the three straight lines $A-A'$, $B-B'$ and $C-C'$ intersect.

So you use the Law of Cosines for spherical triangles...

EDIT as suggested by @Uhoh

If we use lower-case letters to denote the arc-length of the sides of the spherical triangles, and Greek letters to denote the angles at each vertex of the spherical triangle, the Law of Cosines gives: $$\cos(c)=\cos(a)\times \cos(b)+\sin(a) \times \sin(b) \times \cos(\gamma)$$Note: error correctd!

The co-altitude of a point is the altitude measured down from the zenith, rather than up from the horizon. The co-altitude is just 90 degrees minus the altitude..

So:

1. Find $b$ and $c$ by subtracting the altitudes of each point from $90$ degrees

2. Find $\gamma$ by subtracting the smaller azimuth from the larger.( If $\gamma$ is bigger than 180 degrees, then use 360-$\gamma$)

3. Substitute these values into the right-hand -side of the formula above to find $\cos(c)$

4. Take the inverse cosine (arcos) of this value to find $c$, the angular separation of the two points

EDITED AGAIN

Writing a small Excel spreadsheet to demonstrate an example:

All the angles are in degrees. Excel's trig functions insist on radians, so the RADIANS function changes degrees to radians, and the DEGREES function, naturally, changes radians to degrees.

Here's the actual values for one case:

You can mimic all of these calculations with a scientific calculator (and even skip the degrees/radians conversions.