from your profile you seem to be an amateur (gifted?, teenager?, still in school?) self-studying GR. Great! So drop the Old Man's Book (the Meaning of Relativity) and get yourself a modern intro to GR or - my suggestion - the wonderful MTW's Gravitation. I did the same when I was 16. It was published in 1973 and Kip Thorne himself told me two weeks ago that he considers it outdated, but for me it is still the best and most inspiring physics book I ever read and you will get an excellent geometric view of GR after reading it.
Book on GR for self-studying:
https://physics.stackexchange.com/a/376/5854
In order to understand GR you need to learn a bit of differential geometry and tensor calculus. Any introductory book on differential geometry wiil explain the inverse metric too.
Anyway, the idea is this:
The metric $g_{ab}$ is a symmetric tensor.
Since it has 2 indices it can also be thought of as a matrix. You get different matrices in different bases (coordinates, if you prefer), to be more precise.
It turns out that all these matrices are invertible, so we can consider their inverses.
We can show that these inverse matrices form the components of a tensor with 2 upper indices. Let's call this $M^{ab}$.
Then we invent the operation of "raising indices of a tensor". This is done by multiplying a given tensor with as many factors of the inverse metric as its lower indices.
And finally you can show that , if you raise the indices of $g_{ab}$, you obtain $g^{ab}=M^{ab}$. In other words: the metric tensor with its indices raised (LHS) is the same as the inverse metric tensor (RHS).
You might find useful these Wikipedia links:
Metric tensor and inverse metric
Raising indices
By the way:
Eq 55 as shown is wrong (probably a typo). The "dx"'s should have upper indices.
Your analysis of eq. 62 is not really correct. But it is not really wrong either :-) It is just a misunderstanding of what Einstein meant and what it appears to mean to you.
Consider again eq. 62
$$g_{\beta\mu}g^{\mu\alpha} =\delta^\alpha_\beta$$
this equation just says that a matrix ($g_{\beta\mu}$) multiplied by its inverse $g^{\mu\alpha}$ is equal to the identity matrix ($\delta^\alpha_\beta$ is a tensor whose components form the identity matrix in any basis/coordinates). Note that the upper and lower $\mu$'s form a contraction, ie. a summation. So LHS is a matrix multiplication - summation/multiplication of rows (of $g^{\mu\alpha}$) and columns (of $g_{\beta\mu}$).
Now, when Einstein writes $\alpha=\beta$ , he means: take $\alpha$ and give it a numerical value, so that it represents a component (for example 1, the x component), then take $\beta$ and give it the same numerical value. You get
$$g_{1\mu}g^{\mu1} =\delta^1_1=1$$ since $\delta^\alpha_\beta$ is a unit diagonal matrix. When Einstein writes $\alpha\neq\beta$ he means: give them different numerical values, say 1 and 0, and you get
$$g_{1\mu}g^{\mu0} =\delta^0_1=0$$ which is 0 by the fact that it is an off diagonal element of the delta tensor. So this is the meaning of equation 62 which is correct.
You instead (probably) thought: $\alpha=\beta$ means "contract $\alpha$ with $\beta$" in which case you indeed get
$$g_{\alpha\mu}g^{\mu\alpha} =\delta^\alpha_\alpha=4$$ a completely different result.