This answer is essentially an elaboration of Prahar's answer in 5D Ricci Curvature but with explicit details of how to determine the coefficients.
It is an entirely straightforward albeit exceedingly tedious calculation.
Let us denote the $D$ dimensional quantities with a $\; \tilde{}\; $ and the $d$-dimensional quantities without one.
We start by looking at the mass-dimensions of the various quantities. These are $-1$ for $x^\mu, x^d, r $, $0$ for $\tilde G_{\mu\nu}, G_{\mu\nu}, A^\mu, \sigma$, and $2$ for $\tilde R, R, F_{\mu\nu}$.
The most general combination of mass dimension two that we could use to expand $\tilde R$, keeping in mind that we have a $d$-dimensional diffeomorphism so that we should have scalars under that and also that we have a gauge symmetry, i.e. under $x^d\longrightarrow x^d + \lambda$ we have $A_\mu \longrightarrow A_\mu - \partial \lambda$, so that $A_\mu$ can only appear in gauge invariant combinations, is
$$
\tilde R = a R + b F_{\mu\nu}F^{\mu\nu} + c (\nabla \sigma)^2 + d \sigma \nabla^2 \sigma + e\nabla^2 \sigma + f \tag{1}
$$
Here $a,b,c,d,e$ and $f$ are to be determined. They can depend on $\sigma$ as that is a scalar in $d$-dimensional space time, but they cannot depend on $G_{\mu\nu}$ or on $A_\mu$, as any such dependence is already in $R$ and $F_{\mu\nu}F^{\mu\nu}$ respectively.
What else can we say? Under a scaling
\begin{align}
x^d \longrightarrow &\, \lambda x^d \nonumber\\
A_\mu \longrightarrow &\, \lambda A_\mu \nonumber\\
e^{2\sigma} \longrightarrow&\, \lambda^{-2} e^{2\sigma} \label{eq:c8hgehkoe}
\end{align}
and keeping $x^\mu$ fixed, with $\lambda$ a constant,
the line element $ds^2 = G^d_{\mu\nu} dx^\mu dx^\nu + e^{2\sigma} (dx^d + A_\mu dx^\mu)^2$ is manifestly invariant. Under this scaling we obviously have
\begin{align}
F_{\mu\nu} F^{\mu\nu} \longrightarrow &\, \lambda^2 F_{\mu\nu} F^{\mu\nu} \nonumber\\
\nabla_\mu \sigma \longrightarrow& \, \nabla_\mu \sigma
\end{align}
The latter relation follows from $\sigma \longrightarrow \sigma - \ln \lambda$.
How does the curvature scale under this? It turns out that $\tilde R$ remains unchanged under this scaling, and so does $R$ as well.
This is not too hard to see if we investigate the scaling of the metric components. The metric is
\begin{align}
\tilde G_{\mu\nu} = &\,G_{\mu\nu} +e^{2\sigma} A_\mu A_\nu \nonumber\\
\tilde G_{\mu d} = &\, e^{2\sigma} A_\mu \nonumber\\
\tilde G_{dd} = &\,e^{2\sigma}
\end{align}
The first think we notice is that
the determinant of the $D$ dimensional metric is very simply related to the determinant of the $d$ dimensional metric. One easily works out that
$$ \tilde G = e^{2\sigma} \det G $$
The second observation is that the inverse of $\tilde G_{MN}$ is very simple
\begin{align}
\tilde G^{\mu\nu} = &\,G^{\mu\nu} \nonumber\\
\tilde G^{\mu d} = &\, - A^\mu \nonumber\\
\tilde G^{dd} = &\,e^{-2\sigma} + A_\mu A^\mu
\end{align}
Here $G^{\mu\nu}$ is the inverse of $G_{\mu\nu}$. It is easily checked that $\tilde G_{MN} \tilde G^{NK} =\delta_M^K$.
From these expression of the metric we find the scaling
\begin{align}
\tilde G_{\mu\nu} \longrightarrow &\, \tilde G_{\mu\nu} \nonumber\\
\tilde G_{\mu d} \longrightarrow &\, \lambda \tilde G_{\mu d} \nonumber\\
\tilde G_{dd} \longrightarrow &\, \lambda^2\tilde G_{dd}
\end{align}
and for the inverse metric
\begin{align}
\tilde G^{\mu\nu} \longrightarrow &\, \tilde G^{\mu\nu} \nonumber\\
\tilde G^{\mu d} \longrightarrow &\, \lambda^{-1} \tilde G^{\mu d} \nonumber\\
\tilde G^{dd} \longrightarrow &\, \lambda^{-2}\tilde G^{dd}
\end{align}
We see that any upper index $^d$ gives a factor $\lambda$ and any lower index $_d$ gives a factor $\lambda^{-1}$. As the Ricci scalar is formed from the curvature tensor with all indices contracted and any $^d$ can only contract with a $_d$ we conclude that $\tilde R$ indeed remains unchanged. $R$ is also unchanged as both $\tilde G_{\mu\nu}$ and $\tilde G^{\mu\nu}$ are invariant under our scaling.
Let us now go back to (1).
The LHS is invariant under our scaling. So the RHS must be invariant as well. Recall that $a,b,c,d,e$ and $f$ are still allowed to be functions of $\sigma$, but as $R$ is invariant and $a(\sigma) R$ must also be invariant we need to have that $a$ is independent of $\sigma$. Similarly as $F_{\mu\nu} F^{\mu\nu}$ scales as $\lambda^2$, we must have that $b$ scales as $\lambda^{-2}$ and so $b\propto e^{2\sigma}$. Finally $\nabla \sigma$ is invariant, so $c,d$ and $e$ must be independent of $\sigma$. We have thus established that
\begin{align}
\tilde R = \alpha R + \beta e^{2\sigma} F_{\mu\nu}F^{\mu\nu} + \gamma (\nabla \sigma)^2 + \delta \sigma \nabla^2 \sigma + \varepsilon \nabla^2 \sigma + f \tag{2}
\end{align}
for some constants $\alpha, \beta,\gamma, \delta$ and $\varepsilon$. We can still have $f$ to be a function of $\sigma$.
To fix the constants we need to work out the expression for $R$.This starts with getting the connections. The non-zero connections turn out to be
\begin{align}
\tilde \Gamma^\lambda_{\mu\nu} =&\, \Gamma^{\lambda}_{\mu\nu} -e^{2\sigma}\partial^\lambda \sigma A_\mu A_\nu + \frac{1}{2}e^{2\sigma} \big( A_\nu F_{\mu}^{\;\;\lambda} + A_\mu F_{\nu}^{\;\;\lambda} \big) \nonumber\\
\tilde \Gamma^\lambda_{\mu d} = & \, - e^{2\sigma} \partial^\lambda \sigma A_\mu + \frac{1}{2} e^{2\sigma} F_{\mu}^{\;\;\lambda} \nonumber\\
\tilde \Gamma^\lambda_{d d} = & \, -e^{2\sigma} \partial^\lambda \sigma \nonumber\\
\tilde \Gamma^d_{\mu\nu} =&\,\frac{1}{2} \Big(\nabla_\mu A_\nu + \nabla_\nu A_\mu\Big) + \frac{1}{2} e^{2\sigma}\Big( A^\rho A_\nu F_{\rho\mu} +A^\rho A_\mu F_{\rho\nu} \Big)\nonumber\\
& + \partial_\mu \sigma A_\nu + \partial_\nu\sigma A_\mu +e^{2\sigma}A_\mu A_\nu A^\rho \partial_\rho \sigma \nonumber\\
\tilde \Gamma^d_{\mu d} = & \, \frac{1}{2} e^{2\sigma} A^\rho F_{\rho\mu} + e^{2\sigma} A_\mu A^\rho\partial_\rho \sigma +\partial_\mu\sigma \nonumber\\
\tilde \Gamma^d_{d d} = & \, e^{2\sigma} A^\mu \partial_\mu \sigma
\end{align}
If you have problems deriving this, let me know and I will give more details.
Let us set $A^\mu=\sigma=0$. In that case we simply have $\tilde R = R$ and so we find that $\alpha=1$.
Next, let us take $G_{\mu\nu}= \delta_{\mu\nu}$ and $A_\mu=0$, leaving only $\sigma$ free. The only non-vanishing metric component with upper indices and connections are then
\begin{align}
\tilde G^{\mu\nu} =&\, \delta^{\mu\nu} \nonumber\\
\tilde G^{dd} =&\, e^{-2\sigma} \nonumber\\
\tilde \Gamma^\mu_{dd} =&\, -e^{2\sigma} \partial^\mu \sigma\nonumber\\
\tilde \Gamma^d_{\mu d} =&\, \partial_\mu\sigma
\end{align}
The Ricci scalar thus reduces to
\begin{align}
\tilde R=&\, \tilde G^{LM} \left( \partial_N \tilde \Gamma^N_{ML} -\partial_M \tilde \Gamma^N_{NL} +\tilde \Gamma^N_{NK} \tilde \Gamma^K_{ML}
-\tilde \Gamma^N_{MK} \tilde \Gamma^K_{NL} \right) \tag{3} \\
=&\,
\tilde G^{\lambda\mu} \left( \partial_N \tilde \Gamma^N_{\mu\lambda} -\partial_\mu \tilde \Gamma^N_{N\lambda} +\tilde \Gamma^N_{NK} \tilde \Gamma^K_{\mu\lambda}
-\tilde \Gamma^N_{\mu K} \tilde \Gamma^K_{N\lambda} \right) \nonumber\\
& + \tilde G^{dd} \left( \partial_N \tilde \Gamma^N_{dd} -\partial_d \tilde \Gamma^N_{Nd} +\tilde \Gamma^N_{NK} \tilde \Gamma^K_{dd}
-\tilde \Gamma^N_{dK} \tilde \Gamma^K_{Nd} \right) \nonumber\\
=&\, \tilde G^{\lambda\mu} \left(-\partial_\mu \tilde \Gamma^d_{d\lambda} -\tilde \Gamma^d_{\mu d} \tilde \Gamma^d_{d\lambda} \right) +\tilde G^{dd} \left( \partial_\nu \tilde \Gamma^\nu_{dd} +\tilde \Gamma^d_{d\kappa} \tilde \Gamma^\kappa_{dd}
-\tilde \Gamma^\nu_{dd} \tilde \Gamma^d_{\nu d} -\tilde \Gamma^d_{d\kappa} \tilde \Gamma^\kappa_{dd} \right) \nonumber\\
=&\, -\partial_\mu \partial^\mu\sigma -\partial_\mu \sigma \partial^\mu \sigma +
e^{-2\sigma} \left[ \partial_\mu \left(-e^{2\sigma} \partial^\mu \sigma \right) - \partial_\mu \sigma \left(-e^{2\sigma} \partial^\mu \sigma\right)\right] \nonumber\\
=&\, -\partial_\mu \partial^\mu\sigma -\partial_\mu \sigma \partial^\mu \sigma -2\partial_\mu \sigma \partial^\mu \sigma -\partial_\mu \partial^\mu \sigma + \partial_\mu \sigma \partial^\mu \sigma \nonumber\\
=&\, -2( \partial_\mu \partial^\mu \sigma + \partial_\mu\sigma \partial^\mu\sigma)
\end{align}
From (2) we have for this choice of metric
\begin{align}
\tilde R = \gamma (\partial \sigma)^2 + \delta \sigma\partial^2\sigma + \varepsilon \partial^2 \sigma + f
\end{align}
and we thus find that $\gamma=\varepsilon=-2$ and $\delta=f=0$. To link this to the expression in (8.1.8) note that
\begin{align}
e^{-\sigma} \partial^2 e^{\sigma} = &\, e^{-\sigma} \partial_\mu \Big( \partial_\mu \sigma e^\sigma \Big) = \partial_\mu\partial^\mu \sigma + \partial_\mu\sigma \partial^\mu \sigma
\end{align}
and $e^{-\sigma} \nabla^2 e^{\sigma}$ is just the covariant expression of this.
Finally, let us take $G_{\mu\nu}= \delta_{\mu\nu}$ and $\sigma=0$, leaving only the $A_\mu$ free. Here we don't have to do any calculations as the theory we have is just a Euclidean $d$ dimensional flat spacetime with an Abelian gauge field $A_\mu$ and we know that the action reduces to $-\frac{1}{4} F_{\mu\nu} F^{\mu\nu}$ so that $\beta =-\frac{1}{4}$.
Let us check this. The only non-vanishing metric component with upper indices and connections are in this case
\begin{align}
\tilde G^{\mu\nu} =&\, \delta^{\mu\nu} \nonumber\\
\tilde G^{\mu d} = &\, -A^\mu \nonumber\\
\tilde G^{dd} =&\, 1+ A_\mu A^\mu \nonumber\\
\end{align}
and
\begin{align}
\tilde \Gamma^\lambda_{\mu\nu} =&\, \frac{1}{2} \big( A_\nu F_{\mu}^{\;\;\lambda} + A_\mu F_{\nu}^{\;\;\lambda} \big) \nonumber\\
\tilde \Gamma^\lambda_{\mu d} = & \, \frac{1}{2} F_{\mu}^{\;\;\lambda} \nonumber\\
\tilde \Gamma^d_{\mu\nu} =&\,\frac{1}{2} \Big(\partial_\mu A_\nu + \partial_\nu A_\mu\Big) + \frac{1}{2} \Big( A^\rho A_\nu F_{\rho\mu} +A^\rho A_\mu F_{\rho\nu} \Big)\nonumber\\
\tilde \Gamma^d_{\mu d} = & \, \frac{1}{2} A^\rho F_{\rho\mu}
\end{align}
Let us do the four terms of the Ricci scalar (3) separately. Because $G_{\mu\nu} =\delta_{\mu\nu}$ we do not have to make a distinction between upper and lower indices in $d$-spacetime and will move them all downstairs, but we do need to be careful with the order. We start with
\begin{align}
\tilde G^{LM} \partial_N \tilde \Gamma^N_{ML} =&\,\tilde G^{LM} \partial_\nu \tilde \Gamma^\nu_{ML} = \tilde G^{\lambda M} \partial_\nu \tilde \Gamma^\nu_{M\lambda} +\tilde G^{dM} \partial_\nu \tilde \Gamma^\nu_{Md} \nonumber\\
=&\,\tilde G^{\lambda \mu} \partial_\nu \tilde \Gamma^\nu_{\mu\lambda} + \tilde G^{\lambda d} \partial_\nu \tilde \Gamma^\nu_{d\lambda} +
\tilde G^{d\mu} \partial_\nu \tilde \Gamma^\nu_{\mu d}+\tilde G^{dd} \partial_\nu \tilde \Gamma^\nu_{dd} \nonumber\\
=&\, \frac{1}{2}\delta_{\lambda \mu} \partial_\nu \big( A_\mu F_{\lambda\nu} + A_\lambda F_{\mu \nu} \big) -\frac{1}{2} A_\lambda \partial_\nu F_{\lambda \nu} -\frac{1}{2} A_\mu \partial_\nu F_{\mu \nu} \nonumber\\
=&\, \frac{1}{2} \Big( \partial_\nu A_\mu F_{\mu\nu} +A_\mu \partial_\nu F_{\mu\nu} +\partial_\nu A_\mu F_{\mu\nu} + A_\mu \partial_\nu F_{\mu\nu} - A_\mu \partial_\nu F_{\mu \nu} - A_\mu \partial_\nu F_{\mu \nu} \Big) \nonumber\\
=&\, \partial_\nu A_\mu F_{\mu\nu} = \frac{1}{2} F_{\mu\nu}(\partial_\nu A_\mu -\partial_\mu A_\nu) = -\frac{1}{2} F_{\mu\nu} F_{\mu\nu}
\end{align}
where in the last line we have antisymmetrised the result.
Next we have
\begin{align}
- \tilde G^{LM} \partial_M \tilde \Gamma^N_{NL} =&\,- \tilde G^{L\mu} \partial_\mu \tilde \Gamma^N_{NL} = - \tilde G^{L\mu} \partial_\mu \tilde \Gamma^\nu_{\nu L}- \tilde G^{L\mu} \partial_\mu \tilde \Gamma^d_{dL} \nonumber\\
=&\, - \tilde G^{\lambda\mu} \partial_\mu \tilde \Gamma^\nu_{\nu \lambda }- \tilde G^{d\mu} \partial_\mu \tilde \Gamma^\nu_{\nu d }- \tilde G^{\lambda\mu} \partial_\mu \tilde \Gamma^d_{d\lambda} - \tilde G^{d\mu} \partial_\mu \tilde \Gamma^d_{dd} \nonumber\\
=&\, -\frac{1}{2} \delta_{\lambda\mu} \partial_\mu\big(A_\nu F_{\lambda\nu} + A\lambda F_{\nu\nu}\big) +\frac{1}{2} A_\mu \partial_\mu F_{\nu\nu}
-\frac{1}{2} \delta_{\lambda\mu}\partial_\mu \big(A^\rho F_{\rho \lambda} \big)\nonumber\\
=&\, \frac{1}{2} \Big( -\partial_\mu A_\nu F_{\mu\nu} - A_\nu \partial_\mu F_{\mu\nu} -\partial_\mu A_\rho F_{\rho\mu} - A_\rho \partial_\mu F_{\rho\mu} \Big) =0
\end{align}
The third and fourth terms with the products of the connections are a bit more work albeit straightforward. The result is
\begin{align}
\tilde G^{LM} \tilde \Gamma^N_{NK} \tilde \Gamma^K_{ML} = &\,0 \nonumber\\
-\tilde G^{LM} \tilde \Gamma^N_{MK} \tilde \Gamma^K_{NL} =&\, \frac{1}{4} F_{\mu\nu} F_{\mu\nu}
\end{align}
Bringing the four terms together we conclude that
\begin{align}
\tilde R = -\frac{1}{2} F_{\mu\nu} F_{\mu\nu}+ \frac{1}{4} F_{\mu\nu} F_{\mu\nu} = -\frac{1}{4} F_{\mu\nu} F_{\mu\nu}
\end{align}
and hence $\beta=-1/4$.
