Does there exist a finitely generated group $G$ with outer automorphism group $\mathrm{Out}(G)$ finite, whose center contains infinitely many elements of order $p$ for some prime $p$?
A motivation is that the existence of such a group would answer the question in this 2011 MO post. Indeed such a group would be an example of a finitely generated group with finite Out but with finite index in a group with infinite Out (namely $G\times\mathbf{Z}/p\mathbf{Z}$).
Edit: here is an example when we drop the finite generation assumption: the universal central extension $G$ of $\mathrm{SL}_n(\mathbf{Q})$ when $n\ge 5$.
Indeed, the central kernel is isomorphic (by standard stability results for $K_2$ of fields) to $K_2(\mathbf{Q})$, which is isomorphic to $C_2\oplus\bigoplus_{p>2}C_{p-1}$, where $p$ ranges over odd primes (see Milnor K-theory book, Chapter 11). So $K_2(\mathbf{Q})$ contains infinitely many elements of order 2 (or even of any other prime order, by Dirichlet's theorem of primes in an arithmetic progression). On the other hand, by Schreier-Van der Waerden, $\mathrm{Aut}(\mathrm{(P)SL}_n(K))$, for any field $K$, is generated by inner automorphisms, the inverse-by-transposition involution, and automorphisms induced by $\mathrm{Aut}_{\mathrm{field}}(K)$ acting entry-wise. Since here the field has a trivial field automorphism group, we obtain that $\mathrm{Out((P)SL}_n(\mathbf{Q}))$ has two elements only. It is immediate that the canonical map from $\mathrm{Aut}(G)$ to $\mathrm{Aut}(G/Z(G))$ is injective, and it follows that $\mathrm{Out}(G)$ is finite.
Actually I expect that central extensions of $\mathrm{SL}_n$ of some well-chosen finitely generated commutative ring should be a source of finitely generated exemples, but it sounds harder.