There is a previous stackexchange post on nesting. I will try to briefly explain nesting in the context of the Terashima et al. paper. In Fig 3(C) of the paper there is an illustration of the Fermi surface of a material as measured using ARPES. There are two pockets to this Fermi surface, one pocket near $k=0$ (the $\Gamma$ point) and one pocket near $k=(\pi,0)$ (the $M$ point). These two pockets are said to be nested because there is a single $Q$ vector $Q=(\pi,0)$ that connects many states on the Fermi surface. In other words, you can translate the Fermi surface pocket at $\Gamma$ over to $M$ and it would nearly overlap the Fermi surface pocket already at $M$. If the pockets at $\Gamma$ and $M$ were identical then the nesting would be called perfect. As it stands the nesting is imperfect, so-called quasi-nesting. Nesting (an quasi-nesting) leads to enhancements of interactions and ultimately to exotic phases, such as magnetism or superconductivity. A clearer example of a nested Fermi surface can be found here. In Fig 1 the Fermi surface for a half-filled electron band on a square lattice is shown. Notice that at exactly half-filling ($n=1$ since the band can hold a density $n=2$ electrons, spin-up and spin-down) the Fermi surface is a square. Then the nesting vector $Q=(\pi,\pi)$ connects many points on the Fermi surface. In this case nesting leads to antiferromagnetism.