The hardware doesn't directly offer this.
However, to build real tools (e.g., debuggers), some facility like this must exist. This is implemented usually by kernal software for a system, that provides a facility for one CPU to "stop" another, to inspect its register state, to modify its register state, to single-step, to modify the memory map of the other CPU, etc. Such facilities are built on top of low level hardware primitives that allow one CPU to interrupt another, and code organized so that a CPU interrupt causes the interrupted CPU to make its state available to the the interrupting CPU. (This tends to be very tricky code, and extremely hardware dependent).
Windows offers these capabilities (at least the register access part) via the Win32 call, GetThreadContext(). I'm sure there's an equivalent for Win64, and something analogous for Linux, etc.