19

It's pretty well documented how to get the 286 from real mode to its 16-bit protected mode.

It's also widely stated that the 286 had no officially supported way of going back from protected mode into real mode without losing its current state. The "obvious" way to do it was pretty much a no-op.

However, developers still managed to coax the 286 into switching back to real mode from protected mode to access real-mode-only services while protected-mode software was and remained running.

What techniques were used to make the switch back to real mode?

Note: I am specifically asking about the 286 here; I am not interested in answers pertaining to the 386 or newer CPUs.

user
  • 5,286
  • 2
  • 26
  • 43
  • 7
    There was always an official and documented way of going back to real mode while in protected mode, resetting the processor. The trick was doing this in a way that would be useful for operating systems (and the BIOS services that allowed accessing extended memory from real mode.) –  Aug 06 '19 at 16:47
  • 3
    Intel designed the 80286 to boot in real mode, use the basic hardware to things up, and then go into protected mode AND STAY THERE, NEVER TO COME OUT, unless power was cycled. – John R. Strohm Aug 12 '19 at 18:37

3 Answers3

37

I’m aware of three main techniques used to exit protected mode on 286s; they all involve resetting the CPU, they vary on how that is done:

  1. using the keyboard controller (which IBM endowed with the ability to reset the CPU);
  2. using a chipset-specific “fast reset”;
  3. using a triple fault.

There’s a detailed discussion of the first and last approaches on Robert Collins’ site. To reset using the keyboard controller, you write 0xFE (the code for the shutdown command) to the 8042 status port (0x64). To reset using a triple fault, you set the limit of the IDTR to 0, and generate an interrupt; since the segment is now invalid, the CPU faults, but that also involves an invalid segment, so the CPU faults again twice before resetting.

When the CPU resets, it starts running from near the top of memory, in the BIOS; the BIOS checks a code in the CMOS, or a value in RAM, to determine whether it’s running a full boot-up or it’s recovering from a protected mode exit. To set this up, store the restart address (the real mode address your program will resume from) in the DWORD at 0x0040:0x0067, and set the CMOS shutdown type to 5 (see SETPM_RET_ADDR and SET_SHUTDOWN_TYPE for details; these run in real mode before the switch to protected mode, but the setup can also be done in protected mode). After resetting, the program needs to restore the CPU state properly (in particular, it needs to reload all the segment registers and the stack pointer); see Robert Collins’ documentation for details.

Stephen Kitt
  • 121,835
  • 17
  • 505
  • 462
  • 3
    Why does setting the shutdown type needs to happen before entering protected mode? In the code you link to, it seems to be a simple matter of a few OUT instructions; wouldn't it work just as well to do them from protected mode? – hmakholm left over Monica Aug 06 '19 at 22:22
  • @HenningMakholm Can you actually set the return address for the reset if you're already in protected mode? That might be the real problem, rather than setting the shutdown type itself. There are different ways of system shutdown that are controlled by the shutdown type, so it'd be a bit weird if you had to switch to real mode before doing a soft reset, for example. – Luaan Aug 07 '19 at 08:34
  • 1
    @HenningMakholm that’s a good question, and I didn’t think about it when writing the answer! It’s certainly easier to set up the exit from PM while still in RM, but it appears not to be strictly necessary. Setting it up before entering PM does help with handling any involuntary triple faults though ;-). – Stephen Kitt Aug 07 '19 at 08:53
  • @Luaan: The particular code for setpm_ret_addr in Stephen's link will only work in real mode, but nothing would prevent sufficiently privileged code in protected mode from setting up a segment descriptor that will let allow it to write that part of physical memory. – hmakholm left over Monica Aug 07 '19 at 10:39
  • What is easiest probably depends on perspective -- either you're writing a real-mode program that temporarily switches to PM so it can copy data to/from high memory, or you're writing a full-blown protected-mode supervisor that temporarily switches to real mode so it can call BIOS/DOS in real mode.. In each case you'd want to do as much of the set-up work as you can from the same mode the majority of your system works in, whatever that is. – hmakholm left over Monica Aug 07 '19 at 10:41
  • @Henning indeed, if you’re writing a full-blown PM supervisor you’d need a segment descriptor giving access to 0x00400 anyway, so the setup wouldn’t be hard to do from PM. I’ll remove the value judgment from my answer. – Stephen Kitt Aug 07 '19 at 10:54
  • In fact, upon a further look it seems like the setpm_ret_addr routine you link to is designed to run in a "semi-protected mode" where CS still contains the real-mode segment address it had when entering PM bit, but DS contains a PM segment selector. See the "kablooie" comment. – hmakholm left over Monica Aug 07 '19 at 11:20
  • @Henning I think the comment isn’t quite aligned with the code: DS is unconditionally loaded using ABS0 before it’s used at all. The function is called from real mode here. – Stephen Kitt Aug 07 '19 at 11:26
  • @StephenKitt: Ah, that makes more sense. (I assumed based on the comment that ABS0 must be a constant expanding to a selector for a first-page segment). – hmakholm left over Monica Aug 07 '19 at 11:55
  • 2
    Larry Osterman (engineer at Microsoft) describes on his blog the triple fault command they discovered for the 286. –  Aug 07 '19 at 18:44
  • (Software dev here, novice at hardware) I'm curious, is it possible triple fault, whilst still keeping most of the state of the computer (registers, or at least RAM?), or would something like this require writing out to some persistent storage, and reading it back after reset? – Alexander Aug 07 '19 at 20:24
  • 1
    @Alexander in this setup the memory contents are preserved, only the registers need to be restored; the CPU reverts to its startup state on reset, but nothing else changes. – Stephen Kitt Aug 07 '19 at 21:20
  • @StephenKitt Cool to know, thanks. One more question: During a restart, what's responsible for clearing the ram, and resetting any other stateful hardware to its initial blank state? – Alexander Aug 07 '19 at 21:25
  • 2
    @Alexander RAM would generally be overwritten as a side effect, not as a deliberate choice. If you're curious, do consider posting a separate question so the issue can be answered in more depth. – user Aug 08 '19 at 06:41
12

The trick is that resetting the CPU leaves the RAM intact (unless the system designers have done something that will clear some or all RAM on a reset). So if you leave appropriate information in the RAM to be read by the (real-mode) startup code, it can detect that before the reset a "message" was left to it to do whatever needs to be done in real mode before going back to protected mode. (Re-entering protected mode would also generally use the protected mode descriptor tables and other information also left intact in RAM, though of course any setup for other devices (such as interrupt controllers) that were also reset also needs to be redone.)

This is documented in a bit more detail on Wikipedia.

cjs
  • 25,592
  • 2
  • 79
  • 179
1

Fastest way was not to go to protected mode in the first place. Using the undocumented LOADALL opcode the CPU state can be freely set, except it can't be used to return from protected mode to real mode. But actually most compatible way was to cause a triple fault when return from protected mode was needed.

Justme
  • 31,506
  • 1
  • 73
  • 145