23

An MS-DOS .com file is just raw code/data without header, thus no linking information, and was limited to be loaded into just one segment (64kB). That's the reason corrupted binaries would print "Program too big to fit in memory". But I remember there are .com-files larger than 64kB (some demos?). How did these work? How come they do not produce an error message on loading?

hippietrail
  • 6,646
  • 2
  • 21
  • 60
DarkDust
  • 1,488
  • 14
  • 20

1 Answers1

34

Most large files (over 64KiB) with a .COM extension are really MZ executables; the DOS loader doesn’t care whether the extension is .EXE or .COM, it uses the MZ signature to identify the format. This is the only documented way for a .COM file larger than 64KiB to work, so it’s the only approach which can be relied upon.

However it is possible to build a .COM file larger than 64KiB, even though the documented maximum size is 65278 bytes: 65536 bytes addressable without changing segments, minus 256 bytes for the PSP, minus two bytes for the 0-word pushed to the stack (the return address used when exiting a program in DOS 1 style, with a near RET). When loading with a .COM file larger than 64KiB, the behaviour depends on the specific variant and version of DOS in use:

  • early versions of MS-DOS only read the first 64KiB;

  • by MS-DOS 5 if not earlier (and presumably corresponding versions of PC DOS), the implementation sticks to the specification, and complains that the program is “too big to fit in memory” (version 6) or simply that it “cannot execute ...” (version 5);

  • DR DOS and OpenDOS load the program in its entirety, if it fits into available memory, and start execution as normal.

Care must be taken because DOS will initialise the stack to start at the end of the 64KiB block the program is assumed to occupy, which means part of the binary’s image in memory will be overwritten. Typically one of the first things such a program would do, would be to move its stack... All the segment operations required to access anything outside the first 64KiB also have to be calculated manually.

This was supposed to be documented in release 62 of Ralf Brown’s Interrupt List, and there’s a thread on the topic on alt.msdos.programmer.

Stephen Kitt
  • 121,835
  • 17
  • 505
  • 462
  • 9
    I suspect that when later versions of some programs grew past 64K, Microsoft decided that the best way to maintain compatibility with utilities that might want to launch them would be to allow EXE-format programs to be loaded from file names that end in .COM. – supercat Apr 26 '20 at 17:38
  • 1
    And don't forget that hardware interrupts can occur at any time which pushes some registers on the stack (CS, IP and FLAGS, plus anything the handler pushes). Even if your first instruction is CLI there is a tiny, tiny chance that an interrupt will happen just before and a heap of your stack gets corrupted. – Artelius Apr 27 '20 at 00:24
  • 3
    @Artelius: Ah; the OS could reliably have arranged for that to not be the case. If entry to the program is done by an IRET (and since the launch is done via int 21h this is easy to do), one instruction is guaranteed to execute before an interrup can raise. So if the first instruction is CLI, no interrupt will split it. You still have a hole in your program of a few words returning from the syscall. – Joshua Apr 27 '20 at 04:16
  • @Joshua That's brilliant. Does DOS actually do this? – Artelius Apr 27 '20 at 07:29
  • 1
    @Artelius no, it doesn’t. Every program’s stack needs to take into account the possibility of hardware interrupts, right from the start of execution... – Stephen Kitt Apr 27 '20 at 07:57
  • 6
    @supercat I imagine backwards compatibility played a part, but it was somewhat prescient. In MS-DOS 1.x, executables were loaded by COMMAND.COM, and it only looked at file extensions to determine file types. This was moved into the kernel in MS-DOS 2.0, and detection was changed to signature-based. All the .COM programs in MS-DOS 2 are real .COM programs, not MZ files. The only executable which is too large for a .COM file is GWBASIC.EXE, which was new in MS-DOS 2. Allowing COM files to be MZ files certainly came in useful later (COMMAND.COM itself is an MZ in later versions). – Stephen Kitt Apr 27 '20 at 10:09
  • @supercat: Your suspicion has been confirmed by Raymond Chen. – dan04 Apr 27 '20 at 15:03
  • 2
    @dan04 except that that reads like retro-fitting (and I have the utmost respect for Raymond Chen, so I’m not writing this lightly), because the program loader was flexible before the claimed binary size increases happened. What’s more, FORMAT.COM never got anywhere near 64K, and EDIT.COM appeared late in the game (so backwards compatibility wasn’t an issue) and was also nowhere near 64K. Having a flexible loader did help with COMMAND.COM but that wasn’t an issue when the loader was written. See the source code. – Stephen Kitt Apr 27 '20 at 15:13
  • 1
    @StephenKitt: Even if none of the files supplied with MS-DOS 2.0 had grown too big, it's possible that some may have switched to MZ format for other reasons, or that Microsoft was aware of other programs that needed to do so. Note that while there would be no technical obstacle to a compact-memory-model program loading from a linearly-stored COM file (that's how Turbo Pascal and programs compiled with it worked) Microsoft's tools couldn't create COM files from anything other than Tiny-mode programs. – supercat Apr 27 '20 at 15:18
  • 1
    @supercat like I said, “All the .COM programs in MS-DOS 2 are real .COM programs, not MZ files.” I don’t see any evidence to support the idea that the developers were forced to implement the loader in this way. But yes, it’s possible that they were aware of other programs which would need this. – Stephen Kitt Apr 27 '20 at 15:21
  • 1
    Intriguingly, none of the .COM programs in MS-DOS 6.22 are MZ binaries either; some of them are however compressed. In Toshiba’s version of MS-DOS 5, FORMAT.COM is somewhat weird, it’s a raw .COM binary which has been converted from an MZ binary, which is still contained in the .COM file (the MZ signature is 16 bytes into the binary). – Stephen Kitt Apr 27 '20 at 15:34