28

Browsing MS-DOS sources on GitHub, I often see mentions of XENIX:

DOSSYM.ASM:

; XENIX calls all return error codes through AX.  If an error occurred then the    
; carry bit will be set and the error code is in AX.  If no error occurred then    
; the carry bit is reset and AX contains returned info.

ALLOC.ASM:

;
; xenix memory calls for MSDOS
;

MSHEAD.ASM:

; 1.40 06/15/82 Tree structured directories.  XENIX Path Parser MKDIR CHDIR
;               RMDIR Xenix calls

There are even files called XENIX.ASM and XENIX2.ASM.

What's the story here? Did XENIX and DOS share source code?

Igor Skochinsky
  • 1,780
  • 11
  • 23

4 Answers4

44

They didn't share any source, no. However, the TITLE directive twenty lines or so down from the top in both XENIX.ASM and XENIX2.ASM explains what this is:

TITLE   XENIX - IO system to mimic UNIX

Pre-2.x MS-DOS was somewhere between heavily inspired by CP/M and a complete rip-off of it. But with 2.x they decided to go in a quite different direction, and one of the huge differences was using directories and subdirectories, rather than user areas, to make it easier to organize files on disks expected to hold hundreds or even thousands of files.

Unfortunately, the CP/M API was entirely unsuited to this. The main problem was that CP/M (and MS-DOS 1.0) used file control blocks in the memory space of the program to hold information about files and their current state; changing the format of these was a breaking API change. (They couldn't be extended because older programs assumed they were a specific length, and nor could the "public" fields be changed without breaking older programs' expectations.)

Rather than just randomly rolling their own new API the MS-DOS 2.x developers looked elsewhere for inspiration, in this case at Unix via its derivative Xenix, a port of Unix to Intel processors done by Microsoft. Since the new API was modeled on the Unix/Xenix file I/O API, these new API calls were called "XENIX calls."

As well as providing an API that could handle the extra information about directories, this also changed the API to use "file handles" referencing data structures owned by DOS, rather than in the program's memory space, thus allowing these structures to be further changed in the future without breaking compatibility with older software. (APIs that had previously been direct manipulation of the FCB by the program now became functions that took and returned file handles, with the OS updating the data structures it owned.)

cjs
  • 25,592
  • 2
  • 79
  • 179
  • 23
    So sort of like "POSIX compliant", except not POSIX and not compliant :-) – manassehkatz-Moving 2 Codidact Aug 15 '19 at 02:40
  • 4
    @manassehkatz They wanted it to be POSIX compliant. They had to make concessions to IBM and CP/M compatibility. This was a time when MS thought UNIX is the future :) – Luaan Aug 15 '19 at 08:56
  • @Luaan "Concessions" is an interesting term for for "rip-off" :-P But thanks for the comment; it inspired me to expand my answer a bit to discuss this. – cjs Aug 15 '19 at 11:00
  • 16
    @Luaan: at the time that MS-DOS was written there was no such thing as "POSIX compliant". MS-DOS 1.0 dates to 1981. (And as 86-DOS it goes back to 1980). MS-DOS 2.0 came out in 1983. PC-DOS 3.0 was released in 1984. The POSIX standard came out in 1988. – Bob Jarvis - Слава Україні Aug 15 '19 at 11:39
  • 1
    @BobJarvis You're right. It would be better to say "as compatible with Unix-like systems as possible while maintaining as much compatibility with IBM DOS and CP/M". – Luaan Aug 15 '19 at 12:32
  • 2
    More a "follow a similar model" to Unix than actually attempting compatibility, I would say. – Russell Borogove Aug 15 '19 at 15:39
  • 10
    Even though MS no longer thinks so, it turns out that UNIX is the future. :-) – John Bollinger Aug 15 '19 at 19:13
  • 5
    @JohnBollinger: You're a few years behind the times. Now MS does think unix is the future again, which is why they have WSL/WSL2 - they just want you to be running it on top of their system. – R.. GitHub STOP HELPING ICE Aug 17 '19 at 18:29
  • 1
    @JohnBollinger: I remember a PC magazine columnist writing, around 1990, something like "DOS is dead, but it isn't dead. Windows is in the future, but it isn't the future. OS/2 is where everyone is going, but it's going nowhere, and [the part I remember most clearly] Unix is what nobody wants,. but it's already here". – supercat Aug 20 '19 at 15:40
  • @supercat That's a great summary, except that it left out the obvious winner of all those OS battles: LISP machines! :-) – cjs Aug 20 '19 at 16:10
  • @CurtJ.Sampson: Actually, I would think an architecture based on hardware-assisted GC could be a very useful concept, especially if it could handle various lazy-update constructs for certain kinds of "dirty" flags, and could make certain operations use strong memory semantics when certain parts of the GC are running in another thread and weak memory semantics otherwise. – supercat Aug 20 '19 at 19:56
15

The source code files in question appear to have the implementation for the MS-DOS 2.0 'XENIX-style' APIs to open/close/etc. files without a File Control Block used in MS-DOS 1.0 and CP/M.

I strongly suspect the authors used 'XENIX' as a shorthand for 'those new-fangled IO methods'.

Nowadays, of course, everyone uses the 'new-fangled' APIs and the FCB APIs have long since been removed.

Eric Brown
  • 561
  • 3
  • 5
  • 5
    16-bit FCB API was supported in Windows all the time until the whole 16-bit subsystem has been removed from 64-bit Windows. You could still use FCB API nowadays in a 16-bit DOS application running under 32-bit Windows 7. – Egor Skriptunoff Aug 15 '19 at 08:23
  • 3
    @EgorSkriptunoff And 64-bit Windows only really removed the 16-bit subsystem because of Intel's decision to remove support for 16-bit code running in 64-bit mode on their CPUs. It's an annoying compatibility break, given how long both Intel and MS managed to support ancient DOS applications .) – Luaan Aug 15 '19 at 08:57
  • 1
    FCB's described the CP/M 8+3 filenames. XENIX probably refers to Unix path name strings instead. – Thorbjørn Ravn Andersen Aug 15 '19 at 09:45
  • @Luaan: I can't back it up now but it's always been a widely held assumption that all Windows versions only retain compatibility with one step backwards (32 with 16, 64 with 32), not two. – Gábor Aug 15 '19 at 10:26
  • 2
    @Gábor It's an observation that fits available data, but that makes it a mere correlation, without any causal links. 32 was compatible with 16 because it was possible, and Microsoft wanted to keep compatibility (because they understand it's the applications that sell OSes). 64 is not compatible with 32 because the CPU doesn't support that. There's no point in having a 16-bit subsystem if you can't actually run it on the CPU. Keep in mind that 64-bit Windows (except for XP) still have the full 32-bit Windows system; 16-bit support was already there, and had to be removed. – Luaan Aug 15 '19 at 11:36
  • 1
    The decision wasn’t Intel’s, it was AMD’s. It is still possible to run 16-bit code alongside long mode, but it involves a fair bit of effort and no one seems to have thought it worth it. – Stephen Kitt Aug 15 '19 at 11:45
  • @Luaan: now that I checked, there is 16-bit support still on every latest W10/64 shipped. You need to switch it on (well, install, practically, NTVDM). I have absolutely no experience with it I and don't seem to need to find it out, but it's there, they say. – Gábor Aug 15 '19 at 11:55
  • 1
    @Luaan: If the CPU wouldn't allow simultaneous use of 16-bit and 64-bit applications, how is VMWare able to do it? Even the only way to make 16-bit code work was to use an instruction-level emulator, a modern PC running instruction-level emulation of the 8086 could run DOS programs much faster than most of them would have run on the platforms for which they were designed. – supercat Aug 15 '19 at 17:12
  • 1
    @Gábor There's no official support (or practical support) for 16-bit apps in x64 builds of Windows 10. There's some hard-coded support for 16-bit installer data files, but even there, there is no 16 bit code running. – Eric Brown Aug 15 '19 at 17:23
  • 1
    @supercat VMWare runs a virtual machine, and as such, it's effectively emulating the 8086. – Eric Brown Aug 15 '19 at 17:24
  • @Luaan: I know the x86-64 architecture removed support for virtual 8086 mode in long mode, but as far as I know it still supports 16-bit code and data segments for protected-mode programs that think they're running on an 80286. So the technical reason 64-bit Windows won't run 16-bit Windows 3.x programs natively is not as clear as you make it sound. You'd need to emulate the 16-bit syscalls rather than run a real MS-DOS instance, but as far as I know that is exactly what 32-bit Windows NT already did. – hmakholm left over Monica Aug 15 '19 at 17:28
  • @EricBrown: Haven't all versions of Windows since 3.0 Enhanced Mode run DOS programs in a virtual machine? – supercat Aug 15 '19 at 17:32
  • 2
    @EricBrown. No. Virtual machines of today are not emulating the hardware, they are capable of providing the environment, with massive support from the CPU itself, so that a program running doesn't even know it runs in a box in a box in a box. But the individual CPU instructions and the hardware in general is not emulated, it's used directly (hence the native speed; actual emulators like the old PowerPC ones or later, Android emulators on Windows did emulate, together with the associated huge speed reduction). – Gábor Aug 15 '19 at 17:41
  • @EricBrown. Having said that, if MS doesn't want to bundle a 16-bit environment, it's their decision, and hardly anybody misses it nowadays (and those who do are really free to install a VM with whatever OS they want inside, I still have an XP lying around in a VM because, from time to time, I had to interface with a program that only runs there). But even then, it's not emulated, it's virtualized. Rather huge difference. And the CPU supports it all right. – Gábor Aug 15 '19 at 17:43
  • @supercat Actually, that was one of the great features of Windows 3.0 - they figured out how to make DOS programs run natively within Windows, while keeping the memory protection of the rest of the system. Raymond Chen has a great story about this, and how it was one of the things that essentially killed OS/2. – Luaan Aug 16 '19 at 06:58
  • @Gábor Of course it works with virtualization, but virtualization is very different from just running the code in Windows. Virtualization is a special CPU feature that isolates the host from the code running in the guest; this also includes things like the GPU, disk drives etc. Mac OS has used virtualization for backward support, but it doesn't integrate cleanly with the rest of the system the way Windows did. But even in the guest machine, as soon as you start the "virtualized" CPU in 64-bit mode, you lose 16-bit. So you can run 32-bit Windows in a VM, and it will work on a 64-bit CPU. – Luaan Aug 16 '19 at 07:00
  • @Gábor: Until my machine at work was upgraded to Windows 7/64, my favorite text editor (PC Write 3.x) was one I'd been using since 1987, and I still used Turbo C 2.11 a lot. When working with a command line utilities, being able to leave an editor and be in a command-prompt window with a context-appropriate current directory and command history was nice, and programs designed for floppy-era DOS could accomplish many tasks faster on a late 1990s machine than modern equivalents can on today's systems. – supercat Aug 16 '19 at 15:14
  • 1
    I suspect this: they really wanted to say "UNIX-style I/O", and not "new fangled I/O methods"! But then boss cracked down told the programmers to refer to the name of the Microsoft-licensed Unix product, rather than to a trademark name not owned by Microsoft. It's not a far-fetched possibility that the code originally existed with the word UNIX in it and was scrubbed later. Obviously, MS-DOS 2.0 cribbed general features from Unix, not Xenix specifically. – Kaz Aug 16 '19 at 21:54
10

The major relationship between MS-DOS and Xenix is that both were Microsoft products. MS-DOS was originally 86-DOS, from Seattle Computer Products, and was licensed by MS to develop PC-DOS. Xenix was a version of Unix which Microsoft licensed from Bell Labs (which was legally prohibited from selling software to consumers) and re-sold.

  • 1
    And so when MS-DOS 2.0 imitated Unix features in its system calls, they used the word Xenix in the source code instaed, because that was a Microsoft trademark name they could safely use. To someone reading the code it's obviously that Xenix is a code word denoting "features cribbed from Unix". – Kaz Aug 16 '19 at 21:56
7

I found the following in the history section of The MS-DOS Encyclopedia (around "Version 2"). Sorry for the long text but I could not find a good way to trim it without losing relevant details. Emphasis is mine.

In developing the first version, the programmers had had two primary goals: running translated CP/M-80 software and keeping MS-DOS small. They had neither the time nor the room to include more sophisticated features, such as those typical of Microsoft's UNIX-based multiuser, multitasking operating system, XENIX. But when IBM informed Microsoft that the next major edition of the PC would be the Personal Computer XT with a 10- megabyte fixed disk, a larger, more powerful version of MS-DOS--one closer to the operating system Microsoft had envisioned from the start--became feasible.

There were three particular areas that interested Microsoft: a new, hierarchical file system, installable device drivers, and some type of multitasking. Each of these features contributed to version 2.0, and together they represented a major change in MS-DOS while still maintaining compatibility with version 1.0.

[...]

Ultimately, it was a hierarchical file system that found its way into MS-DOS 2.0 and eventually convinced everyone that it was, indeed, the better and more flexible solution to the problem of supporting a fixed disk. The file system was logically consistent with the XENIX file structure, yet physically consistent with the file access incorporated in versions 1.x, and was based on a root, or main, directory under which the user could create a system of subdirectories and sub- subdirectories to hold files. Each file in the system was identified by the directory path leading to it, and the number of subdirectories was limited only by the length of the pathname, which could not exceed 64 characters.

In this file structure, all the subdirectories and the filename in a path were separated from one another by backslash characters, which represented the only anomaly in the XENIX/MS-DOS system of hierarchical files. XENIX used a forward slash as a separator, but versions 1.x of MS-DOS, borrowing from the tradition of DEC operating systems, already used the forward slash for switches in the command line, so Microsoft, at IBM's request, decided to use the backslash as the separator instead. Although the backslash character created no practical problems, except on keyboards that lacked a backslash, this decision did introduce inconsistency between MS-DOS and existing UNIX- like operating systems. And although Microsoft solved the keyboard problem by enabling the user to change the switch character from a slash to a hyphen, the solution itself created compatibility problems for people who wished to exchange batch files.

Another major change in the file-management system was related to the new directory structure: In order to fully exploit a hierarchical file system, Microsoft had to add a new way of calling file services.

Versions 1.x of MS-DOS used CP/M-like structures called file control blocks, or FCBs, to maintain compatibility with older CP/M-80 programs. The FCBs contained all pertinent information about the size and location of a file but did not allow the user to specify a file in a different directory. Therefore, version 2.0 of MS-DOS needed the added ability to access files by means of handles, or descriptors, that could operate across directory lines.

In this added step toward logical device independence, MS-DOS returned a handle whenever an MS-DOS program opened a file. All further interaction with the file involved only this handle. MS-DOS made all necessary adjustments to an internal structure--different from an FCB- -so that the program never had to deal directly with information about the file's location in memory. Furthermore, even if future versions of MS-DOS were to change the structure of the internal control units, program code would not need to be rewritten--the file handle would be the only referent needed, and this would not change.

Putting the internal control units under the supervision of MS-DOS and substituting handles for FCBs also made it possible for MS-DOS to redirect a program's input and output. A system function was provided that enabled MS-DOS to divert the reads or writes directed to one handle to the file or device assigned to another handle. This capability was used by COMMAND.COM to allow output from a file to be redirected to a device, such as a printer, or to be piped to another program. It also allowed system cleanup on program terminations.

[...]

At IBM's request, version 2.0 of MS-DOS also possessed the undocumented ability to perform rudimentary background processing--an interim solution to a growing awareness of the potentials of multitasking.

Background print spooling was sufficient to meet the needs of most people in most situations, so the print spooler, PRINT.COM, was designed to run whenever MS-DOS had nothing else to do. When the parent application became active, PRINT.COM would be interrupted until the next lull. This type of background processing, though both limited and extremely complex, was exploited by a number of applications, such as SideKick.

To summarize:

  1. With bigger disks, hierarchical organization of files has become necessary and MS chose the directory tree used by XENIX.
  2. The MS-DOS 1.x FCB APIs could not deal with directories, so they added new APIs which operated on file paths (instead of just name.ext) and returned handles, again apparently inspired by XENIX.
  3. [conjecture] To properly support print spooler, the free-for-all memory management of DOS 1.x ("all memory after the load address belongs to the user program") was no longer usable and DOS needed a way to track which memory area was used by which program. Apparently the memory management code was also borrowed from/inspired by XENIX.
Igor Skochinsky
  • 1,780
  • 11
  • 23