From: jeh@ (Jamie Hanrahan, Kernel Mode Systems)
Subject: Re: FX!32, affinity etc.
Date: 1997/05/28
Message-ID: <1997May28.101149.7776@cmkrnl>#1/1
References: <009B44A8.70132C31.4@maxwell.ph.kcl.ac.uk> <338A488D.6FC7@videotron.ca>
Organization: Kernel Mode Systems, San Diego, CA
Newsgroups: comp.os.vms
In article <338A488D.6FC7@videotron.ca>,
jfmezei <"[nospam]jfmezei"@videotron.ca> writes:
> David Cathey (Remove MX to mail) wrote:
>> culmulated into OpenVMS.  The sad thing is Cutler had to capitulate
>> to a Windows assimilation of his theoretically sound base O/S.
>> You'd think he would have taken the tenets of procedure calling
>> standards with him as well.
>
> I think it is quite pretentious of the VMSer to think that Cutler
> had so much design authority over NT and that he would have taken
> his VMS knowledge with him.
Not pretentitious at all.  We're simply aware of facts you apparently
haven't heard about.
From the internals point of view there is utterly no question that NT
is VMS re-implemented.  
> Cutler was told to replace DOS with a real kernel over which Windows
> could run, over which teh windows API could run etc etc.
This is off-topic, but you're incorrect here also.  Originally there
wasn't even going to be a Win32 API - 16-bit Windows had not gotten
all that popular when NT was conceived.  NT was originally to be a
follow-on to OS/2, and a cooperative effort with IBM.  But somewhere
along the way Windows got pretty popular, IBM and MS split the beast,
IBM getting the OS/2 parts and MS keeping the NT parts.
> If you look at the PSION PDA operating system (called EPOC), you'll also
> find many many similarities with VMS. Event Flags, Interprocess
> Mailboxes, shared memory between processes, process priorities, and even
> a utility (SPY) which is the equivalent to SHOW SYS. Its IO system is
> similar to VMS (an equivalent to $ASSIGN with the device name
> determining which driver to use, and $QIO which is more or less
> independant of the device itself.).
That's out at the UI and API level.  We're talking about internal
similarities.
> NT is an WINDOWS operating system with modern operating systems
> services. They were implemented with the Windows API mentality.
> Stop thinking that NT is VMS with WINDOWS above it.
No, not "VMS with Windows above it", but a VMS-derived design with
Windows above it, most certainly.
> NT differs from DOS in that it has real operating system features, but
> the later are not the exclusivity of VMS.
The "real operating system features" you speak of are at the UI and
API level.  They are not the reasons we consider NT to be a
reimplementation of VMS at the internal level.
How about:
The scheduler.  (process scheduler in VMS, thread scheduler in NT)
32 scheduling priorities, divided into the "real-time" (16-31) and
"variable" (0-15) priority ranges.  identical preemption at ready by
higher-priority threads; identical quantum and priority boost
implementations; identical CPU starvation avoidance mechanism to get
out of priority inversion situations; a null thread for each CPU;
etc., etc.
Memory management.  0-7FFFFFFF is per-process, mostly
user-mode-accessible only; 80000000-FFFFFFFF is systemwide, mostly
kernel-accessible only.  Functionally identical implementations of
paging vs. swapping.
I/O.  I could write a book (in fact, I am), but briefly, IRPs are
IRPs, UCBs are "device objects", CRBs are "controller objects", ADPs
are "adapter objects", FDT routines are "dispatch routines",
EXE$QIODRVPKT is IoStartPacket, StartIO routines are StartIO routines,
fork routines are DPC routines, ASTs are APCs... etc., etc., etc.,
etc., etc.
Interrupt handling.  32 levels of interrupts (some simulated but this
is nevertheless the way the code is written).  IPLs on VMS, IRQLs on
NT.  In order:  Passive level, APC (AST) Level, Dispatch (fork) level,
then the IO hardware interrupts, then some "hardware maintenance"
functions like the hardware timer, IPI, power fail notification, and
HIGH_LEVEL to block all interrupts.
Face it, JF, you're wrong.  Worse, you are writing not just in
misunderstanding but in ignorance of the facts.  Please go read
_Showstopper_ and _Inside Windows NT_ (Custer) before opining further
on this subject.
--- Jamie Hanrahan, Kernel Mode Systems, San Diego CA
Internet: jeh@ (JH645)  CompuServe: 74140,2055
drivers, internals, networks, applications, and training for VMS and Windows NT
NT driver FAQ, links, and other information:  http://www./
If you post a reply in news, please don't e-mail it too.

Editor's Notes:

1. I've added some highlighting to the above newsgroup posting but have not changed anything else
2. To develop a useful technology and then not use it is almost a crime against humanity. Unlike some others in this news group, I do not see Bill Gates or Microsoft as evil; likewise I do not see Dave Cutler as a traitor. Kudos to all involved.

An excerpt from "Inside Windows NT" by Helen Custer
Copyright (c) 1993 by Microsoft Press

Note: In 2003 this book was out of print but is still available from used Book sellers (my copy came from www.)

FOREWORD (By David N. Cutler)

In 1965, I graduated from college with a B.A. in mathematics, a minor in physics, and an overwhelming desire to be an engineer and to build things. So I took a job with DuPont in Wilmington, Delaware, as a materials testing engineer. After about a year of absolute boredom, I was lent to the mathematics and statistics group assigned to construct a computer simulation model for a new foam-making process that the Scott Paper Company was developing. Working with machines that never did what I meant them to was humiliating, but within six months I was hooked, and what I had shunned coming out of school -- computers -- turned into my life's vocation.

Soon after, I transferred to DuPont's engineering department, where I could program full time. DuPont had a small group that built online computer system applications. My real motivation for joining this group was to get closer to computers, and in fact, I wanted to work on implementing an operating system. While in the group, I had the good fortune to work on several stand-alone real-time systems where the project involved writing the central control program that scheduled the various tasks and monitored system activity as well as writing the actual application code.

It soon became apparent that the only way I was going to get the opportunity to work on implementing a real operating system was to join a company that made computers for a business. And so in 1971 I left DuPont for a job in Maynard, Massachusetts, with Digital Equipment Corporation (DEC). As it turned out, this put me in the operating system business for quite some time to come. Little did I know that I would be fortunate enough to develop several operating systems in my lifetime; developing one is a rare opportunity for anyone.

My first operating system project was to build a real-time system called RSX-11M that ran on Digital's PDP-11 16-bit series of minicomputers. At the time, our goals seemed very ambitious. We were asked to build a multitasking operating system that would run in 32 KB of memory with a hierarchical file system, application swapping, real-time scheduling, and a set of development utilities. The operating system and utilities were to run on the entire line of PDP-11 platforms, from the very small systems up through the PDP-11/70 which had memory-mapping hardware and supported up to 4 MB of memory.

I have many fond memories of how RSX-11M took shape, I had a rubber stamp made that proclaimed "Size Is the Goal" and proceeded to stamp every last bit of correspondence to make sure that all the programmers and project managers understood how important it was to achieve our goals. We also learned the power of conditional assembly (high level language use in operating systems was in its infancy at this time), and whenever someone added a feature, we just made it a system-generation option.

While developing RSX-11M, we spent most of our time engineering solutions to memory problems. Because the system had to run in 32 KB, we generated a memory budget that divided available memory equally between the between the operating system and the utility programs. That left a mere 16 KB for utility programs and led to long hours tuning overlay structures to achieve acceptable performance for many of the RSX-11M system programs.

Although RSX-11M had some very stringent size and performance constraints, of the systems I've worked on it was probably the easiest one to develop. It involved re-implementing an existing system but allowed us the freedom to change and subset the programming interfaces as long as programs could be reassembled or recompiled with minimal source-code changes. RSX-11M was introduced in 1973, 18 months after we started building it. It proved to be very successful and helped make the PDP-11 the most popular 16-bit minicomputer of its time.

The PDP-11 provide better price/performance than mainframes, was affordable at the department level, and along with other popular minicomputers of the same era, led to the first wave of "downsizing" in the computer industry. Downsizing was an attempt to "bring down" mainframe applications to the minicomputer systems. Many of the mainframe programs were larger than the PDP-11 could easily accommodate, and almost immediately Digital was up what Gordon Bell has deemed the single most important reason that computer architectures become obsolete: the lack of enough address bits.

Out of this need, the VAX architecture was born, and it became one of the most popular architectures of the late '70s and remained popular throughout the '80s. The VAX architecture provided 32 bits of virtual address space and eliminated the need to wrestle programs into what into what seemed to be an ever-decreasing amount of virtual address space.

My second opportunity to develop an operating system arrived with the VAX. I was very fortunate to be chosen to lead the operating system effort for the VAX-11 architecture, the result of which was the VMS operating system.

VMS was Digital's second general-purpose time-sharing system, developed specifically for the VAX architecture. Because the VAX architecture had grown out of the tremendous success of the PDP-11, however, this time it was mandatory to provide more than source-level compatibility for applications.

Thus the VAX-11 architecture included a PDP-11 compatibility mode which PDP-11 instructions were executed directly by hardware. At that time, it was inconceivable that a single operating system could support more than "compatibility environment". Although it wasn't the best known of the PDP-11 operating systems (amazingly, Digital had no fewer the 10 PDP-11 operating systems at one time or another!), RSX-11M was chosen as the operating system interface that would be emulated in PDP-11 compatibility mode on the VAX. This decision probably didn't make sense to a number of people outside the company, but RSX-11M had the largest number of application development tools, and had the most general-purpose operating system features, supporting multitasking, and had a file system structure that could be compatibly extended. Ultimately, the VAX-11 systems ran the RSX-11M binaries right off the distribution kit; it allowed RSX-11M volumes to be directly mounted and their files to be accessed and shared between RSX-11M compatibility-mode programs and native VMS programs.

From a technical perspective, the biggest mistake we made in VMS was not writing it in a high level language. At that time, we had a group of very accomplished assembly language programmers, some stringent size constraints, and no complier with the appropriate quality for operating system development. So to ensure that we would ship the system in a marketable time frame, we wrote it in assembly language. Looking back on what happened, it would still be hard to make the decision to write VMS in a high-level language. (Moral: The right thing to do technically isn't always the best thing to do financially.)

Early in the '80s, while minicomputers were busy absorbing mainframe and other new applications, two important technologies were emerging: the personal computer (PC) and workstations. After the VMS project, I spent a few years developing compliers and then led a group that built Digital's first MicroVAX workstation -- the MicroVAX I.

Workstations like the MicroVAX provided individual, high-performance computing for applications such as computer-aided design (CAD), whereas PCs supported business applications aimed at personal productivity, such as spread sheets and word processors -- two very successful early PC products. Although workstations were relatively pricey, personal computers had to be affordable to small businesses.

In order to meet price objectives, the original PCs were built with 8-bit, and later with 16-bit, microprocessors. They were constrained in much the same way RSX-11M had been and required considerable effort on the part of programmers and operating system designers to accommodate their limitations. Hardware resources were so scarce that operating systems existed mainly to handle a few low-level hardware functions and to provide a set of file system libraries. But the personal computer offered something that minicomputers did not -- a market in which independent software developers could sell their programs at a higher volume. As a result, the breadth and variety of applications that run on PCs and exploit their capabilities is truly amazing.

In the mid-'80s, microprocessors gained 32-bit addressing, and workstations were quick to take advantage of this capability. However, because of the very large installed base of personal computers and their applications, it was not easy to simply roll in another computer and then recompile and relink all the application software. End users of PCs simply didn't have the source code for all their programs, and they demanded binary compatibility.

In the summer of 1988, I received and interesting call from Bill Gates at Microsoft. He asked whether I'd like to come over and talk about building a new operating system at Microsoft for personal computers. At the time, I wasn't too interested in working on personal computers, but I thought this would be a good time to meet Bill and discuss what he had in mind. What Bill had to offer was the opportunity to build another operating system, one that was portable and addressed some of the concerns people had about using personal computers to run mission critical applications. Form me, it meant the chance to build another operating system!

Bill finally convinced me that this was an opportunity I couldn't pass up, and in October of 1988, I came to Microsoft and started to build the team that would build the new operating system. I didn't realize it at the time, but this would be the most ambitious operating system project on which I had ever embarked.

Our goals for the system included portability, security, POSIX compliance, compatibility, scalable performance (multi processor support), extensibility, and the ease of internationalization. Of all these goals, by far the one that was hardest to achieve and that had the most profound effect in the structure of the system was compatibility. Hundreds of thousands of PDP-11 systems had been sold, but tens of millions of personal computers were in operation! As if that weren't enough, we needed to compatibly support three separate 16-bit operating environments and add new 32-bit capabilities to free personal computer applications from the same kind of virtual address constraints that had existed for the PDP-11. To top it off, we wanted to support the UNIX standard interface specification called POSIX.

Now almost four years later, we are on the brink of bringing this system, Windows-NT, to market. Helen Custer started work on this book when the operating system design began. As our design has matured, the book has undergone continual change to track the operating system architecture. This has been an arduous task -- keeping up-to-date and writing and rewriting the various chapters of the book as the design evolved. Although it is our design, Helen is the one who has captured the essence of that design and made it understandable to more than just serious operating system implementers. For this, we owe Helen a great debt.

It is impossible to acknowledge all the people who have contributed to the design of Windows NT. I must say that I did not design Windows NT -- I was merely one of the contributors to design the system. As you read this book, you will be introduced to some, but not all, of the other contributors. This has been a team effort and has involved several hundred-person years of effort. Perhaps the most important contribution of all was made by the people who have tested and stressed the system. Without their effort, Windows NT could not have achieved the level of quality that it has achieved.

I hope you enjoy this book about Windows NT as much as we enjoyed designing the system.

Dave Cutler
Director, Windows NT Development

January-2006 Update

This next paragraph came to me last week (Jan-2006) in a personal email from an ex-Intel employee.

<quote>

"Given your interest in VMS you might find this amusing. In the early 1990's we visited Microsoft to try to ensure that their new OS "Windows NT" would be available on IA32. We met with Dave Cutler, and he was adamant that IA32 was doomed and would we please get lost so he could target Alpha and then
whatever 64-bit architecture was certain to replace IA32 by Intel. It was not a polite disagreement; that guy HATED IA32 and wasn't reluctant to transfer his displeasure to IA32's representatives (us). What an ugly business meeting. Smart guy, though."

</quote>

I asked for a clarification on the phrase "IA32" as it pertains to this quote and was told that it refers to both 486 (which was in production in 1990) and Pentium (586 or P5) which was still in development.

{ note: P6 is the core name for Pentium-Pro which was actually somewhat different from Pentium }

Back to Home
Neil Rieck
Kitchener - Waterloo - Cambridge, Ontario, Canada.