Unix

UNIX® or Unix is a family of multi-task and multi-user operating systems that feature a portable, compatible command line and C program support. Unix has diverged into several Unix operating systems and Unix clones, but they have since converged around the Single Unix Specification.

Unix is what beats deep within Mac OS X. The Mac OS X Terminal application is the Unix command line. Many Unix programs are open source and run equally well on Mac OS X, BSD, GNU/Linux, Solaris, and other Unix. You can find some in the UNIX & Open Source section of Apple's website or by running Fink or MacPorts.

History
As is often the case with developments that go on to become highly popular and influential, the beginnings of Unix are unremarkable.

Multics
In the late 1960s, three organisations in the USA created an experimental operating system. These were the Massachusetts Institute of Technology, a university; AT&T Bell Labs, the research arm of the telephone monopoly; and General Electric, a commercial corporation. They called their system the Multiplexed Information and Computing System, or for short Multics. They wanted the system to be interactive (so that you could type commands into it, instead of leaving it to perform batch processing) but to include novel capabilities like enhanced security. They created production released for the GE-645 mainframe computer, but these first releases were slow.

Bell Labs removed itself from the trio to work on its resources elsewhere. Yet one of their developers, Ken Thompson, continued to program the GE-645. He wrote a game called Space Travel. Upon the GE-645, this game was slow and expensive, $75 per go. Thompson decided to move the game to the Digital Equipment Corporation's PDP-7 computer. This required reprogramming the game in PDP-7 assembly language. Another Bell Labs employee, Dennis Ritchie, assisted Thompson with this PDP-7 port.

AT&T Unix
Thompson and Ritchie, having ported the game Space Travel to the DEC PDP-7, led a team at Bell Labs to develop an interactive operating system for the same computer. Their team eventually included Rudd Canaday, Douglas McIlroy and Brian Kernighan. The system became the Uniplexed Information and Computing System, or Unics. People said that "eunuchs" was a castrated Multics.

They renamed it Unix and moved it to the PDP-11/20, where it became a text processing system, using troff to typeset Bell Labs' patent applications. In 1973, they rewrote Unix in C, a new programming language. As the telephone monopoly, AT&T faced restrictions when selling Unix:


 * AT&T "was a regulated monopoly restricted to the telecommunications business and prohibited from entering other businesses. As such, all inventions from Bell Telephone Laboratories unrelated to the telecommunications business could only be licensed externally." – "The AT&T AST OpenSource Software Collection"

Because AT&T was the telephone monopoly, regulations barred it from developing Unix as a commercial product, so AT&T instead gave cheap Unix licenses to commercial firms, university and government, including the C source code. The last version widely spread this way was AT&T Unix Version 7 and its VAX port, Unix 32V.

BSD
The University of California at Berkeley possessed a DEC VAX, a computer with hardware support for virtual memory. DEC's own Virtual Memory System could use it, but UCB's copy of AT&T Unix 32V could not. So students hacked in virtual memory support. UCB added other features including the vi editor, the curses library, and the original TCP/IP networking support. Their Berkeley Software Distribution became free (after acquiring a copy) to anyone with an AT&T Unix license.

The Unix licenses from AT&T, plus the development of BSD, together transformed Unix from a two-person port of the game Space Travel to a popular family of operating systems.

The 1991 release of BSD Net/2 (also known as 4.3 BSD-Lite) separated the networking code and the other free parts of BSD from the encumbered AT&T code. This spawned the creation of NetBSD and FreeBSD, free operating systems that spread worldwide; some NetBSD and FreeBSD code is in Mac OS X. It also spawned two commercial operating systems called BSD/OS and NeXTSTEP. NeXTSTEP eventually became Mac OS X.

Systems III and V
AT&T gained permission to sell a commercial operating system after it removed itself from the local telephone companies:


 * "In the late 70's and early 80's, AT&T wanted to expand its business into new markets including the computer market. In exchange for this right, AT&T agreed to split off the now famous consent decree to that split off the various Regional Bell Operating Companies or RBOCs.... The UNIX system remained with AT&T Bell Laboratories and was commercialized for the first time. With the entry into the computer market, AT&T also became more restrictive in the release of potentially commercializable software." – "The AT&T AST OpenSource Software Collection"

The year 1982, AT&T started selling UNIX System III. At 1983, AT&T combined this system with code from other early Unix variants, including BSD, to form UNIX System V Release 1. This and later "SysV" releases spawned numerous commercial variants of Unix. By 1993, most commercial variants of Unix had System V Release 4 at their base, though sometimes with BSD features mixed in. These SysV variants were expensive operating systems, suitable for servers, databases, document processing and business applications.

Many firms that produced workstations or servers wanted to offer Unix upon them. These firms purchased source code and licenses for UNIX System V, ported it to their hardware, then sold their Unix (without the source code) to customers. HP made HP-UX and IBM made AIX. Apple, too, wanted Unix on their Macintosh hardware.

A/UX
The first 68k Macintoshes each had three software components, the ROM and the System file and the Finder. There was no multiple users. Though System 4.2 that introduced the MultiFinder and cooperative multitasking, that was slower than the preemptive multitasking of Unix.

To support Unix software and to provide better performance, Apple ported UNIX System V to the Macintosh. A/UX, for Apple's UniX, had drivers for only some Mac models. Some models required upgrades to the hard disk or RAM or processor. The result was a normal Unix workstation, allowing multiple users and preemptive multitasking. There was also a virtual machine with an A/UX Finder to manipulate Mac applications and HFS partitions, while still using the terminal and MacX. A/UX released appeared from 1986 to 1995.

Over nine years, A/UX never displaced System 7. The $795 license was too expensive for most users. There was a requirement to create separate A/UX and HFS partitions, plus the need for knowledge of root and the command line, so A/UX was never easy to administer like Mac OS X is today. Apple abandoned A/UX after the switch from 68k to PowerPC processors, but Apple forgot not Unix.

"Free software"
Now $795 would never have been a reasonable price for Mac OS X. A price of $130 or less for a Unix-based Mac OS X would never be unless there existed a cheaper alternative than UNIX System V. The first effort to create a free alternative to Unix was the GNU Project from 1983:


 * "By the 1980s, almost all software was proprietary, which means that it had owners who forbid and prevent cooperation by users. This made the GNU Project necessary.... We decided to make the operating system compatible with Unix because the overall design was already proven and portable, and because compatibility makes it easy for Unix users to switch from Unix to GNU." – "Overview of the GNU System"

The second effort to create a free alternative to Unix was the separation of BSD from the encumbered AT&T Unix code. BSD Net/2, the first of the BSD-Lite releases, appeared 1991. Though GNU and BSD-Lite were free software, neither comprised a complete operating system. Then Linux became free in 1992, and GNU/Linux formed an operating system from free software. Linux developed quickly and Apple noticed.

MkLinux
Apple abandoned A/UX with its switch from 68k to PowerPC processors. Then it noticed the rapid development of Linux, a free replacement for Unix. Though Apple had ported the expensive UNIX System V to the 68k Mac creating A/UX, a port of Linux to the PowerPC Mac would encounter greater obstacles. System V had ports to many hardware platforms. Linux 1.3.x hugged the Intel 386 platform.

Meanwhile, the Open Software Foundation Research Institute noticed Linux. At 1990, the OSF had already ported System V (plus part of BSD) to Mach, producing the OSF/1 operating system. Later, the OSFRI mulled a port of Linux to the Mach microkernel, thus combining the filesystems, networking and Unix features of Linux with the multiprocessing, device drivers and portability of Mach.

The confederation of OSFRI and Apple occurred because if one could port Linux to Mach, one could then port this microkernel Linux to the PowerPC Macintosh.


 * "During the early years, most MkLinux development occurred either at Apple or at The Open Group Research Institute in Grenoble, France. MkLinux Developer Release 1 (DR1) was released in early 1996." – "About MkLinux"

Mission accomplished.

Apple abandoned MkLinux after DR3 in 1998. MkLinux was not Mac software; MkLinux offered nothing approaching support for Mac applications. Apple turned its attention to NeXTSTEP which it had purchased in 1997.

However the work done with the Mach 3.0 kernel in MkLinux was extremely helpful in bringing up NeXTSTEP on the Mac platform, which would later become OS X.

Darwin
NeXTSTEP CEO Steve Jobs and his firm had their own experience with Unix and Mach. The Unix code that NeXTSTEP ported to Mach was not System V, and was not Linux, but it was BSD. It called its operating system NeXTSTEP and later OpenStep. Its chief feature was its object-oriented programming interface, now the Cocoa of Mac OS X.

BSD and Mach provided Unix features such as preemptive multitasking, multiple users and compatibility with other Unix operating systems. If Apple used NeXTSTEP for Mac OS, then Mac OS would provide the same features for which users previously had A/UX or MkLinux upon their Macs.

So Apple ported the Platinum interface from Mac OS to NeXSTEP, then called the result Rhapsody or Mac OS X Server 1.0. The Blue Box (being a virtual machine running Classic Mac OS) and the Yellow Box (being the OpenStep environment) were separate environments that acted like separate computers, though both were Platinum.

To go from Rhapsody to Mac OS X, Apple had to implement Aqua and Carbon. Apple also had to bring its older BSD and Mach code to par with other BSD and Mach implementations. Taking code from FreeBSD 5.0 and Mach 3.0, then drafting the Apple Public Source License, Apple established the open source operating system called Darwin. Mac OS X was and is Darwin with extras like Aqua, Carbon, Cocoa, Quartz, QuickTime, and other Mac features added.

Now that the transition to Mac OS X is complete and Panther, Tiger and Leopard are advancing this system, Apple is giving less emphasis to Darwin, instead describing Mac OS X as Unix-based. Darwin exists not to be its own operating system, but to power Mac OS X.

Standards
As operating systems diverged from the original AT&T Unix, and later from both BSD and System V, compatibility between these systems diminished. The creation of certain standards eventually restored compatibility. Many apps from Fink or MacPorts are trivial recompiles of BSD or GNU/Linux apps.

The two main caveats are that:
 * Programs in C (or C++ or another language that compiles to machine code) must be recompiled, because different Unix-based systems have different machine code.
 * Programs that use platform-specific features are not portable. For example, Apache web server is portable, but programs that need Aqua or Spotlight or even resource forks only work with Mac OS X!

The best and most influential standard is POSIX, the standard for portable operating systems from IEEE. Other standards included the System V Interface Definition and the X/Open Portability Guide. These three standards overlapped in places. They have since converged into one document, the Single Unix Specification. BSD, GNU, and Mac OS X tend to follow the SUS despite having certain deviations and not being formally tested for conformance.

The BSD sockets interface (for networking) form a de facto standard; one most easily implements them by using the BSD code, as Mac OS did. Some software packages have become de facto standards, like CUPS for printing. Mac OS X 10.2 and later versions include CUPS.

ELF versus Mach
Most Unix platforms use the Executable and Linkable Format or ELF for machine code objects, including executables and libraries. Mac OS X is a notable exception, because it uses instead the Mach object format. The bad consequence is that programmers can load ELF shared libraries like dynamic modules, but Mach shared libraries and modules are distinct and incompatible. The good consequence is that Mach allows "fat" binaries with multiple processor architectures, our Universal applications; ELF does not allow this.

The difference between ELF and Mach objects is a portability problem with Unix apps that load dynamic modules. Mac OS X 10.4 eases the situation by providing compatible versions of the ELF dlopen and dlsym functions, which upon Mac OS handle Mach modules. One still cannot dlopen a shared library upon Mac OS.

Impact
The influence of Unix has extended to other operating systems, even Microsoft Windows. Users of both OpenVMS and Microsoft Windows have experienced the C language, the TCP/IP networking and other technologies that started with Unix. There are even Unix compatibility layers for both systems. Some say that Unix is "the most important operating system you may never use."

Mac OS X users can just open a Terminal and use the Unix shell and tools already installed.

Branding
In 1994, Novell, the company that owned the rights to the Unix System V source at the time, sold the right to use the name of the Unix software to the X/Open Company (now The Open Group), but they sold the rights to the actual software to the original SCO company (not to be confused with Caldera, which renamed itself to SCO after it bought parts of the first SCO).

Now, "UNIX" is a trademark of The Open Group and, like all trademarks, should be used as an adjective followed by a generic term such as "system." By decree of The Open Group, the term refers more to a class of operating systems than to a specific implementation of an operating system; those operating systems which meet The Open Group's Single UNIX Specification should be able to bear the "UNIX" and UNIX98 trademarks today, after the operating system's vendor pays a fee to The Open Group. Systems licenced to use the UNIX® trademark include AIX, HP-UX, IRIX, Solaris, Tru64, A/UX and a part of z/OS. In practice, the term, especially when written as "UN*X", "*NIX", or "*N?X" is applied to a number of other multiuser POSIX-based systems such as GNU/Linux, Mac OS X, FreeBSD, NetBSD, OpenBSD that do not seek UNIX branding because the royalties would be too expensive for a product marketed to consumers or freely available over the Internet.

The term "Unix" is also used, and in fact was the original capitalisation, but the name UNIX stuck because, in the words of Dennis Ritchie "[when presenting the original UNIX paper to the third Operating Systems Symposium of the American Association for Computing Machinery], we had just acquired a new typesetter and were intoxicated by being able to produce small caps" (quoted from the Jargon File, version 4.3.3, 20 September 2002). Additionally, it should be noted that many of the operating system's predecessors and contemporaries used all-uppercase lettering, because many computer terminals of the time could not produce lower-case letters, so many people wrote the name in upper case due to force of habit.

Several plural forms of Unix are used to refer to multiple brands of Unix and Unix-like systems. Most common is the conventional "Unixes", but has a penchant for playful use of language, and "Unices" (treating Unix as  word) is also popular. The plural form "Unixen" is not common, although occasionally seen.

Canonical UNIX Commands
The most basic UNIX commands/utilities are:


 * Directory/file creation/navigation: ls cd pwd mkdir rm rmdir cp
 * File viewing, editing: touch less more ed vi emacs ex
 * Text processing: echo cat grep sort uniq sed awk tail tee head cut tr split printf
 * File comparison: comm cmp diff patch
 * System administration: chmod chown ps find xargs su w who
 * Communication: mail telnet ssh ftp finger
 * Shells: sh csh ksh bash tcsh zsh

Here is a list of the 60 user commands from section 1 of the First Edition:

ar as b bas bcd boot cat chdir check chmod chown cmp cp date db dbppt dc df dsw dtf du ed find for form hup lbppt ld ln ls mail mesg mkdir mkfs mount mv nm od pr rew rkd rkf rkl rm rmdir roff sdate sh stat strip su sum tap tm tty type un wc who write

For a more complete and modern list, see the list of Unix programs.

See also: //dev/null, //dev/random, //dev/urandom, //dev/zero.

Sources and References

 * The UNIX System Home Page
 * UNIX history with large GIF timeline
 * The AT&T AST OpenSource Software Collection (PDF paper)
 * comp.unix.aux FAQ
 * A/UX - The Unix for the rest of us?
 * A/UX - netfreak's A/UX Experience
 * Overview of the GNU System
 * http://www.mklinux.org/
 * About MkLinux
 * Mac OS X Server/Rhapsody FAQ
 * ADC Open Source
 * Darwin Source Code Releases