Linux as an Embedded Operating System
by Jerry Epplin
Does Linux have potential as an embedded operating system? Should vendors of high-end commercial RTOSs quake in their Bruno Maglis? This article assesses Linux's features, robustness, limitations, and most importantly, its real-time facilities.
The increasing use of PC hardware is one of the most important developments in high-end embedded systems in recent years. Hardware costs of high-end systems have dropped dramatically as a result of this trend, making feasible some projects which previously would not have been done because of the high cost of non-PC-based embedded hardware. But software choices for the embedded PC platform are not nearly as attractive as the hardware. You can choose DOS, with its well-known limitations; Microsoft Windows, with its lack of real-time capability; or one of the high-end real-time operating systems, which are expensive, proprietary, and mostly non-portable. The Linux operating system presents an attractive alternative to these options, having none of the above disadvantages. Previously used almost exclusively on the desktop computers of Unix enthusiasts with too much free time, Linux has evolved into a sophisticated and reliable operating system that must be taken seriously. One of the most recent developments has been the addition of real-time facilities to the OS, which completes the transition of Linux from a hobbyist's toy to a valuable tool to be used by embedded system designers. These real-time facilities are not yet as sophisticated as those available in high-end RTOSs, and Linux will never be appropriate for systems that must minimize RAM and ROM use. However, for many applications, the advantages of Linux overcome these drawbacks.
The advantages of using PC hardware in embedded systems are by now well known. In contrast to much hardware developed specifically for the embedded market, PC hardware is mass-produced, easily available, and cheap. You can expect interface boards such as analog and digital I/O boards, network interfaces, and image acquisition and processing boards to cost more than twice as much when designed for the VME bus as for one of the PC buses. And with the increasing use of the higher-performance PCI bus, throughput no longer suffers when a PC platform is used.
But no comparable revolution in operating system capabilities has occurred. Along with a demand for lower hardware costs, high-end embedded systems have required much more sophisticated capabilities, such as graphical user interfaces and networking capabilities. Many high-end RTOS vendors have provided these capabilities, often as options which command higher prices. Microsoft Windows also provides them, but it lacks the real-time capabilities most embedded systems require. One could probably
piece together a system based on DOS using separate third-party tools for each component, but the exercise would be frustrating and support provided for developing such a system would be non-existent. What is needed is an OS for which support is available-one that is cheap, mature, and provides the features that high-end embedded systems must have.
The Linux operating system has recently begun to attract attention from the mainstream press for many of these very reasons.1 Many desktop PC users are attracted to its features and robustness, and to the fact that it is available for only the cost of downloading it via FTP. It comes with a full complement of the features and development tools which Unix users have come to expect. Nearly all Unix system and application software has been ported to Linux. TCP/IP-based network protocols are
supplied. Internet client and server software is provided. X Windows is provided, and one can choose among several window managers. Good compilers for C, C++, Java, and other languages are available. Users find these features to be more mature, complete, and easy to use than those provided by Windows. Many companies have at least one Linux enthusiast who, when presented with a problem difficult or impossible to solve with Windows (for example, setting up a PC as a dial-in Web server), responds with "Well, if we just had Linux. . . ."
Now, whether Bill Gates has anything to worry about is a subject for debate in some other magazine. What is important is that Linux, which is owned by, and therefore supported by, no one company, is starting to be accepted by desktop computer users, many of whom can't be considered computer nerds. What makes this possible now is both the maturity of the OS and the increasing prevalence of the Internet over the last few years. Linux users who encounter problems can draw on the expertise of thousands of on-line users through Internet newsgroups and mailing lists. Any problem you encounter has certainly been encountered before by one of these users, and most are willing to help. In my experience, you can usually solve problems more quickly by using Internet resources than by relying on RTOS tech support departments. You may have to sort through a dozen uninformed responses from other newsgroup participants, but at least one answer is likely to be helpful. In contrast, you get only one answer from a tech support department. If it isn't correct, you have to start the whole process again. In addition, companies dedicated to providing support for Linux have begun to appear, presenting an option for those users who feel more comfortable dealing with more conventional technical support arrangements. Also, all of Linux is provided with source code, providing the means to answer even the most difficult questions.
Some embedded system designers will find Linux usable just as it stands. It presents a good alternative to Windows or DOS for applications without real-time requirements or for those with real-time requirements that are met with dedicated hardware or a coprocessor. But such applications are rare. What is needed is a way to implement a real-time system using Linux, and sufficient progress has been made on this front to allow many high-end embedded applications to be implemented. Two general approaches to real-time Linux are being pursued, which I'll call the POSIX approach
and the low-level approach.
POSIX and Linux
POSIX is a movement to standardize the features and interfaces that a Unix-like operating system must have. The idea is to improve the portability of software written for Unix, making the jobs of Unix programmers much easier. Some real-time extensions, known as POSIX.1b or IEEE 1003.1b, have been added to the standard. These extensions include facilities such as semaphores, memory locking, clocks and timers, message queues, and preemptive prioritized scheduling.
Using POSIX as a basis for standardizing real-time operating system features has been rightly criticized.2 The standard is big, clumsy, and bloated with features appropriate for desktop Unix workstations but unhelpful in embedded systems. The standards-making body is dominated by workstation manufacturers unwilling to make concessions to vendors and users of RTOSs. Also, the POSIX system calls reflect the arcane and cumbersome syntax of Unix system calls, so that an operation that takes
one or two calls in VxWorks or pSOS+ may take close to a dozen POSIX calls. Unix programmers are accustomed to this annoyance, but embedded system programmers find it frustrating.
A number of Linux developers are working on implementing POSIX.1b features in Linux.3 This movement has already seen some success, and the effort continues. The POSIX memory-locking facilities and the functions which determine the scheduling algorithm have been implemented. On the other hand, the timer functions and POSIX.1b signals are not yet complete. And perhaps most damaging, the POSIX semaphore and message queue features, which are essential for any serious RTOS, are not yet available.
One promising development for POSIX-based Linux is the implementation of POSIX threads, which are defined in POSIX.1c (or IEEE 1003.1c). Within a process you can have multiple threads, all sharing the same address space. This matches quite well the familiar embedded system concept of a task. Some implementations of POSIX threads are already available for Linux.
While the POSIX approach to implementing a real-time Linux holds some promise, presently and for the foreseeable future only the "softest" real-time applications can be implemented using the POSIX.1b functions. The fundamental problem faced when attempting to graft POSIX.1b functionality onto Linux is the fact that Linux has a non-preemtable kernel. So implementing "hard" real-time features without radically
changing the design of the kernel would seem to be impossible. However, at least one group has succeeded in accomplishing just that.
Low-level Approach to Real-time Linux
Of more immediate interest than the POSIX-based movement are the efforts to implement a "hard" real-time Linux, the most promising of which may be the Real-Time Linux (RT-Linux) project at the New Mexico Institute of Mining and Technology. Observing that Linux is an operating system designed by desktop computer users, researchers there concluded it would be fruitless to try to graft real-time functionality onto an OS designed for timesharing.4 Instead they implemented a simple real-time kernel underneath the operating system, with Linux itself running as just one task within that kernel. Linux runs at the lowest priority, and it can be preempted at any time by higher-priority tasks.
The design philosophy behind RT-Linux was to minimize the changes made to Linux itself, providing only the essentials necessary for implementing real-time applications.5 Minimizing the disruption to Linux makes it easy to port RT-Linux to new versions of Linux as they appear. As a result, RT-Linux relies on Linux itself to provide almost all services needed, with RT-Linux providing only low-level task creation, installation of
interrupt service routines, and queues for communication among the low-level tasks, ISRs, and Linux processes.
One result of this design is that an RT-Linux application can be thought of as having two domains-the real-time and the non-real-time. Functions placed in the real-time domain can meet their real-time requirements, but they must be simple because the resources available to them are quite limited. On the other hand, non-real-time functionality has the full range of Linux resources available for use, but cannot have any real-time requirements. Facilities for communication between the two domains are
provided. But before using RT-Linux, an embedded system designer has to be sure that all of the needed functionality could fit into one of the two domains. Using RT-Linux does not magically make pre-existing Linux functionality real-time. Suppose, for example, a designer has a Linux driver for a serial port, and wants to toggle a parallel output line (using a real-time task) within some fixed time after receiving a byte
sequence on the serial port. The Linux serial driver can't be used because it resides in the non-real-time domain, and you can't predict when the serial driver would awaken the real-time task driving the parallel output line to perform its work. Thus, both the serial and the parallel ports would have to be done in the real-time domain, which
would require redesigning the serial driver.
The RT-Linux facilities for task handling are basic. There is rt_task_init(), which creates and starts a task. The stack size and priority can be specified. Linux itself is run as a real-time task with the lowest priority. The task is set up to run at periodic intervals by rt_task_make_periodic(). The rt_task_wait() facility blocks the calling
task. The tasks are run using a simple preemptive scheduler.
The primary means of communication between the real-time tasks and the Linux processes is the FIFO. The rtf_create() facility creates a FIFO of a desired size. Data is enqueued onto the FIFO by rtf_put(), returning an error if the FIFO is full. Similarly, rtf_get() dequeues data from the FIFO, returning an error if the FIFO is empty.
The most obvious use for this FIFO scheme is data streaming. In a data acquisition application, for example, a real-time task could be set up using rt_task_init() and rt_task_make_periodic() to acquire samples from an I/O board at fixed intervals. This task would send its data to a Linux process using rtf_put(). The Linux process would be in a loop, reading data from the FIFO and perhaps writing the data to disk,sending it over a network, or displaying it in an X window. The FIFO would serve as a buffer, so the Linux process could operate without real-time constraints.
Implementing data streaming systems appears to have been the primary motivation of the RT-Linux designers. But the FIFO scheme provides a pretty good way of implementing semaphores. A binary semaphore can be implemented by creating a FIFO of size one. The "give" operation (also known as "V" or "signal") is then simply an rtf_put() of size one, with the content of the data insignificant and the error return ignored. The "take" operation (or "P" or "wait") is an rtf_get() with size one. A
counting semaphore can be implemented simply by creating the FIFO with a size large enough to accommodate the expected number of "give" operations. So the FIFO mechanism provides most of the functionality needed for task synchronization in real-time applications. The current implementation lacks some functionality to which RTOS users are accustomed, such as priority inheritance (to prevent priority inversion)
and task deletion safety. But careful design can almost always prevent the problems these features are intended to address. In addition, although the FIFO operations can be set up to block when data is not available (when reading the FIFO) or when space is not available (when writing), the syntax for doing so is rather cumbersome, as blocking capability appears not to have been a priority of the designers. Nevertheless, at least one effort is under way to provide an easy syntax for blocking operations on FIFOs.6 This effort also implements timeouts when blocking, an important feature in many embedded applications. The simple, open design of RT-Linux allows users to implement such additional favorite features quite easily.
An interesting aspect of RT-Linux is the way by which the designers made the Linux kernel preemptable. Linux, like most Unix-type operating systems, has a kernel that disables interrupts for long periods of time. This is, of course, exactly what makes Linux a non-real-time OS. Two approaches to solving this problem might be taken. The first is to redesign the kernel so that it can be preempted. But the Linux kernel is
big, complex, and subject to frequent modification. It was designed by programmers with little interest in real-time applications in mind. Thus, imposing a real-time mindset onto the existing code would be impractical. And even if done once, the modifications would have to be reexamined and redone every time a new version of Linux was released-also impractical. The RT-Linux designers instead took a different approach to making Linux preemptable. They divide the interrupts into two groups: those under
the control of Linux, and those controlled by RT-Linux. RT-Linux interrupts, like RT-Linux tasks, are restricted in what they can do; they cannot make Linux calls. So there is no reason that they can't interrupt the Linux kernel. After all, they can't interfere with anything in the kernel if they don't change anything in it. On the other hand, Linux interrupts can't be allowed to interrupt the kernel. So RT-Linux implements a virtual interrupt scheme in which Linux itself is never allowed to disable interrupts. Linux uses "cli" and "sti" macros to implement disabling and enabling interrupts. In standard Linux, these macros simply execute the corresponding x86 instructions. RT-Linux modifies these macros so that instead of disabling interrupts when a "cli" is executed, it simply reroutes the interrupt to some RT-Linux code. If the interrupt is an RT-Linux interrupt, it is allowed to continue. If it is a Linux interrupt, a flag is set. When a "sti" is executed, any pending Linux interrupts are executed. In this way, Linux still can't interrupt itself, but RT-Linux can interrupt it.
RT-Linux is simple, providing only a bare minimum of functionality necessary for implementing a real-time system. But this simplicity is to the system designer's benefit. You want to implement the bulk of the application in Linux processes because Linux itself is solid, stable, and popular with a lot of desktop users-so you know you can get help if you have trouble. Real-time tasks should have only the functionality necessary
to perform real-time I/O and pass data to and from the Linux processes. The simplicity of RT-Linux has two advantages: first, its very simplicity makes it less likely that it will be buggy; and second, if you do find a bug, it's likely to be easy to find and fix. These factors are important. Because real-time systems are a minuscule portion of Linux applications, the amount of help you can find for developing code using RT-Linux is
certain to be low. So a feature-rich RT-Linux is not necessarily something to be desired. The functionality now implemented is also sufficient for the vast majority of real-time systems if properly used.
Linux is clearly not the best platform for all embedded PCs. Because of its size, a full GUI-based system must be implemented as a disk-based system or one connected to a network from which it can boot. Still, a large and growing number of embedded applications can run with a disk and need the GUI and networking features that are built into Linux. For example, many medical devices must have attractive user interfaces to be competitive, and industrial machine controls must have both GUIs and
networking. Patching together such a system with DOS or a low-end RTOS is
impractical. Inflating the selling price with a $700 royalty paid to a high-end RTOS vendor is out of the question. Linux provides a way to incorporate these features for free. And not only are they free, they are usually more up-to-date than those supplied by RTOS vendors, who usually provide features well after they are available on desktop OSs.
In addition to the recent developments that make Linux usable in disk-based embedded systems, some progress has also been made in making it bootable from EPROM. By installing only those components that are necessary for the application, in many cases a diskless system can be put together using Linux. Thus, for example, at least one full system including Linux networking (but without X Windows) has been put together using only 2.7MB of EPROM for Linux.7 So, practical stand-alone diskless embedded systems can now be developed using Linux. In addition, the ability to boot Linux from a network is well established. Thus, a system that resides on a network can boot the entire system, including X Windows, from a disk somewhere on that network.
The Future
Linux follows the GNU movement, which produces high quality software through the combined efforts of thousands of programmers. GNU, like Linux, was ridiculed early on as the hobby of software anarchists and nerds with too much free time. But by now all the skeptics have to admit that some world-class software has been produced by GNU, notably the gcc and g++ compilers. These are competitive with the finest commercial compilers around, and even high-end RTOS products like VxWorks and LynxOS use them. What happened to RTOS compilers is likely to happen to the OSs
themselves. RTOS products based on Linux will begin to appear, with vendors emphasizing their added real-time features, as well as their support. The result will be a bonanza for developers of high-end embedded systems, with even the cheapest RTOS products providing more features than are currently available from the most expensive products.
Jerry Epplin is an embedded software consultant in St. Louis, MO. He designs software primarily for medical and industrial device manufacturers. He can be reached via e-mail at JerryEpplin@worldnet.att.net.
References
1. Mohr, Jim, "The State of Linux," Byte, January 1997.
2. Joseph, Moses, "Is POSIX Appropriate for Embedded Systems?," Embedded
Systems Programming, July 1995, p. 90.
3. Kuhn, Markus, "A Vision for Linux 2.2-POSIX.1b Compatibility and Real-Time
Support," ftp://informatik.uni-erlangen.de/local/cip/ mskuhn/misc/linux-posix.1b,
December 1996.
4. See http://luz.cs.nmt.edu/~rtlinux
5. Yodaiken, Victor, and Michael Barabanov, "A Real-Time Linux,"
ftp://luz.cs.nmt.edu/pub/rtlinux/papers/usenix.ps.gz.
6. See http://stereotaxis.wustl.edu/ ~jerry.
7. Bennett, Dave, "Booting Linux from EPROM," Linux Journal, January 1997.
SIDEBARS
Linux Resources
Because of the increasing popularity of Linux as a desktop operating system, information is easy to find on the Internet. The Linux Documentation Project, at http://sunsite.unc.edu/mdw/linux.html, is a good place to start. The site contains plenty of general information, as well as links to the commercial distributors of Linux. Because it is "free" software, controlled by no one organization, the distribution of
Linux is somewhat haphazard. Any company is free to download Linux, add some value to it, and sell it to potential users. In practice, companies have added value by improving ease of installation and by placing the software onto a CD-ROM. As a result, distributions of Linux differ primarily in their installation procedures. The best known distributions are Slackware (available from various distributors), Red Hat (http://www.redhat.com), and Debian (http://www.debian.org). Each has its own
advantages, but most embedded system designers are primarily interested in ease of installation-by that standard, the Red Hat distribution is probably currently the most attractive.
Any of the distributions can be downloaded via FTP free of charge, but because of the large size of Linux, the exercise would be painful even for a user with a high-speed Internet connection; dialup users shouldn't even consider it. CD-ROMs containing everything needed are easily and inexpensively obtained, usually costing less than $50. Further confusing the issue is the fact that anyone can take an entire distribution, place
it on CD-ROM, and sell it. For example, InfoMagic (http://www.infomagic.com) puts the Slackware, Red Hat, and Debian distributions on a set of CD-ROMs and sells it, leaving it to the user to decide which distribution to use. Other vendors provide similar products-see the Linux Documentation Project Web site for a list.
This system of distribution, although chaotic, works well. Users must simply remember to obtain tech support from the vendor from whom they bought their CD-ROM. If, for example, you install the Red Hat distribution from an InfoMagic CD-ROM, you must get support from InfoMagic rather than Red Hat. Tech support is generally available
from the CD-ROM manufacturer, but is typically limited to help with installation problems. For help on more advanced issues, dozens of newsgroups, and perhaps hundreds of mailing lists, are active with discussions of every aspect of Linux. In addition, many companies provide fee-based support and consulting services. For example, Yggdrasil Computing, which provides its own Linux distribution, also provides support for other distributions on a fee basis. A list of companies that provide support and consulting services can be found at
http://www.cyrius.com/tbm/Consultants-HOWTO/Consultants-HOWTO.html.
The recent improvements in ease of Linux installation and use are real, but a word of warning is appropriate for those embedded system designers accustomed to using Windows as a development platform. It is possible to install and use Windows for cross-development without ever learning anything significant about that operating system. This is not true of Linux-by the time you have installed Linux and are using it for developing code, you will know a lot about the operating system and the hardware you are using. Linux is still an OS for enthusiasts; to do any significant work with it you must first spend some time learning it. The learning curve is not at all daunting (especially for current Unix users), but one should expect to invest some time before getting results from Linux.
RT-Linux
Information on obtaining and using Real-Time Linux is available at
http://luz.cs.nmt.edu/~rtlinux. Some papers describing the design of and philosophy behind the effort can be obtained there, and are well worth reading. As of this writing, the latest version available is 0.5; it can be obtained via FTP directly from the Web site. Installation is usually straightforward-install Linux and then apply the RT-Linux patch. Also included at the Web site are links to a few projects that use RT-Linux.
One of the most recent and welcome developments is the start of an RT-Linux mailing list. The developers of the system, as well as other experienced RT-Linux users, frequently participate, providing guidance for newcomers and discussing potential future enhancements. Subscribing to this list is a must for anyone using or seriously considering RT-Linux for an application. To subscribe, send mail to majordomo@luz.cs.nmt.edu with "subscribe rtl" in the body of the message.
Return to Embedded.com
Send comments to: Webmaster
Copyright ?1998 Miller Freeman, Inc.,
a United News Media company.