OS Development Skeptically
This document has been written to address certain questions
of the OS development that often arise when attempting to make a homebrew
operating system. The document is probably of no use to those, who are
professionals in the area. So, if youíre an expert in OS dev and you know the
thing from inside out, donít waste time reading this document. This document is
aimed at newbies, wannabies, and other unprofessional coders, who want to make
their own OS and have already faced (or are yet to face) a lot of trouble down
the way. The document shall give some ideas and hints as to how to do certain
things and how not to do them.
Why Developing an OS?
Right, why developing an OS at first place? What does this
give to one succeeded? What is so fascinating in doing it?
Well, there are several reasons for making an OS. To name a
few reasons and attitudes (add your own if not listed):
a fucking cool coder, much better than those kiddies from the next block.
I can code everything. An OS is a big/small yet interesting project for
me. I can manage. Letís get started!
bored with Windows, Linux, whatever. Itís now time to roll my own. I canít
wait anymore for the better OS; I need to write some code. Letís start
with some fancy code that will boot up my PC off the floppy diskÖ
wanna be famous!
interesting how things really work. Making an OS is a way to learn, itís a
way to improve my programming skills and extend my experience of
making/using systems software. Finally, itís fun.
have a plan to make an embedded device, for which Iím gonna need an OS to
run task-oriented software under it.
got a course project in OS development, need to make some OS and submit it
before the deadline.
Those, who can put their sign next to items 4, 5 or 6 of the
list, should read the document with attention. Those, who can put their sign
next to items 1, 2 or 3 should read it with double attention, or maybe even
triple. The life has proven that the humanís ignorance and stupidity is
practically unlimited. So, you never know what factor should be used as a
measure of increase of the attention or how many more times one needs to reread
something to get the idea. In short, it depends. It depends because the guys
not having college or university education in computer science have very
different knowledge, skills and experience. Those, who have that education,
have at least some basic prerequisites.
And when youíre through with an OS (meaning youíve done it),
youíll be able to say you acquired extra knowledge, skills and experience. If
your OS isnít a lousy piece of software, it may be useful for your future work.
How Hard is it and How Long does it Take to Make an OS?
Just like I said, it depends. It really depends on the
knowledge, skills and experience, which are very diverse among the people
attempting to do an OS on their own, to do it from scratch.
It also depends on the kind of the OS in mindÖ If thereís
nothing in mind, it will probably take unfortunate (if not indefinite) to make
an OS. Actually, this is probably on of the very first things that must be
decided early on, thus this contributes to the prerequisites, to which Iíll
OK, speaking in terms of time, OS development takes months.
Or years. Yes, that much. Take my word for that or, if you donít trust me, try
and see yourself. :)
Just how hard is it? Iíd say the difficulty and amount of
time needed are proportional to one another. Itís inappropriate to measure the
difficulty in some everyday units, especially mechanical-like ones (e.g. number
of source code lines or number of key presses per day multiplied by number of
days spent in typing :). There are problems that can only be solved by thinking
and finding correct solutions, not by writing more code, not by fixing holes in
the algorithm that is just plain wrong to work in general or in particular.
Some OS Dev Myths
is easy: itís actually not. If my words arenít convincing, give it a try
and see :)
assembly language required: it is always required. It can be as little as
1% of the whole source code, but it needs to be there to take advantage of
the target hardware (e.g. CPU) specific facilities, of which your HLL
compiler (say C/C++) knows nothing
must be everywhere so things work nicely: the 1st point is that
you can make flexible software w/o what is meant to be OOP (e.g. in C, w/o
C++), use pointers to achieve flexibility.
The 2nd point is that if you decide to use OOP, say you go for
C++, you will have to be very careful about writing any OOP code because
it relies on the proper runtime support of the OOP functionality (memory
management to store classes/objectects and exception handling, to start
with). You must either provide this runtime support for these OOP things
or refrain from using them altogether. Unstable or buggy OS kernel is of
little to no valueÖ You might find that it is wiser to make the kernel
core in say C and ASM and leave OOP to the user applications and things
like a GUI. The problem with runtime support for OOP (in case of C++) is
that this runtime support isnít standardized and may differ not only from
compiler to compiler but also from one version to another of the same
The 3rd point is that if you want your OS to be OOP from inside
out, you must understand where you can have benefits from using the OOP.
Just declaring the OS as an OOP OS doesnít make it any OOP-like. Using OOP
blindly doesnít either. First, find areas where OOP is really beneficial
for the sake of the OS and user applications. Figure out all those base
classes and abstract APIs, which will make the thing flexible, modular,
structured, reusable, extendable, etc etc. Use OOP where it is beneficial
in terms of functionality and where it simplifies programming without
making any trouble.
Iíve seen projects with extensive use of C++ and OOP where they were not
need, helped little in solving the task in hand and actually made the
source code less readable.
Iím not against OOP. It just has to be in the right place and amount, just
There should be more myths, but these are just the ones off
top of my head. There have been many posts in OS dev related newsgroups about
What are the Prerequisites and Desirable Background?
If you think
education is expensive, try ignorance.
- Some smart guy
to do an OS
to work independently and actively search for solutions, not relying on
the help from anyone else. Ability to work in team is also a good thing
skills (calculus is probably not necessary, but it may be useful)
solving and logic reasoning skills
language skills (such as Pascal/Ada or C/C++ and the like, but forget
language skills (I really mean it)
of the digital logic (including Boolean algebra, deMorgan laws,
understanding of finite state machines, see for example the book by Brown
of the basic computer architecture (arithmetic and logic unit, memory
organization, caching, bus organization and operation, program/data and
I/O address spaces, I/O capabilities and interrupt servicing, etc; see for
example the book by Tanenbaum on this, etc)
of the fundamental algorithms such as searching and sorting (not limited
to these only) and corresponding data structures organization (see e.g.
Sedgewick, Knuth, etc)
of the basics of the OS theory (e.g. memory management, multithreading and
scheduling, priorities, race conditions, synchronization primitives,
deadlocks, file systems, networks, device drivers, terminal/console I/O
and GUI, etc etc; see for example books/docs by Tanenbaum, Finkel, etc)
basic OS plan, e.g. overall structure, required functionality and
performance plus some ideas on how to do such and such part
If anything of the above is missing, you will have two
options in general:
or catching up and keeping going
Many guys had done some lousy work and then quit because
they were not able to continue due to the time limitation or just ignorance,
which they couldnít overcome.
Just donít overestimate yourself. Face the reality. You will
need to learn a lot of stuff to make your OS dream come true. I started
programming at the age of 12 in Basic. When I was 13, I learned the Z80 CPU
assembly language. I got an x86 PC when I was around 15. I had been programming
in Pascal and learning x86 CPU and x86 PC hardware until I turned 19. Then with
newer (faster) PC and newer interests, I started learning C and doing some OS
dev related stuff. Iím now 26 and Iíve been doing all that OS dev stuff for 7
years now, whenever studies and work allowed to. Meanwhile, Iíve begun paying
attention to the fundamental algorithms and performance because itís important
for the code to be not just correct but also efficient to a certain degree.
What else? I havenít mentioned my background. A few words about itÖ At school I
was attending a math intensive class. I studied physics at university, so itís
again quite some math, not to mention the importance of physics and
engineering. Later graduate studies at an ECE dept helped me with computer
science even more. Itís a good background. Math, physics and engineering wonít
do any harm, quite the contrary. But all in all, Iíve been programming for more
than half my life now (14+ years) and I havenít made an OS. I must be having a
lot of knowledge and experience but I still donít have an OS. Why? :) Because
itís not easy to make one, because itís time consuming and because I donít
wanna end up with some crap that can hardly be referred to as an OS. That
simple. And so far, it all mostly has been just for fun and education :)
However, recently, it has begun to help me at work. If anyone is still
questioning my occupation, Iím a full-time software engineer.
What an OS is and what itís not
Well, an OS is a piece of systems software, whose primary
purpose is providing user applications with easy ways of communicating with
various computer devices and each other. The ease is achieved through hiding
all the hardware specifics from the user applications inside the OS (normally,
inside the hardware device drivers) and providing a unified API to the
applications for accessing the devices. Nowadays, we have millions of different
kinds of computer devices. It is impractical (if not impossible) to incorporate
support for each and every device into the user applications, which should run
on millions different computers with different hardware and performance. And
this task has naturally moved from application developers to the OS developers
and device developers and made user application development easy (well, easier
than in the times, when the computers had no operating systems and thus
software engineers had to write hardware specific code for every computer on
which the application was meant to run). These days an application can run
without any changes on practically any computer. This is a good thing.
Now, when someone is saying that their OS boots up, enters
system/kernel/supervisor/privileged mode (the protected mode on i80386+) and
handles display and keyboard I/O and then this person asks for help with
something, all I wanna do is laugh. It is just ridiculous for some ďhello
worldĒ program to be referred to as an OS. It has none of the functionality
that an OS must naturally provide to user applications. Null, void, zilch. Yes,
it can be very fascinating for a newbie to watch their program boots up and
prints something to the screen. But whereís an OS? That program is nowhere near
to an OS and it wonít become much closer to such in the near future, say, in a
few weeks or months. One could argue that there may be small OSes with little
of functionality and blah, blah, blah. Fair enough. Make something like MSDOS,
which is, basically, just a file system, a memory manager, a few I/O drivers
and a collection of programs/tools/utilities, most of which are used rarely or
never used at all. OK, make it like MSDOS, if itís what you want. Make it have
a file system and memory manager. Thatíll be enough for a tiny OS. Yet, many of
the coders concentrate on anything but the things that will enable them to have
at least these key features. They hurry up making bootable sectors,
bootloaders, etc etc. All right, Iíve done this all boot stuff as well. But I
wouldnít give an OS name to a bootsector. It is just not it.
Where to Start and Where not to
Many coders rush into coding something. Like it has already
been mentioned, many start with the bootstrap process and protected mode. Itís
of no surprise to hear that they donít know what to do next. And many of those,
who havenít done anything yet, donít know where to start at all.
All right, starting with some hardware specific thing is
good only if this specific thing is gonna be used. Itís also good if one has
never done anything similar before. No other good reasons exist for starting
with hardware specific stuff. It is just a detail, which will be different on a
different computer, especially on a totally different hardware platform. This
detail should be hidden inside the driver and not shown to the entire outer
Letís put hardware and drivers aside for the moment. Then
what remains? Surprisingly, lots of important things. If we consider any OS
function, then weíll see that thereís some idea behind it, some algorithm.
Weíll see that there should be certain structure and behavior of the system and
its smaller parts.
So, where to start:
the OS one will need to manage various kinds of data in different ways.
Here come lists, trees, FIFOs, stacks, caches, hash tables, etc etc. Find
out where youíll need them. Thereíre many places, so why not start with
part of the OS needs memory, for code, for stack, for static data and
heap. Why not write a general-purpose memory manager, which youíll extend
later if need be? The memory manager can be entirely developed under any
OS (DOS, Windows, Linux, whatever). Just allocate some memory by standard
means (say, Cís malloc()) and manage this allocated memory by your own
you want a file system in the OS, you can design and implement one. And
you donít need to perform low level (sector-based) disk I/O for that.
Emulate the disk with your file system inside a file, under a working OS
(e.g. DOS, Windows, Linux, whatever). Donít try to make it work under your
incomplete, unusable, and buggy OS.
you want a console/terminal, window manager and GUI, Iím pretty much sure,
itís also possible to develop things using the existing display I/O
functionality of the host OS (e.g. VGA and banked VESA modes can be used
under DOS/Windows, and text I/O can be done under any OS). And donít
forget that you can always create a file, to which you can save a picture
instead of drawing it on the screen. If you store it in some standard
graphics file format (.PCX, .BMP, .GIF, etc), it will be easy to view it.
Itís probably not all, but by now you should be able to see
the picture, Iím trying to render.
Hints and Recommendations that May Prove Useful
test, debug and bugfix as much code outside the OS being written as
possible. Make this code working under the OS in which you normally work
on your PC (DOS, Windows, Linux, whatever). It will make your life much
every module you do, make at least a small test program. Run tests to
ensure correct work of your code.
big pieces of code and big tasks/problems into smaller ones. I mean it!
Attack them separately from each other. First, make these smaller parts
work. Then put everything together and correct any remaining
interoperability problems. If you do not do so, youíll fail because of
being unable to locate a bug, isolate the erroneous portion of the code
from the rest.
the code being developed is still big and complicated by numerous
condition checks (if-then-elses, switches, etc), make use of the state machine
concept. It often helps. It will help you to do practically any kind of
parsing and implement various data/control protocols.
the code, check error conditions, donít let the errors accumulate and
propagate further than allowed, catch them and report of them by say
logging. If you donít do it, your system will most likely have hidden
holes bigger and severer than in famous commercial OSes.
underestimate the power of such tools as a pen(cil) and paper. Donít rush
into coding something complicated. First, analyze it on paper. Draw
pictures, diagrams, solve equations (youíll need surely to solve equations
in OS dev), etc. Make the solution to your complicated problem clear with
all the illustrative means you can have. Keep these papers along with the
other project documentation. If you have an idea, you can always implement
it in code and have a product. If you only have some unfamiliar/forgotten,
unreadable and probably incorrect code, you wonít make a product out of
it. Keep the solutions. You can even photograph your papers and put it
under source control! :)
and comment your code to make it possible to maintain in future by you and
others. Even the primary developer can forget what and why (s)he did in
the code. Donít make it happen. Document the code by outlining the
algorithm behind it and keeping this document along with the code.
documentation (on hardware, on algorithms, on everything) carefully.
Reread it, if you donít get it. Really do it. Donít give up. Try finding
docs that explain the same things but easier. Discuss things on the
Internet. But donít expect anybody doing your work (especially homework
assignment :) as nobody is obligated to help you with everything and in
some version control software (like CVS, MS Visual Source Safe, etc) to
keep track of the previous versions and development history. This will
help you to roll back or find a new bug that wasnít there in some of the
previous versions of the code.
PC emulators (such as Bochs, VMWare, Virtual PC, etc) to avoid unnecessary
hard disk data damages and simplify testing (if you run your code on a
real PC, the PC is hardly useful when itís busy running your code, whereas
with emulated PC you can do some additional work in parallel to booting
and running your OS).
use of the map files generated by the linker. It will help you to see
incorrect code/data placement, wrong addresses, right addresses, function
addresses, etc. It will also help you to find a place of exception inside
your OS by the address of offending CPU instruction.
use of disassemblers to check that the addresses used in the instructions
(in the OS image file) are OK. Many newbies have problems with addresses
when setting up protected mode. Iím happily using HIEW and BIEW as file
viewers with built-in hex editor and x86 disassembler.
burn the bridges! E.g. donít put the code that switches the x86 CPU to the
protected mode inside a small boot sector. Donít or you will loose all
disk I/O and wonít be able to load the OS or kernel (disk I/O isnít
possible in the x86 protected mode, this mode is not supported by the BIOS
that offers disk and other I/O functions). The same applies to the file
system. If you donít want to waste too much time, donít reinvent the
wheel. Make a driver for the existing commercial file system (like MS
FAT12/16/32) and avoid all problems of transferring data to and from your
Thereís rationale for each and every item on the above list.
Try to understand it. It all will only help you to get done.
Just donít get me wrong. Iím not against anyone trying to
make their own OS. Nope. Iím simply trying to point out a few key moments that
one should think of when doing it. If all what Iíve been talking here sounds
gibberish, then it simply means that youíve not understood a bit of it. This is
not my fault, maybe not even entirely yours, but it simply means youíre not
ready or you misread something. Reread the thing. Give it some thought. Let
things settle down a bit in the head and then return to it. If it still doesnít
help, as usual, you have two options: quitting and learning. First is simple,
though you may not have a will to quit and be a loser. Second is tough, but the
more you learn, the closer you are to the success and the more chances that
this my article will be understood in the future :)
December the 13th,
Alexei A. Frounze