How can I build a small operating system on an old desktop computer?

This might be in vain, as I know writing an operating system is unbearably complicated (especially by oneself).

• I don't expect to build the next linux, or windows.

• I know it will be horrible, and buggy, and won't work, but that's fine.

I want to write everything myself, in Assembly, C, and (some) C++.

This is a future project, as I'm busy with some other things at the moment and don't have the time immediately, but I figured I would ask it now, so maybe I could get lots of answers to this, and it could build up and be a useful resource for this kind of approach (everything else I have seen involved building off of minix, using an existing bootloader, building it in a virtual booting program thing, etc).

I want to set up one of my older desktops with a monitor, keyboard and mouse, and start working on a blank hard drive.

I want to learn how to write my own bootloader (I've found lots of resources about this, but for completeness, please still add some good ones), my own USB driver (if that's necessary), a CD driver (if that's necessary), etc. Everything, from the ground up.

• How do I put the code onto the computer? Is it best to do it with a floppy disk? Can most computers do it from a USB stick?

• What drivers do I need, and can you suggest any references to building those?

• After the booting sequence--then what? How do I get into protected mode etc.

• How do I manage memory without the help of an operating system? Do I just use whatever addresses I want? No initialization necessary?

• What will I undoubtedly run into that will confuse me?

• How can I make it either a command line O/S, and a graphical one?

• What is a graphical O/S built on? Like, how would I do something like, a command line, with a font, and a picture at the top?

• Where can I read about setting up a multitasking environment? (ie., having two graphical-like command lines running side-by-side).

• How would I set up a sort of windowing system? How do I display graphics on the screen once simple multitasking is set up?

Believe me, I understand that this is a very complicated project, and I probably will never get around to completing it or writing anything on it of any use.

There are lots of other pieces to this I haven't mentioned, if you think of any, feel free to add those too.

Please put one "topic" per answer--for example, USB drivers, and then maybe a list of resources, things to look out for, etc.

Also, please don't suggest building off of another O/S or pre-existing code. I know I will read a lot of pre-existing code (such as the linux kernel, or example resources, existing drivers, etc) but ultimately I want to do all the writing myself. I know I should build off of something else, and there are lots of other questions on SO about that that I can read if I change my mind and go that route. But this one is all about doing the whole thing from scratch.

Any suggestions on how to get it graphical? Different video modes and how to work with them, etc?

First things first. Read, read, read, read, read. You need to have a firm understanding of how the OS works before you can hope to implement your own.

Grab one of Andrew Tanenbaum's books on operating systems. This is the one we used in my OS class in college:

Modern Operating Systems PDF

Modern Operating Systems on Amazon

Despite the ridiculous cover, it's a fantastic read, especially for a textbook. Tanenbaum is really an expert in this area and his explanations of how the OS works underneath the hood are clear and easy to understand. This book is mostly theory, but I believe he also has a book that discusses more of the implementation. I've never read it, though, so I can't comment on it.

That should help you bone up on process management, memory management, filesystems, and everything else your OS kernel needs to do to get it up to a bootable state. From that point on it's basically a matter of writing device drivers for the hardware you need to support, and offering implementations of the C library functions to make kernel calls for things like opening files and devices, reading and writing, passing messages between processes, etc.

Read up on x86 assembly (assuming you are designing this for an x86 machine). That should answer a lot of your questions with regards to moving between processor operating modes.

If you've got any electronics knowledge, it may be easier to start with writing an operating system for an embedded device that has ample documentation, because it will generally be simpler than an x86 PC. I've always wanted to write my own OS as well, and I'm starting with writing a microkernel embedded OS for This Development Board from Digilent. It can run the soft-core MicroBlaze processor from Xilinx, which has very thorough documentation. It's also got some RAM, flash data storage, LEDs, switches, buttons, VGA output, etc. Plenty of stuff to play around with writing simple drivers for.

One of the benefits of an embedded device is also that you may be able to avoid writing a VGA driver for a long time. In my case, the Digilent development board has an onboard UART, so I can effectively use the serial output as my console to get the whole thing up and booting to a command line with minimal fuss.

Just make sure that whatever you choose to target has a readily available and well-tested compiler for it. You do not want to be writing an OS and a compiler at the same time.

http://www.osdev.org/ and http://www.osdever.net/

welcome to the OS Development world.

See also other x86 documentation links in SO's [tag:x86] tag wiki: Intel and AMD manuals, compiler / assembler docs, and various guides.

It also suggests using BOCHS or other virtual environment for debugging, since you can single-step your bootloader and examine registers.

I would suggest working, at least at first, on Bochs or some other virtual machine the reason being that you can take it with you wherever you want, it's easier to debug (you can see the exact state of the hardware), and if you need outside help debugging they can use the exact same 'hardware' as you.

The most useful advice I have is to get yourself into a state where you can be running C code as quickly as possible -- i.e. boot up, setup your descriptor tables, and get yourself to a point where it's safe to run compiled C. Most if not all the kernel should be in C if you want to stay sane and keep working on it. Assembly, while required in some places, is tedious and tends to be hard to debug.

At its lowest level the minimum that an operating system needs to be able to do is to drive a system's hardware in some way and somehow load an execute some sort of "user code." If you're going to start with a PC then you need to write code that can be loaded by it from some device or another. Older PCs have a BIOS in firmware which determines how the hardware performs some initialization (at least video, keyboard, and some form of storage or boot loader). (Update October 2017: Newer PCs have EFI or UEFI firmware ... which is largely a pedantic difference; the serve the same purposes for this discussion).

So start by learning the low level details of how to use the BIOS or other firmware, on your target system. That is, learn how to write a program that the BIOS can load and execute. That will eventually morph into your boot loader. Start small. Just get a program that prints: "Hello, Linus" directly from the firmware boot process (on a floppy, or USB thumb drive, would be a good start ... or on a hard drive if you like).

From there I'd recommend writing a very simple serial driver ... update your boot loader to initialize some serial port, and start a download there from. Then it can execute the code it pulls across. From there write a bit of bootstrap that can write to another set of blocks (we haven't implemented a file system yet ... not even partition table parsing; so we'd just deal with raw ranges of blocks on the disk at first).

At that point your boot loader should be able to pull new code across the serial line, dump it into a partition (yes, implement partition table handling of some sort ... whether it conforms to standard PC conventions is up to you at this point), and execute it.

From there you should be able to work on far more sophisticated features. From this base you can write and compile a new "kernel" ... reboot your testbed, and have the new kernel deployed to it.

(Your bootloader should take some signal, such as a BREAK over the serial handshaking lines as a command to skip the download and just boot the existing image; and it should handle some timeout in this way as well).

From there write a very simple terminal layer and command shell? A filesystem? Implement commands to download new executable content other than the kernel (files or objects of some sort). And so on.

Naturally you could have started with a console driver using the PC keyboard and video (the BIOS INT 0x16h and INT 0x10H stuff, respectively, if I recall correctly). However, I'd suggest starting with a serial driver since you can then automate your build/deploy/test cycle from any other existing (functional) system. Since your new OS will start as a cross-compiled project it's going to be essential for you to have a streamlined way of handling that.

I don't know how far you want to take your project. A reasonably impressive goal would be to achieve "self hosting." If you can create a simple assembler/compiler that can allow you to use your new OS to (re-)build, link, and boot into a working version of your new OS ... then you've achieved that goal. (Note that's not a requirement. Many embedded systems are never going to be self-hosting and there's nothing wrong with that).

If you don't mind using hardware virtualisation, there is a course (book + lectures + software) that will take you 'From Nand to Tetris'. You create a full computer system entirely yourself from the (for these purposes atomic, and given) electrical NAND gate, right through to building the OS, a language, and finally coding a simple game on your personal machine.

I think it's a great idea and something I fully intend to get stuck into soon. The book is surprisingly cheap and I believe the course is taught at MIT. I can imagine no greater feeling than having the full, complete knowledge of an entire system you built yourself from the ground up.

Link: http://www1.idc.ac.il/tecs/

I would start small and purchase an 8086 embedded development kit, and develop a multitasking OS on that. Once you have a kernel and are acquainted with working at the hardware level, you will be ready to do something more challenging.

Building even a VGA display DOS clone is a fairly challenging thing. The details are enormous. :-)

specific topics.

How do I put the code onto the computer? Is it best to do it with a floppy disk? Can most computers do it from a USB stick?

The BIOS will do elementary bootstrapping.

---

What drivers do I need, and can you suggest any references to building those?

anything that isn't direct cpu/memory operations. Anything that isn't directly in the CPU reference manual.

---

After the booting sequence--then what? How do I get into protected mode etc.

Protected mode will be part of the boot sequence.

then you start multitasking and figuring out how to start processes.

---

How do I manage memory without the help of an operating system? Do I just use whatever addresses I want? No initialization necessary?

Correct. You will probably want to sort out a virtual memory system eventually.

---

What will I undoubtedly run into that will confuse me?

not having debugging tools, not having IO

---

How can I make it either a command line O/S, and a graphical one?

With grief. Look up Windows 3.1 and Linux, in particular X windows.

---

What is a graphical O/S built on? Like, how would I do something like, a command line, with a font, and a picture at the top?

Look up X windows.

---

final advice: study linux/x windows. It's not perfect, but it provides an understanding of one approach. Also study embedded systems.

I see plenty of good references to OS development sites, so I'll describe a different approach:

If you want the experience of implementing an OS from bare metal, there are way better hardware choices than an old PC. With the PC architecture, you will spend an inordinate amount of your time coding around uninteresting artifacts of its 30 year design history. For example, just the bootloader part of the project has probably burned out many a brave programmer.

For example, you'll need one set of drivers to read your kernel off of disk and/or the network. Then you'll need code to get into protected mode. At that point, you need another set of drivers! Very little of the work you do to get the chip into protected mode will be transferable after that point. You want to run it on a different PC +- 4 years and you'll need yet another set of drivers.

Look into bootstrapping an ARM or other 32-bit "embedded" chip. Inexpensive development boards are available, or you can solder your own! Some have built-in ethernet and usb. I think you will have more fun working on a sane, non-crusty architecture, and perhaps end up with some re-usable skills.

More than anything else, if you want this to run on real hardware, you absolutely need a copy of your processor's manual. The Intel manuals (http://www.intel.com/products/processor/manuals/) are invaluable. They go over everything from switching modes (real/protected) to virtual memory management (if you choose to go that far) to making syscalls (if you ever get to doing user mode). Most importantly they explain in great detail a number of things that must be set up for things to function, like the TSS and segment registers, that most OS texts don't discuss because they're more concerned with higher-level concepts than processor-specific details.

The OS Development Series @ BrokenThorn might interest you.

Try to read the code of a small, basic open-source OS, such as MikeOS.

Alternatively, I suggest the following steps (should be fun!):

1. Write a virtual machine. Define all your processor instructions so that you know your system inside and out. Interface it with SDL for keyboard, mouse, screen, audio access. Keep it simple so that you can fit everything in your head at once. It doesn't need to be a state-of-the-art virtual machine, just one that can emulate the function of a 'real' computer.
2. Write an assembler for your virtual machine's processor. Remember that this assembler need not be a program written in the virtual machine's language, but anything that can convert your assembly language to machine code.
3. Define an executable format, and write a simple linker.
4. You've got all the parts to write your operating system now! Write it in the assembly language, assemble it ... etc .. you don't need such a complicated bootloading process, just get your machine to run your OS first.

The above steps may seem kinda stupid for writing a simple OS, but hey, it's damn fun.

check out MikeOS. Its a fairly simple OS written is readable (as in commented) assembly. Even though its fairly simple, it does have a GUI and supports some networking and multimedia.

edit: MenuetOS is graphical. Its also written is straight asm, but its more sophisticated than MikeOS

You have an ambitious goal. But execution is key.

Most of the structured approaches (textbook or college class) will walk you through the process, but they supply a lot of the nitty-gritty code that glosses over the arcane details of your chosen platform and lets you focus on the big-picture ideas: process scheduling, memory management, deadlock prevention, I/O, and so forth.

My advice is this: cut way back on your expectations and start with a basic question.

What is an operating system?

A computer scientist will (hopefully) never say that an OS is a graphical user interface, or a web browser, or a way to hook up USB devices, or anything that a user can actually see or touch. Instead, an OS at its most fundamental level are those things I mentioned above. They all fall under one big umbrella: resource management.

An operating system is nothing more than a program that manages the hardware resources of the computer: memory, CPU, and peripherals.

Here's a simple operating system: a program lets the user type in a program (in hexadecimal or binary) using a serial connection. Once the program has been typed in, it runs the program. When the program is finished, control is returned to the user where they can either run the program again or type in a new one.

Do this on a "clean" architecture such as an embedded ARM processor with 64K of memory or so. You can code this up in assembly after a few days of learning the ins and outs of the ARM. And voila!, you've got yourself an operating system.

It does everything an OS is supposed to do:

• It manages memory by not letting the user overwrite the OS itself.
• It schedules a single process to run.
• It handles the I/O to the single serial peripheral.

This gives you a building block to start from. You have many choices now. Perhaps one of them would be to allow two programs to be loaded into memory and let the user decide which one to run next.

Or you could let the user suspend execution of one program, switch to the other, suspend, and switch back. This is rudimentary multitasking, even though it's entirely manual.

Your choices are unlimited but each one is a baby step from what you had before.

It's fun if you don't set your sights too high!

I maintain a list of resources on StackOverflow: How to get started in operating system development. Please add new resources to it as you begin you're adventure (I'm about to begin :) )

Many schools have OS classes that do much of what you describe. My school (CMU) taught OS in C, and we wrote a kernel, file system, and shell, and were provided with code for a boot loader.

Unfortunately, I wasn't able to find any definitive resource for this course (15-412) on the web, and it's evolved over time. But perhaps people could post links to sources and assignments for schools that do have good resources on the web.

You might enjoy this tutorial named "Roll your own toy UNIX-clone OS", its very insightful and should help you on your way.

Good luck.

Study the A2 System (formerly called the Oberon system) for ideas you can steal. It is a graphical OS built by just two people, although admittedly one is Niklaus Wirth. First released around 1990 and the speed is astonishingly good. There is a book by Gutknecht.

You might want to consider taking a look at Operating Systems Concepts, by Abraham Silberschatz - Understanding Systems Programming concepts is a fundamental requirement, take a peek into F/OSS kernel internals of OSes linux *BSD and GNU/Linux, especially earlier versions, which are probably a bit more documented.

About low-level graphics programming, this will give you a lot of information: http://www.osdever.net/FreeVGA/home.htm. (It's very interesting for text mode as well.)

>What will I undoubtedly run into that will confuse me?

You will realize that on PC there's an awful lot of stuff that is unclear: the x86 instruction set itself is complicated, and when you get direct access to hardware, it can take quite a while before you understand how to write a single character on screen.

Don't worry about floppy disk and such, most of the time you will use an emulator such as Bochs or QEmu.

Take a look at Minix. Study the source code along with "Operating Systems Design and Implementation". Consider making contributions to the project. I think Minix is a really good and promising OS in the making. It is also well funded project. That means, you might even get paid for your contributions!

It is very easy thanks to the fact that the BIOS already have many input/output functions built into it for changing screen mode, changing pixel color, writing text to the screen and many other things. However, it does not include support for file systems, that is one of very few things you must incorporate in your OS.

The BIOS loads the first sector on the hard drive and starts executing the code from there, so your OS must be placed with the first instruction in the first sector on the hard drive.

This Wikipedia article should get you started on BIOS interrupts for writing text to the screen, receiving keys from the keyboard and other stuff. https://en.wikipedia.org/wiki/BIOS_interrupt_call

Even if you plan to use c++ I would strongly recommend reading up on assembly programming, as it is vital to understanding how the hardware works.