Show HN: MyraOS – My 32-bit operating system in C and ASM (Hack Club project)

225 points by dvirbt 21 hours ago

Hi HN, I’m Dvir, a young developer. Last year, I got rejected after a job interview because I lacked some CPU knowledge. After that, I decided to deepen my understanding in the low level world and learn how things work under the hood. I decided to try and create an OS in C and ASM as a way to broaden my knowledge in this area.

This took me on the most interesting ride, where I’ve learned about OS theory and low level programming on a whole new level. I’ve spent hours upon hours, blood and tears, reading different OS theory blogs, learning low level concepts, debugging, testing and working on this project.

I started by reading University books and online blogs, while also watching videos. Some sources that helped me out were OSDev Wiki (https://wiki.osdev.org/Expanded_Main_Page), OSTEP (https://pages.cs.wisc.edu/~remzi/OSTEP), open-source repositories like MellOS and LemonOS (more advanced), DoomGeneric, and some friends that have built an OS before.

This part was the longest, but also the easiest. I felt like I understood the theory, but still could not connect it into actual code. Sitting down and starting to code was difficult, but I knew that was the next step I needed to take! I began by working on the bootloader, which is optional since you can use a pre-made one (I switched to GRUB later), but implementing it was mainly for learning purposes and to warm up on ASM. These were my steps after that:

  1) I started implementing the VGA driver, which gave me the ability to display text.
  2) Interrupts - IDT, ISR, IRQ, which signal to the CPU that a certain event occurred and needs handling (such as faults, hardware connected device actions, etc).
  3) Keyboard driver, which enables me to display the same text I type on my keyboard.
  4) PMM (Physical memory management)
  5) Paging and virtual memory management
  6) RTC driver - clock addition (which was, in my opinion, optional)
  7) PIT driver - Ticks every certain amount of time, and also  
  8) FS (File System) and physical HDD drivers - for the HDD I chose PATA (HDD communication protocol) for simplicity (SATA is a newer but harder option as well).
     For the FS I chose EXT2 (The Second Extended FileSystem), which is a foundational linux FS structure introduced in 1993. This FS structure is not the simplest,
     but is very popular in hobby-OS, it is very supported, easy to set up and upgrade to newer EXT versions, it has a lot of materials online, compared to other
     options. This was probably the longest and largest feature I had worked on.
  9) Syscall support.
  10) Libc implementation.
  11) Processing and scheduling for multiprocessing.
  12) Here I also made a shell to test it all.

At first I just wanted to declare the OS as finished after completing the FS, and a bit of other drivers, but because of SoM my perspective was changed completely. Because of the competition, I started to think that I needed to ship a complete OS, with processing, GUI and the bare minimum ability to run Doom. I wanted to show the community in SoM how everything works.

Then I worked on it for another 2 months, after finishing the shell, just because of SoM!, totalling my project to almost 7 months of work. At this time I added full GUI support, with dirty rectangles and double buffering, I made a GUI mouse driver, and even made a full Doom port! things I would've never even thought about without participating in SoM.

This is my SoM project: https://summer.hackclub.com/projects/5191.

Every project has challenges, especially in such a low level project. I had to do a lot of debugging while working on this, and it is no easy task. I highly recommend using GDB which helped me debug so many of my problems, especially memory ones.

The first major challenge I encountered was during the coding of processes - I realized that a lot of my paging code was completely wrong, poorly tested, and had to be reworked. During this time I was already in the competition and it was difficult keeping up with devlogs and new features while fixing old problems in a code I wrote a few months ago.

Some more major problems occurred when trying to run Doom, and unlike the last problem, this was a disaster. I had random PFs and memory problems, one run could work while the next one wouldn’t, and the worst part is that it was only on the Doom, and not on processes I created myself. These issues took a lot of time to figure out. I began to question the Doom code itself, and even thought about giving up on the whole project.

After a lot of time spent debugging, I fixed the issues. It was a combination of scheduling issues, Libc issues and the Qemu not having enough (wrongfully assuming 128MB for the whole OS was enough).

Finally, I worked throughout all the difficulties, and shipped the project! In the end, the experience working on this project was amazing. I learned a lot, grew and improved as a developer, and I thank SoM for helping to increase my motivation and make the project memorable and unique like I never imagined it would be.

The repo is at https://github.com/dvir-biton/MyraOS. I’d love to discuss any aspect of this with you all in the comments!