silvajoao
Joined 18 years ago
Joined 18 years ago
Very cool - writing a browser has been on my own project todo list for going on 10 years now; like you I also write everything myself with few exceptions.
Maybe you'd like to follow a book building a browser? :-)One of the ideas is to allow readers to build along in their own programming language, or using AI. The main material is more about the building blocks and what needs to be done; the C coding can be skipped.
Text rendering, but really layout/shaping, especially for complex scripts like Arabic.
There are lots of problems involved: parsing TTF, rasterizing the glyph shapes, line layout in CSS, avoiding floats, multiple styles (line-height, vertical-align, decorations, etc), just to make something quite simple.I'm not doing text shaping, so Arabic will look strange. Also no RTL like Arabic and Hebrew, and no support for CJK scripts yet (that should be easier, using the Google Noto font. Needs a huge builtin map from codepoints to which fonts have those glyphs; each TTF file has at most just 2^16 glyphs, which isn't enough.)
JavaScript, which unfortunately seems like a necessity for the modern age.
Indeed, it's a pity that React sites won't work at all without Javascript. Doing a Javascript interpreter is very doable; I strongly recommend Crafting Interpreters as an excellent book that builds an interpreter for a language that is very similar to Javacript.But the bigger piece is doing all the DOM bindings and web APIs.
The HN <script> payload is very easy to handle though: https://news.ycombinator.com/hn.js?FmJw8tvF5GVURtPJ592v
The very relaxed parsing of HTML/CSS. It's surprising just what browsers will tolerate in terms of malformed syntax yet still get a functional website out of it.
You'd be surprised! The HTML5 spec is actually very comprehensive about how to handle errors cases as well. Take a look:https://html.spec.whatwg.org/multipage/parsing.html#parsing-...
I'm not handling all of the edge cases, only the standard <!DOCTYPE html> stuff i.e. I don't handle quirks in any special way. I guess most of those quirks are meant to handle older web pages and the way they rendered in IE back then.
My goal was always going to be to get Wikipedia working also.
That's a great goal, and it helps a lot to scope down what needs to be built.Here's a screenshot of Wikipedia so far: https://imgur.com/a/vG0U3in
(small hack in that screenshot: the flex and grid layouts are getting replaced with blocks instead)
I'm thinking of doing an Advent of Code series for next year, to build a very minimal browser just for Wikipedia but with lots of shortcuts :-) For example, using a local HTTP proxy to avoid having to handle TLS, which is quite a big of work to get working.
After having a PNG decoder, and a BMP and ICO decoder to support favicons in tabs, only then did I find out that this resource is an SVG :-) An SVG renderer is quite a piece of work...
Yet another interesting detail: the Javascript is quite simpler, it seems to be mostly about submitting votes and collapsing comments. A small Javascript interpreter and just a few DOM hooks would be enough to support comment collapse toggling. Maybe that will come later :-)
I left the job 2 years ago to focus on many personal pursuits, including this one.
I still don't know whether to continue after opensourcing, it will depend a lot on reception and whether there's any funding or support. We'll see!
Now, when you say no libraries, exactly how far does that go?
No libraries at all like stb_image, openssl, libcurl, etc. But it does use the platform APIs to create a window, get keyboard and mouse events, create TCP sockets, etc. It uses some libc calls like printf and malloc, but then it has its own memory allocators and arenas for DOM objects, CSS rules, and other common objects.On Windows it just targets Win32.
On macOS it uses a bit of Objective-C to create an NSWindow and for the main message loop.
On Linux it uses libxcb to connect to X11. I planned on using Xlib originally, but libxcb is closer to the protocol and IMO cleaner. After using it I also got a sense that the X11 protocol is pretty straightforward, so I might just replace libxcb with a custom implementation.
For example, I need to query DBus for things like the desktop theme preference (dark/light) but didn't want to link build the DBus protocol messages from XML in the build, so it just queries libdbus directly and parses all the messaging. Maybe it should have a custom DBus client too...
(Linux doesn't have a "native" UI, so I made this exception on that platform to be able to make windows, etc as in the other platforms)
---
There are lots of reasons to "do it yourself" (and lots of good reasons NOT to do it too -- I wouldn't do this professionally!)
My goal here was to make a presentable and polished product, so I spent some time on code quality, correctness, performance, etc. and in presenting a nice browser that feels familiar. Here's a screenshot: https://imgur.com/a/ojWvciR
More than anything, I really like being in the flow and feeling the joy of programming -- building something step by step and seeing the results incrementally. A browser is quite a nice project for that, since you have the whole web to interact with and lots to implement.
No Javascript :-) React sites are completely blank, like instagram.com.
No GL/DirectX/Vulkan. It's all software rendered and it just blits a pixel array to screen on every frame. Highly portable though!
Font rendering is antialiased but it's not hinted, and small glyphs at low DPI look start looking blurry.
No Wayland support, only X11 because it's the most widely available. I might do a Wayland port later.
All in one process, one thread per tab. No process isolation (also something to do later)
Only HTTP/1.1 and TLS 1.2, no HTTP2 or later. No IPv6, only IPv4.
No SVG, no <canvas>, no form controls (buttons, etc.), no cookies support (for now...)
The CSS support however is bigger than CSS 2.2: there's var(), --custom-properties, @media queries, etc. and I'm planning to do flex and grid layout too (because Wikipedia uses that a lot).
It's very encouraging to see that there was a positive reception on HN, this is the first time that I talk about it publicly. Really enjoying the extra motivation to getting this released now :-)
TLS is always version 1.2, and only the mandatory cypher suites are supported: TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256 and TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256.
This requires SHA256, AES and two elliptic curves (x25519 and secp256r1).
AES is used in GCM mode, so that's implemented too.
However, the browser doesn't verify the server signature over its certificate. So ECDSA and RSA are not implemented :-)
Getting TLS to work was indeed one of the more frustrating parts. The cryptographic functions are very well documented and there are lots of good test vectors, so that wasn't too hard (just a lot of work). But getting TLS working was very frustrating -- the handshake would complete, I'd switch to encrypting mode, and... the server would just disconnect my client instantly, with no further feedback or a stacktrace or anything.
I ended up debugging that by building openssl with LOTS of printfs all over the place and fixed nonces, talking to a local HTTPS server, and tried to make my client emit exactly the same bytes until I found the issue -- memory corruption, of course :-)
I built on amazing tutorials available online. For example, elliptic curve cryptography and x25519 in particular are SUPER well explained at [1].
JPEG was fun too. Tutorials like [2] gave a great overview of how it works, and the spec [3] is super clear (but includes TONS of optional stuff that can be skipped)
PNG was the easiest, the RFC is very straightforward [4].
(I actually did the PNG decoder before doing fonts, because the TTF format is so crazy that I always decided to using bitmaps fonts loaded via PNGs instead)
It's super satisfying to build ~300 sources in ~2 seconds from scratch using just MSVC, Xcode or Make, and then launching Wikipedia -- no libraries, and it works :-)
[1] https://www.cl.cam.ac.uk/teaching/2122/Crypto/curve25519.pdf
[2] https://www.youtube.com/watch?v=0me3guauqOU
I love how John Carmack put it: [1]
You can't know everything, but you should convince yourself that you can know anything.
The Handmade Hero series [2] channels this feeling really well, and I wanted to convey it again by building a browser from scratch but with a limited scope (Wikipedia, HN; no Javascript, but also no libraries and no AI).I also loved tutorials in the 90s like "build your own OS", "build your own bootloader", "build your own C compiler", etc; build your own printf(), your own malloc, etc. Learned a lot this way!
might I ask how you load images? Are you using libraries, or did you write any decoders yourself?
No libraries means no libraries! :-) The decoders are written by hand, following the specs. They don't support every single configuration (e.g. JPEG has TONS of variants), but good enough for the media in Wikipedia.This browser is a learning experience and it's definitely NOT secure (e.g. TLS "works" but it doesn't verify server certificates at all...). The decoders use a set of shared utilities for decoding big-endian/little-endian, 16 bits, 32 bits, etc and checking for overflows (using setjmp/longjmp for convenience). I think it's pretty good, and I also bet that Claude would find many problems. I'm prioritizing (1) code that works, then (2) code that's performant, and then (3) code that's clear.
I'm ambivalent about throwing AI at this codebase. It would definitely find issues and improve it, but I also want to keep the spirit of "Do It Yourself" and the feeling of "We built this and it works".
[1] Searched the quote from memory, Google found it here: https://transcript.baribarilab.com/video/HOv_dWDgIRM/
I'm currently planning to opensource early next year and I will definitely post to HN.
(The original name for this project was "Hacker's Browser", as in, an HN browser :-) But then I figured that Wikipedia worked pretty well and grew in scope from there...)
I'll also be looking for early testers soon, to verify that this thing builds on various platforms before the public release. If you're open to try this early on then see my profile to get in contact.
Just looked at a few more metrics:
v1 was 148 commits over about a year; gave up when it came to TTF font parsing and rendering.
v2 was 1075 commits over 3 months and in C++, gave up after doing sockets but realizing that TLS and crypto was going to be required. I also strongly disliked the C++ codebase :-)
The current repo is v3 and back in C, started in Oct 2024 (2 years), and it's 4822 commits. About 600 .c and .h files, about 68k lines of code (14k blank, 7k comments).
I had a lot of previous experience with networking and systems programming, but not with rendering.
A huge inspiration was https://guide.handmadehero.org/ and Bob Nystrom's https://www.craftinginterpreters.com. I wanted to create a similar book and video series, but by building a browser.
This is finally approaching a usable state: navigates and renders Wikipedia, supports TLS, builds on Windows, macOS and Linux, loads PNGs and JPEGs, etc.
I'm going to opensource this once it supports most of CSS 2.2 layout and renders HN too (HN is totally blank right now, because it's a giant <table> which is still a TODO :-) )
I still don't know what to do with this. It's been a huge passion project and I'm very happy about it. I planned to create a book or video course about C programming, starting from printf("Hello world!\n") and ending up with a working browser; but the advent of AI coding agents in the past 2 years changed a lot the market for such a course.
I'm still finishing this due to stubbornness and some sunken cost fallacy effect :-)
Would love to hear feedback and ideas about such a project.
It's a great way to experiment with all the Gemini models that are also available via the API.
If you haven't yet, try also Live mode at https://aistudio.google.com/live.
You can have a live conversation with Gemini and have the model see the world via your phone camera (or see your desktop via screenshare on the web), and talk about it. It's quite a cool experience! It made me feel the joy of programming and using computers that I had had so many times before.
You can also use https://aistudio.google.com to use base models directly.
The idea was to have TCP sockets and Websockets to enable that.
Basically, have the same APIs you're familiar with in a Browser, but in a much smaller package that you can ship independently of the Browser.
(this is very similar to Electron)
Past work in Chrome, Google Search, Assistant, Google Lens, Google AI Studio.
Open sourced https://windowjs.org/
Contact is my username at gmail.com.