Back in 2006 I wanted to learn more about how search engines worked, so I started porting Lucene to Common Lisp. Actually, I wrote a Common Lisp port of Ferret. Ferret is a Ruby port of Lucene. Lucene is sort of Doug Cutting's Java version of Text Database (TDB), which he and Jan Pedersen developed at Xerox PARC, and which, to complete the circle, was written in Common Lisp.
I didn't know Ruby, and I didn't know search engines, but I did know Common Lisp. It took me 7 months to create a binary-compatible, pretty dang functional port of Lucene. I called it Montezuma[1], and it's still actually used by people.
About half the time was spent implementing the text analyzer, document store, and indices. The remaining 50% was spent implementing search (and parsing the query language). It was a very rewarding experience--It was one of the largest projects I'd worked on, mostly solo (I got some help near the end), in an area I knew nothing about, working in an extremely test-driven fashion (over 2000 unit tests when I was done).
I did not, however, learn as much about indexing and search as I expected. I learned a lot, yes, but quite a bit of the Ruby code was so easy to translate to Common Lisp that I didn't have to understand everything in order to make it work.
I still recommend writing a small search engine as an interesting exercise, though.
1. https://github.com/sharplispers/montezuma
I didn't know Ruby, and I didn't know search engines, but I did know Common Lisp. It took me 7 months to create a binary-compatible, pretty dang functional port of Lucene. I called it Montezuma[1], and it's still actually used by people.
About half the time was spent implementing the text analyzer, document store, and indices. The remaining 50% was spent implementing search (and parsing the query language). It was a very rewarding experience--It was one of the largest projects I'd worked on, mostly solo (I got some help near the end), in an area I knew nothing about, working in an extremely test-driven fashion (over 2000 unit tests when I was done).
I did not, however, learn as much about indexing and search as I expected. I learned a lot, yes, but quite a bit of the Ruby code was so easy to translate to Common Lisp that I didn't have to understand everything in order to make it work.
I still recommend writing a small search engine as an interesting exercise, though.
1. https://github.com/sharplispers/montezuma