Many years ago I was working at a firm that investigated stock pump and dump schemes on Yahoo Finance message boards.
We had to scrape the boards and then analyze messages to see if we could identify the identity of the people running the schemes.
This was 2002 and I wasn't aware that the LWP::Simple existed in Perl (aka the requests library in Python).
I ended up using basic TCP socket libraries to connect to port 80 and do http requests. It was, to put it mildly, a major pain in the ass.
That being said, I learned a TON about how http, tcp, html, etc all work together. 20+ years later, I still use some of that knowledge when analyzing network protocols at work.
I mention this b/c nothing is stopping people from doing similar projects now. e.g. Linux from scratch [0] is one great example of learning fundamentals even though we now have 1 click "launch me a Linux VPS" options.
Joel Spolsky make this point too. Even if you are using Java, it's still helpful to understand how CPUs interact with cache because the highest performance comes from optimizing the whole stack.
0 - https://www.linuxfromscratch.org/
We had to scrape the boards and then analyze messages to see if we could identify the identity of the people running the schemes.
This was 2002 and I wasn't aware that the LWP::Simple existed in Perl (aka the requests library in Python).
I ended up using basic TCP socket libraries to connect to port 80 and do http requests. It was, to put it mildly, a major pain in the ass.
That being said, I learned a TON about how http, tcp, html, etc all work together. 20+ years later, I still use some of that knowledge when analyzing network protocols at work.
I mention this b/c nothing is stopping people from doing similar projects now. e.g. Linux from scratch [0] is one great example of learning fundamentals even though we now have 1 click "launch me a Linux VPS" options.
Joel Spolsky make this point too. Even if you are using Java, it's still helpful to understand how CPUs interact with cache because the highest performance comes from optimizing the whole stack.
0 - https://www.linuxfromscratch.org/