1jreuben1

If someone has side-projects with paying customers, why would they be seeking employment ?

You never stop learning to code. Experimenting with LeetCode and ClaudeCode - revising the landscape of DS&A (again) enough to steer the which, when and what, and validate the how. Eg: I know what a SkipList does, when to use and fuzzily how it works. The hardest thing about leetcode: 1) knowing you will forget it 2) knowing you will never use it much 3) knowing you are forgoing time to learn other stuff 4) knowing that AI makes knowing it redundant

Spend a few years in big corp, and you will miss working in startups - the grass is always greener on the other side !

What is the impact on Software Architects ? AI Architecture guidance can be very ivory tower in regards to transformer model internals and ML pipelines, but when it comes to CodeGenAI refinement, you really need to touch the grass. On the plus side, system design skills means knowing what questions to ask, and good mermaidJs markdown designs are solid agent inputs. I guess the role will shift to researching and enabling the devs to best leverage CodeGenAI infra for full SDLC velocity with quality

From a technical perspective, I do not enjoy RAG prompt engineering. It is so brittle and non-formal, and the barriers of entry are extremely low.

There is a complete trifercation of ML: 1. ML Engineers: the high priests, with access to 10K GPU hours, designing novel Transformer architectures using Tensorflow / PyTorch / JAX. 2. Data Scientists: conducting SFT on pre-trained models via the HuggingFace APIs + MLOps & model optimization (eg via TensorRT). 3. GenAI devs: building LangChain orchestrations and RAG prompt flows using off the shelf LLMs commoditized behind APIs - no stats or linear algebra required.

Too many are jumping on this GenAI bandwagon, which will result in a massive hype-cycle trough of dissalusionment and potential VC AI winter.

Furthermore, GenAI is a local maxima on the path to true AGI. COT / REACT heuristics lack the integrated differential aproach of Hybrid AI, ignoring everything previous generations of researchers focused on: problem solving, planning, probabilistic logic, reasoning etc. For true AGI, we need some focus on: 1. concept representation. 2. goal formation. 3. code introspection and self-modification. GenAI is a big distraction from that kind of R&D.

A startup needs to move fast - quick and dirty prototypes in JS / Python turn into MVPs. dynamically typed, low barrier of entry, high velocity. Startups that survive to the point of needing to rewrite for stability and performance, will usually opt for Rust or Go. They often dont need the bloated enterprise-specific baggage that comes with C# / Java - transactional models, IoC, ORMs, complex inheritence models - YAGNI.

Learning is always worth it, but you dont need to pay for a 2nd certification for that. Disclaimer - I did an MSc in Bioinformatics 20 years ago. I've been working in the software industry for 30 years. I contemplated doing a PhD, but up until now, there has just been multiple firehoses of free information online to absorb (plus many good books).

just as ML engineers of 5 years ago generally knew nothing about SVMs, Gaussian Mixture Models, MLE vs MAP ... ML engineers of today know little about VAEs, GANs, LSTMs, Reinforcement Learning.

Cold topics of yesteryear quickly fall off the map.

I would go further and claim that a 2024 ML Engineer may well be unaware of 2019 BERTology and GPT internals as well as 2020 fast Attention kernels and be today focused solely on: a) Huggingface pipeline SFT, Diffusion, and Langchain RAG chunk GenAI orchestration. b) the ever-prevalent mind-numbing drudgery of data munging that occupies 50% of a data scientist's time.

I mean, all that effort put into learning PyTorch / Tensorflow implementations of Transformer architectures - is it relevant without 10,000 TPU year resources, when big tech GPU hoarders expose simple GenAI REST APIs for mass consumption ? An analogy: learning how CPUs are optimally engineered, when you dont have access to a fab, is completely irrelevant to app dev. a complete bifurcation of skills between the laymen and the high priests. OOTOH, if AGI ever takes off, it may be advantageous to have some idea of how it works.

A lot of low-lying fruit has surely been plucked by AI models in the fields of Code Generation, NLP and Computer Vision, and a lot of this is now shrink-wrapped as fine-tunable or zero-shot models behind APIs or abstracted in HF pipelines. IMHO, there are plenty of rapid advances yet to be made in focusing coding and ML engineering in more esetoric fields that require specialized domain knowledge (eg Cheminformatics, Quant Finance).

Rust, CUDA and JAX - I'm guessing one or more of these will be in increasing demand with a relatively high barrier of entry compared to other software tech

Paradoxically, Curiosity and Passion for Learning can work against you: - The time and effort put into continuously building tech skills has an opportunity cost on networking for jobs - many positions seek for narrow and deep expertise of specialists, so if you are driven to broad life-long learning, being a generalist can dilute and blur the specifics.

keep a balanced life - family, exercise, study and work. You are not your job; but you are to a large extent, your career. There is a difference.