inconfident2021

> It turned out that architectural improvements really did matter

Transformer (2017). NET based Diffusion (2015). Score based diffusion (2019). DDPM (2020). Uses unet (2015). Clip (~2021) uses Resnet (2015) + ViT (2020). stable diffusion also uses Unet.

Yes, deep learning is easy. Throw compute, you get the answer.

Those with insights on the old problem are now churning papers because they throw compute and deep learning to the stuffs they understood well.

You can look into any paper and see inspiration from old methods. Applying deep learning is hard, deep learning itself is quite easy. If it were really hard, it wouldn't have been popular at all! Nowadays most of the people don't even bother with architectural gains.

Sure if better architecture eventually comes up, people will throw new ones into the compute. I don't believe in bayesian stuff either. However it is worth learning otherwise you might miss insights on a lot of papers!

Whatever you do, just having the idea to understand problem matters a lot. Then later comes the part of deep learning and throwing compute.

People say our job is safe, but aren't we in similar position to the horses who got replaced by cars or telegraph operators who just vanished?

There seems to be many people who thinks sales is the way to go. So does it mean that being a average engineer will have no value in substantial future?

How can one demonstrate this skill? When academics look for grades and HR looks for buzz word. Is there some keywords that people are missing which will help them demonstrate it?

I agree with public speaking. Whenever I see someone able to even create a video and upload with a face, I am in awe with their confidence.

Public speaking just takes everything to the next level. How can people be so confident about themselves?

For example some technical blogger post it like they own their posts. Going through that feels like the author is a all-knowing being.

How does that even kick in within a person?

I call this survivorship bias. No matter what you make, only few care enough. Find this few is hardest part. Because there are millions others doing the same damn thing.

Everyone has a kick for the survivorship bias. At the end it is just that.

How about other who made similar apps? How about all those aspiring game devs who actually put games, web developers who launched products or even some guys who made something apple would want?

The OP managed to get lucky.

You can now read new papers and follow ups from the community! It is really useful. You already know chain of thoughts. So you are on a good track!

Try to use these LLMs to automate more of the mundane tasks. Like scripting in bash, compiling videos, converting documents, refactoring data, transpiling code, transcribing code, etc. You will begin to see what works and what doesn't!

At the same time try to come with fun challenges for yourself to fool it. That will aid in learning ways to make it more obedient to you.

They would probably not pass in the div or filler elements.

The reason I passed the div was I wanted things to be surrounded in that exact space. So when the model output it, the button would be in right place with right size.

The extra filler forms a guiding factor that helps will be stored in the context! I did that using GPT3.5.

This is a good list of points! Knowingly or unknowingly I have been doing most of these things. 4-5 is currently unreachable for me. Others are quite manageable!

You write requirements and your expectations and make the model match your expectations. Until you have clear expectations of what you want from it, prompting LLM is pretty useless. It cannot do highly specific task because those are limited in the original training corpus too. However, for more generic task, it has seen most of the stuff out there, so it should be good enough. Having clarity on your problem is the key.

You have to make sure to couple chain of thought with branching, analysis and evaluation, then you can get pretty good results.

> understand X number of words, or pages, or paragraphs.

LLMs do not have the ability to reason with numbers. Most of the time they are hallucinating. One good strategy is to make it output in list and define the structure for each item of the list. If you give an example of what your list should look like, it will give you something close it.

> has trouble carrying a list forward without losing items.

This is the fundamental problem with these models because of the context limit. When you are prompting always remember that is processing a huge paragraph and emitting the next sentences of the paragraph. If you want information to be carried onwards, you have make it output on every prompt or you can also try to use specific identifiers. LLMs are good at in-context learning. It will not work 100% of the time, but it is usually good than having nothing at all.

> I want to speak chinese to the AI and I want it to correct me.

Give it a role of tutor and describe the instructions what the tutor should do.