albystein

Hi Peter,

Thanks for hosting this AMA. I’ve an immigration issue related to status transition. So I came to the U.S on an F-1 visa about 6 years ago. However, during the course of my undergraduate study, I found out that I qualified for TPS. So I applied and received it. Afterwards, I decided to take a leave of abscence from school to work on a startup because TPS allows me to. I was also made to change my status from F1 to TPS. And my I-20 status period expired a few months after I took the leave at the beginning of 2024.

Owing to the recent TPS terminations of various countries, I’m afraid that my TPS will be terminated once it’s up for review in a few months. My question regards transistioning back to F1 status. I’m wondering if I can change status back from TPS to F1 without having to leave the country. This is because a visa ban is currently in place for the country for which I hold TPS. As such I can’t risk traveling outside the country as I won’t be able to get back in because of the visa ban. Btw the ban anpplies to all visa categories including F1 and O1.

Thanks!

Just increased message limits to 100 per day

this seems like a plausible outcome, and if true could spell disaster for OpenAI models relative to the competition and open source models. Currently, reliability is one of the core obstacles preventing widespread adoption of LLMs in many business critical workflows. And if these rumors, that GPT-4 is inherently un-deterministic and unreliable, are true then most enterprises are better off finetuning open source LLMs—which are just as capable—for their specific domains. they stand to gain better performance that way anyways, as domain-specific models will always outperform generalist ones

this hypothesis makes a lot of sense. if indeed gpt-4 is a sparse MoE—which i believe it is—then OpenAI must have tested and proved their initial idea of a large capacity MoE LLM model first training/building a smaller one. this smaller test model might be gpt-3.5-turbo.

I think hallucination is akin to encountering a problem on a test to which you don’t fully remember the right concept to solve it. You might attempt the problem with whatever little knowledge you can recall, but the answer is ultimately incorrect. Likewise, these LLMs exhibit such behavior when they hallucinate. I think what distinguishes the LLM’s hallucinations from humans is the confidence level. A human who is unsure of their answer might tell you that they’re not certain or they don’t know. LLMs like chatgpt, however, will just confidently make stuff up.

I think it depends on the sampling/decoding method. Also, perhaps you could elaborate more on what you mean by top_p

this is not what language models typically output as logits though. perhaps the term now is being misused and it’s causing a lot of confusion

Basically, logits are the raw outputs of the last linear layer of a neural network just before the softmax(for multi-class classification tasks) or the sigmoid(for binary classification tasks) is applied to the outputs.

This is exactly how I understand it. Although, I feel the thermodynamics analogy might be more intuitive to most people

I’m not sure if comparing it to the greedy algorithm is the correct way to think about it. Even after applying the temperature, if you use greedy decoding, you’ll still pick the output that has the highest probability. Where the concept of temperature is effective though is when you use a sampling method like sampling from a multinomial distribution, nucleus sampling, etc.

Typically, in new markets and product categories, you should not consider early complimentary products as competitors, but rather as partners. During these early days, all participants are still trying to validate the product value proposition. It’s only after this value proposition has been validated and an initial market has been established that you should then start treating these other products as competing products.I think this mental model applies to the emerging generative AI markets

I think this line of reasoning is misguided. What’s striking and more important to focus on are the abstract reasoning abilities of these systems. Language, as you mentioned, abstracts real world objects and phenomena, so it’s a good approximation of the real world. Thus, if an LLM can reason this well using language, it’s safe to say that perhaps they’re doing something akin to what the human mind does.

Your critique about lack of grounding in these systems is an easy problem to solve. It’s as easy as teaching an LLM to associate words with real world objects or phenomena. Image-classification models, text-2-image models, audio transcription models, and many other modal specific systems already do this to some extent. And more recently there has been a push towards multi-modal language models(Deepmind’s flamingo), so this line of argument will be debunked very soon.

I actually believe GPT-4 will be multi-modal and it’s capabilities will dispel majority of these criticisms

This a very well put comment with a great analogy. A new emerging paradigm of action-driven LLMs is taking the approach of using the reasoning abilities of LLMs to drive agents that can take actions, interact with other tools and computer programs, and perform useful tasks like autonomously programming, customer support, etc

And I think you’re right when you say that they’re lacking in recursive thinking abilities. However, their reasoning abilities are pretty excellent which is why when you prompt them to think step-by-step, or break down problems to them, they correctly output the right answer.