YossarianFrPrez
Joined 5 years ago
Karma
843
Submissions
231
Member since
May 25
Recent submissions
Comment · 4 months ago
Between this and the "Sunset time and the economic effects of social jetlag: evidence from US time zone borders" paper [0], it seems like the issue is the size of the discontinuous jump in time, not necessarily that we change the clocks. So why not "smear" the DST<=> ST transitions by having four half hour transitions, once each quarter?
Comment · 4 months ago
While I have some experience, I'm afraid I am learning mostly through a) trial and error, and b) using an llm chat interface to double check my thinking.
This being said, here are my top recommendations:
1. Build your system against real targets. Had HackerRank continually tested their prompt against 2-3 real resumes that were scored by hand, I think some of the issues would have immediately popped out. The people who built the prompt thought they could magically skip the hard part of articulating a preferred decision making process by having the LLM do it. But LLMs are much better at scaling a pre-existing decision-making process rather than inventing one, let alone the same one, from scratch, every time.
2. Think about what it would take to get motivated undergraduate interns to do the task from end to end, step by step. That's essentially what your workflow will need.
3. If the LLM can't do a step or sub-task reliably, then it's time to decompose those subtasks into even smaller chunks.
I'm sorry I can't be more helpful!
Comment · 4 months ago
Looking at the linked scoring prompt (resume_evaluation_criteria.jinja) [0], I immediately see several red flags that suggest the output won't be reliable. (I'm developing an LLM intensive application where the stakes are high enough that I need the LLM output to be reasonably correct.)
[0] https://github.com/interviewstreet/hiring-agent/blob/main/pr...
In no particular order:
1. The prompt is trying to get the system to do all of the evaluation steps at once. Instead, the system should break down the task of resume evaluation into its subcomponents and have separate prompts for each component. Like "evaluating open source contributions" should be its own task. Same with "assessing the complexity of software projects on the resume." Fwiw, each of the tasks contained within the prompt is woefully underspecified.
2. The prompt leaves spreads of ~10 points up to the LLM, when it's doubtful that humans are that well calibrated. Take for example:
> SCORING CRITERIA Open Source (0-35 points)
HIGH SCORES (25-35 points):
- Contributions to popular open source projects (1000+ stars)
- Significant contributions to well-known projects
- Google Summer of Code (GSoC) participation
- Substantial community involvement
3. The authors of this prompt have left an incredible number of judgement calls up to the LLM, when that's the very thing you want to minimize. Using the same example as above...
- Are all contributions to open source projects with 1000+ stars equal?
- What counts as a "significant contribution"? Doesn't that imply that the LLM has to know or read through all of the commits in like the last ~6 months at minimum for the project to understand what the given contribution meant to the project? That itself isn't impossible with tool usage, but again, that'd be a separate task.
- What on earth counts as "Substantial community involvement"? Why didn't the prompt authors define this, or at least give a few examples?
Honestly at this point maybe someone should build a tool that scans prompts for adjectives...
4. This sort of thing is just asking for trouble:
> SCORES MUST NEVER DEPEND ON:
Candidate's name, gender, or personal demographic information
5. Obviously this one depends on the LLM in question, but instead of writing things like:
> DO NOT RETURN A RESUME SUMMARY. RETURN ONLY THE SCORING EVALUATION IN THE SPECIFIED JSON FORMAT. Analyze the following resume and provide a JSON response with this EXACT structure (all fields are required):...
6. One major thing that's not in the prompt is anything about traceability. This system should be designed so that humans can review the logs and make sure this is working as intended.
7. Another thing that is missing in the file is what I'll call evidence of a theory of coding / coder quality. Most of the examples are designed to have the LLM assess proxies for code quality, not code quality itself. Surely both should be taken into account?
I'm not an expert at evaluating coders. But two pretty basic LLM-answerable thing I would ask is: How well do a candidate's 5 most recent commit messages match the contents of those commits? Do the claimed technical skills on the resume match their GitHub code? (i.e., if they say they know R, is there any evidence of that on their GitHub?)
8. The prompt also seems unaware of what it's asking the LLM to do:
> LIVE DEMO BONUS: Projects with working live demos should receive 10-20% higher scores
Comment · 4 months ago
Hmm. Maybe they are concerned about state actors trying to train equivalent models without the safeguards?
Comment · 4 months ago
I'd like to offer a counter-point to many of the comments here. While I understand being stymied and frustrated by a product one is paying for...
At the same time, I personally think the tradeoff between "having guardrails" and "some users are unhappy with the product" is well worth it. Think of what would happen if all of us who aren't so well intentioned could exploit Fable in terrible ways. Surely this tradeoff is better than saying "we can't make it perfect, so whoops, we aren't going to have any guardrails at all"? Especially because Anthropic did pretty extensive red-teaming of Mythos & Fable...
Comment · 7 months ago
Reactions to this are a bit curious. It's a satirical comment on how (presumably) initially well-intentioned younger founder-types get swept up in / by perverse incentives. The implication is that younger people who are still figuring out who they are and coming into their own may be more susceptible to these kinds of incentive traps.
The first section that showcases the fraud that has been committed is something I have no problem with, just as I have no issue with web3isgoinggreat.com. The "at risk" section is based on a mathematical/algorithmic joke. This is explained by the "methodology" section below it, which makes it clear that the equation used to calculate "risk" here is not entirely unlike the Drake equation for the probability of extra-terrestrial life.[1]
Comment · 7 months ago
Obviously this is quite unfortunate. While these cases can highlight latent mental health problems, it's still an issue that such things being exacerbated. I also think it will be interesting if anyone ever quantifies whether some LLMs are more likely to induce AI Psychosis than others. I'd be surprised if the guard rails are functionally identical from one LLM to the next, and there is a clear role for regulation to play here.
Some choice quotes:
> “What we’re seeing in these cases are clearly delusions,” he says. “But we’re not seeing the whole gamut of symptoms associated with psychosis, like hallucinations or thought disorders, where thoughts become jumbled and language becomes a bit of a word salad.”
> There seem to be three common delusions in the cases Brisson has encountered. The most frequent is the belief that they have created the first conscious AI. The second is a conviction that they have stumbled upon a major breakthrough in their field of work or interest and are going to make millions. The third relates to spirituality and the belief that they are speaking directly to God. “We’ve seen full-blown cults getting created,” says Brisson.
Also, for her podcast, the well-renowned couples therapist Esther Perel recently counseled a data scientist who was starting to fall in love with a chatbot he created, even though he is well aware of how the algorithm works [1]. I found it worth listening to. Perel very gently points out that a) he deluding himself and b) the deeper issue is the individual's sense of self-worth / self-esteem.
[1] https://podcasts.apple.com/us/podcast/where-should-we-begin-...
Comment · 8 months ago
I worked in tech for ~8 years and am now finishing up a PhD. One thing that stood out to me is how that in most scientific fields review papers tend to receive more citations than empirical work. This post looks at citation patterns and argues that scientific progress depends just as much on abstraction and synthesis as it does on empirical tests.
Comment · 10 months ago
Fair question. While I'm not an expert on AI Alignment, I'd be surprised if any AI alignment approach did not involve real math at some point, given that all machine learning algorithms are inherently mathematical-computational in nature.
Like I would imagine one has to know things like how various reward functions work, what happens in the modern variants of attention mechanisms, how different back-propagation strategies affect the overall result etc. in order to come up with (and effectively leverage) reinforcement learning with human feedback.
I did a little searching, here's a 2025 review I found by entering "AI Alignment" into Google Scholar, and it has at least one serious looking mathematical equation: https://dl.acm.org/doi/full/10.1145/3770749 (section 2.2). This being said, maybe you have examples of historical breakthroughs in AI Alignment that didn't involve doing / understanding the mathematical concepts I mentioned in the previous paragraph?
In the context of the above article, I think it's possible that some people are talking to ChatGPT on a buzzword level end up thinking that alignment can be solved via "fractal recursion of human in the loop validation sessions" for example. It seems like a modern incarnation of people thinking they can trisect the angle: https://www.ufv.ca/media/faculty/gregschlitt/information/Wha...
Comment · 10 months ago
What a terrible, awful tragedy!
A few months ago, OpenAI shared some data about how with 700 million users, 1 million people per week show signs of mental distress in their chats [1]. OpenAI is aware of the problem [2], not doing enough, and they shouldn't be hiding data. (There is also a great NYT Magazine piece about a person who fell into AI Psychosis [3].)
The links in other comments to Less Wrong posts attempting to dissuade people from thinking that they have "awoken their instance of ChatGPT into consciousness", or that they've made some breakthrough in "AI Alignment" without doing any real math (etc.) suggest that ChatGPT and other LLMs have a problem of reinforcing patterns of grandiose and narcissistic thinking. The problem is multiplied by the fact that it is all too easy for us (as a species) to collectively engage in motivated social cognition.
Bill Hicks had a line about how if you were high on drugs and thought you could fly, maybe try taking off from the ground rather than jumping out of a window. Unfortunately, people who are engaging in motivated social cognition (also called identity protective cognition) and are convinced that they are having a divine revelation are not the kind of people who want to be correct and who are therefore open to feedback. Because one could "simply" ask a different LLM to neutrally evaluate the conversation / conversational snippets. I've found Gemini to be useful for a second or even third opinion. But this means that one would be happy to be told that one is wrong.
[1] https://www.bmj.com/content/391/bmj.r2290.full [2] https://openai.com/index/strengthening-chatgpt-responses-in-... [3] https://www.nytimes.com/2025/08/08/technology/ai-chatbots-de...
Comment · 10 months ago
Per the article, this seems even better than the headline would suggest:
> Histotripsy generally seems to stimulate an immune response, helping the body attack cancer cells that weren’t targeted directly by ultrasound. The mechanical destruction of tumors likely leaves behind recognizable traces of cancer proteins that help the immune system learn to identify and destroy similar cells elsewhere in the body, explains Wood. Researchers are now exploring ways to pair histotripsy with immunotherapy to amplify that effect.
https://bsky.app/profile/evanwarfel.bsky.social
evanwarfel.com