XTXinverseXTY
Joined 3 years ago
Karma
76
Submissions
33
Member since
Jul 12
Recent submissions
Comment · 9 hours ago
It's a pretty straightforward geometric argument, from your own analogy
I'm sure there's a coherent rebuttal but "That is plainly rubbish" isn't the reply I'd expect in good-faith discussion
Comment · 3 days ago
What's a Kofuni expansion?
Comment · 11 days ago
I'm afraid this only serves to more vividly depict scientists as rent-seeking
Comment · 1 month ago
Laya came after Jev, the original post is clear about this much [0].
Moreover the specific prior art claim is absurd (self-plug) [1]. GLiClass[2] is at least a coherent precedent.
[0]: https://laya.convaiinnovations.com/
[1]: https://xtxinversexty.com/layas-prior-art-claim-is-absurd/
Comment · 1 month ago
Hard to separate the memorability of the abstract from the underlying material.
IMO the most memorable papers contain some unexpected simplification: a complicated problem turns out to reduce to something much simpler [0], perhaps for a counterintuitive reason [1].
So a memorable abstract should advertise that the paper contains some cute little trick. This may not help you actually publish though
[0]: Attack the RLHF problem with a simple classification loss: https://arxiv.org/abs/2305.18290
[1]: Frustratingly Easy Meta-Embeddings: https://arxiv.org/abs/1804.05262
Comment · 1 month ago
besides overstepping any reasonable definition of creditworthy, it's really stupid.
Comment · 1 month ago
OP's was leaky slop from day one [0][1], as is his article [2]
It is arrogant and entitled for the author to take credit for the concept of RL over sequence embeddings, and none of the work that went into pretraining, not to mention the egregious target leakage [1]
[0]: Author fails to grasp the concept of virtual environments https://www.reddit.com/r/LocalLLaMA/comments/1kl0uvv/comment...
[1]: his `train.py` has `outcome` as a model input (conversation_metrics built from _parse_conversation which includes outcome): https://huggingface.co/DeepMostInnovations/sales-conversion-... https://huggingface.co/DeepMostInnovations/sales-conversion-...
[2]: 100% of this post is AI-generated https://www.pangram.com/history/97e0be84-391d-46b8-9c16-2d8f...
Comment · 1 month ago
Scaling laws project that a model with more parameters trained for longer on more data yields predictably better performance, and that generally you want to scale these factors commensurately. More of the compute budget is being spent on RLVR [0] for which we also fit scaling laws
Researchers tweak data mix, reward shape, model architecture, etc etc, breakthroughs which reduce the cost to train a just-as-smart model. But this increases the returns to scale, which further incentivizes bigger models trained for longer on more data
[0] "...to run reinforcement learning training...at pretraining scale." https://x.ai/news/grok-4?_bhlid=b9339d7816a05adeb52bae7050cc...
Comment · 1 month ago
Puts the onus on the AI companies to provide a specific replacement mechanism, no? Unless I'm unfamiliar with something else he's written that proposes something more specific and constructive
To Tao’s credit he obviously identified the problem very clearly and admits understandably "we did not have the time to have a more consultative process, as with Leiden; but we decided that the urgency of the situation was such that we needed to release a statement sooner rather than later".
Comment · 1 month ago
If they could declare with certainty that Buckminster's and Alpoge's usage data had been totally excluded from training, would that set a worse precedent and reflect poorly on their de-identification process (and data access safeguards moreover)?
This may sound like a charitable interpretation of OpenAI's remark, but consider that the lie would be (I think) impossible to falsify from the outside. They could easily just say "no sir we didn't peek" unless:
1. The conspiracy to peek at codex sessions involved enough people that the risk of one snitching is non-negligible
2. Lawyers advised it would be a bad idea to make such a remark, whether true or false
Comment · 1 month ago
Apparently this post was prompted by a scary-sounding headline in The Information[0], that Astra is a looped transformer, implying CoT monitorability may be less reliable. The day after the report, Jakub tweeted[1] that he "wanted to prevent a race into unmonitorability kicked off by confused reporting. The depth of the computation graph for our present frontier models, including Astra, is within a factor of two of GPT-4." This post seems to elaborate on that.
I imagine that the AI labs have an uneasy truce to prioritize alignment and monitorability. Following the HF incident, OpenAI probably feels especially sensitive to being perceived as reckless, lest other labs feel obligated to defect.
[0] https://www.lesswrong.com/posts/PLisnSFir8y5AHkmP/how-concer...
Comment · 1 month ago
unnecessary condescension
Comment · 3 months ago
So you admit that chapter 7 does not read almost as poetry?
Comment · 6 months ago
I noticed that Clawdbot’s initial acolytes seemed to skew towards solo founders and hustler/grifter types. The Mac minis were likely to spam leads over iMessage. The single top downloaded skill was for Twitter. The fastest way to monetize an openclaw agent is by spamming fake social proof for your product (including for openclaw itself).
xtxinversexty.com