georgeven

This is my recent PhD work. I want to show that AI can be used for good, rather than just optimizing ads and keeping people on TikTok for longer. Modern VLMs (LLMs with vision capabilities) are just good enough for labeling that they can replace route-label work for bioacoustics, and I find that distilling these labels into a bioacoustic encoder -- such as SongMAE, my previous work -- can work fairly well. Interestingly, the student exceeds the teacher model, and doesnt asymptote at teacher performance. This work was actually roughly inspired by Simon Willison (pelican guy), he made a post that showed Qwen 27B is an effective labeler for images.

ARC-AGI-3 is actually about action efficiency not solve rate, current LLMs just make a bunch of moves that inefficient, thereby lowering their score

What are you building if you dont mind sharing? I am moving and all these online applications and such are so annoying.

No! because they did not release GPT-2 XL until analogs appeared a year later. So the SOTA model during GPT-2 days was closed weights.

PCIE 16 gig is <300 dollars these days a piece

on eBay you can buy 4 of them for 1000. PCIE 16 gigs.

I have a 1500 dollar machine that can run it at 50 tok/s (3 V100s)

innovation requires monopoly is a Theil one liner -- probably true too

I wonder if you can sue him in civic court, could teach him a lesson.

The issue then becomes it writes insane one liners that are impossible to read. Would be difficult to set up an RL environment where the goal is correctness and readability.