moonu

Can you instruction-train a duplex model like this? How would that look? I also wonder, if a model trained only on speech can develop world knowledge like this, could you train an image model only on images of text to essentially act as an LLM?

They have a separate decision model called Mercury Decide

I really wish they would make these multimodal. Seems like a definite improvement over 2.5 though in my testing

Curious why you use Gemini 3.5 Flash Lite instead of SAM/Moondream? Any specific workflows where it outperforms?

Even though they're technically trained for music, it might be worth testing Suno/Lyria to see if they're able to do this. You might have to isolate it afterwards, but seems like it could be viable

This is an interesting question where the answers feel tied to the training data, because with 2D, you primarily have declarative data with HTML/CSS, frontend frameworks, scene graphs. With 3D though, I would think most of the training data is imperative, because of libraries like Three.js and Python scripting for Blender.

For your first point though, I don't think raw vertex data necessarily needs to be the output, it could instead be based on shape primitives with raw points as an escape route, with an architecture similar to the one Quiver/StarVector uses for SVG generation. I think llama-mesh and mesh-gpt prove this semi-viable for 3D?

Even some of those smart 3D object ideas you mentioned could still work with declarative, where you build constraints and resolvers and such into the compiler instead of treating each design as a program.

In my own experimentation with 2D, I haven't actually seen a major gap between the outputs from the agent for imperative vs declarative but I'm curious if you've done any testing with declarative languages for 3D and whether you see a quality gap. It should in theory be much more token efficient?

I've been working on an agentic graphic design tool, so vision is quite useful for having the model check its own work. I'm already seeing improvements with this model vs the text-only one.

I imagine this is based on their 'Thinking with Visual Primitives' paper, and they had mentioned that the weights would be released for that

Zed is a good one, and it has a single toggle that lets you turn off all the AI features

I like the demos on your landing page, quite fun. I don't think I've seen VLMs used in this kind of real-time context outside of robotics before. I feel like the actual ideas behind JEPA make a lot of sense for increasing sample efficiency and is probably a lot closer to how our own brains work (considering that people with aphantasia or those that lack an internal monologue still function just fine).

This has been a pretty big topic of discussion recently with Prime Intellect's new harness making leaps on ARC 3's public dataset. This is probably a bit overstated considering that a big focus of the benchmark is to test the model without custom harnesses, but you can imagine how for some tasks, especially long running ones, this kind of on-the-fly context management + tool generation could be quite useful. It gets better over time, it can build shorthand and tools for token efficiency, like a loose approximation of continual learning.

Yeah, I have it enabled for just one of my projects, and it says "You're enrolled for complimentary daily tokens." Haven't been billed for any usage with this. Their tooltip doesn't mention the newer models, but it works for those too. I used them in a recent project that I knew wouldn't hit the rate limits. Not sure how eligibility is decided though.

I'm pretty sure that was a fake screenshot made as a joke, not an actual job posting