Jackson__

It does really take a load off my mind, knowing that he's completely out of binance now, and instead they have an independent CEO like... his wife. Yep. Can't get more independent than that.

>To be done right, such vision benchmark should consider the peculiarities of the vision tower

As per my previous comment, I used the _exact same weights_ for the comparison. The vision tower is always kept as an unquantized BF16 file for GGUF, as I believe is the default.

>Second of all, the "find coordinates" of something is super difficult task of any model

It is in fact not (anymore), and most recent VLMs I tested have been standardized to point and bbox in a relative 0..1000 coordinate system with decent enough accuracy.

And finally, none of this excuses that Strata performs so much worse with the same weights. Which does kinda leave me confused as to what the point of this reply is.

I've just tested Strata on a simple 50 image vision benchmark. The task is to output the exact coordinates of a requested object. The result via Strata had a median error distance of 154.8 pixels, avg of 168.8. Running the exact same GGUF and vision adapter weights on llama.cpp gives me a median error of 46.5, avg 81.4.

To put that into perspective, here are some more numbers from other models via llama.cpp:

Median/Average

Qwen 3.5 9B BF16: 46.5 / 193.3

Qwen 3.6 35B Q4 K XL: 38.4 / 76.4

Qwen 3.5 122B Q3 K M: 32.9 / 68.6

The difference in vision performance is as large as the jump from a 9B model to a 35B model. All tests were performed at temp=0.

I have done no further testing, as these results line up perfectly with my expectations.

Parallel construction has been a thing for a very long time to get around this.

Judging by the performance of the shown examples, the quality is closer to pre-2022 Photoshop content aware fill than actual 10B models.

I think it is safe to say this is pretty far from a "scientific" work.

If you can't trust them to act ethically on the small scale, why would you expect that to turn around once it gets to a larger much more important scale?

How many government sanctioned school bombings does it take for them to quit working with said government? For now we know that number is somewhere between infinity and 1.

Oh what is that, the most "ethical" AI company on the planet making deals with literal democracy undermining fascists?

I'm starting to think the problem with "ethical" AI was always that no company could ever act ethically in the long term. They are and always will be a cancer to society and AI will only serve to amplify this further.

Alternative take: Because no designers are getting paid to move "rm" to "fileops rm" or otherwise between releases.

Hell, I would have thought it likely that anthropic was doing the same thing. Of course that was proven wrong, but for OAI I wouldn't even be guessing. This has always been what sama does.

As someone who is new to the whole google cloud ecosystem, the amount of dark patterns they employ are absolutely shocking. Just off the top of my head:

1. You never know how much a single API request will cost or did cost for the gemini api

2. It takes anywhere between 12-24 hours to tell you how much they will charge you for past aggregate requests

3. No simple way to set limits on payment anywhere in google cloud

4. Either they are charging for the batch api before even returning a result, or their "minimal" thinking mode is burning through 15k tokens for a simple image description task with <200 output tokens. I have no way of knowing which of the two it is. The tokens in the UI are not adding up to the costs, so I can only assume its the first.

5. Incomplete batch requests can't be retrieved if they expire, despite being charged.

6. A truly labyrinthine ui experience that makes modern gacha game developers blush

All I have learned here is to never, ever use a google product.

It is literally not even a vision model.

It's not just any pile of melting glaciers, its a pile of melting glaciers that looks really big in mercator projection!!