johndough

The upper chart shows the availability of Ox Alpha and the lower chart shows the availability of GLM 5.3 by Z.ai. They had a blip at exactly the same time.

Comment · 2 months ago

    > 1 quadrillion tokens per day on Nous portal

If you are referring to this number (https://xcancel.com/NousResearch/status/2090899914700054780), they are either mistaken, or they mean that they can route 1 quadrillion tokens per day, but the provider behind Ox Alpha certainly can't provide that. Almost all of my requests have hit a rate limit so far.

Comment · 2 months ago

     Don't post generated text or AI-edited text. HN is for conversation between humans.

https://news.ycombinator.com/newsguidelines.html

Comment · 2 months ago

For counting, there are specialized counting models, e.g. https://huggingface.co/spaces/MengqiLei/count-anything-demo

I tried to parse hand-drawn ER diagrams in the past and did not have much success with any model, frontier or otherwise. If you have annotated data, I'd recommend finetuning a recent (dense) VLM, but don't expect 100% accuracy. https://unsloth.ai/docs/basics/vision-fine-tuning

Comment · 2 months ago

The order is:

    LLM issues tool call to read high res image ->
    harness sends high res image to server ->
    server downsizes it to 800x800 (blurry) ->
    LLM issues bash command (e.g. `convert`) to crop a small subimage (e.g. 600x600) from the high res image ->
    LLM issues tool call to read subimage ->
    harness sends subimage to server ->
    server does not resize the subimage because it is small already, so it is not blurry when finally ingested by the LLM

Comment · 2 months ago

LLMs read images by splitting them up into e.g. 16x16 patches, which are then converted to embedding vectors and fed to the LLM, so from a technical point of view, feeding a big image as many 20x20 patches all at once is not too different from cropping subimages from the image, splitting those subimages into patches and feeding them to the LLM. Of course, the LLM has to be trained to understand that those images belong together, but it can be done.

Comment · 2 months ago

Yes. When the LLM tries to read an image, it will be resized by DeepSeek's server to 800x800, which might be a bit blurry. The LLM will then crop a smaller image from the high resolution image (using e.g. the `convert` tool via bash) and will then read the small cropped image. This image will still be resized to 800x800 by DeepSeek's server, but since it is already small, there is no or little loss of quality.

Comment · 2 months ago

It was explicitly said that they are pursuing multimodal support. A quote from the meeting transcript: https://github.com/demo-zexuan/liang-wenfeng-investor-meetin...

    Nevertheless, as a component, we will undoubtedly implement multimodal support — and we are already doing so. We plan to develop relevant models, ensuring that versions like V4 and subsequent iterations will natively support multimodal functionality.

Earlier, the following was said, which might match more what you had in mind.

    Achieving excellence in AI training does not require a global model or even multimodal approaches—by narrowing the scope of AI training and eliminating multimodality, certain tasks may remain unachievable without compromising the algorithm's validity.

    Multimodal approaches ultimately need to be implemented.

It is difficult to tell who said what, since the speaker ids are missing.

Comment · 2 months ago

There are models specifically for splitting an image into text regions, e.g. PP-DocLayoutV3 https://huggingface.co/PaddlePaddle/PP-DocLayoutV3

I am using a stripped-down minimal version of it which I uploaded here, since I am not a fan of huge dependency trees: https://github.com/99991/simple-pp-doclayoutv3

Another recent model for this task is Unlimited-OCR: https://github.com/baidu/Unlimited-OCR

Comment · 2 months ago

Might still be fine. The most recent crop of vLLMs proactively use whichever programs are available on the system (e.g. ImageMagick or PIL) to "zoom in" by cropping subimages if they can't quite make out the details.