Comment by johndough

For counting, there are specialized counting models, e.g. https://huggingface.co/spaces/MengqiLei/count-anything-demo

I tried to parse hand-drawn ER diagrams in the past and did not have much success with any model, frontier or otherwise. If you have annotated data, I'd recommend finetuning a recent (dense) VLM, but don't expect 100% accuracy. https://unsloth.ai/docs/basics/vision-fine-tuning