I got tired of LLM outputs silently failing in pipelines, so I built a small scoring layer around it.
It checks three things before your output moves forward: does it match the schema you expected is it consistent across runs
does it actually align with the context you provided
Returns a confidence score and a risk level. That's mostly it.
Works with OpenAI, Anthropic, Gemini, Ollama and a few others. Sync and async both supported.
It's heuristic, not a guarantee. If your context is bad, the scores will be too. Hit a star, if you found this useful.
It checks three things before your output moves forward: does it match the schema you expected is it consistent across runs does it actually align with the context you provided
Returns a confidence score and a risk level. That's mostly it.
Works with OpenAI, Anthropic, Gemini, Ollama and a few others. Sync and async both supported. It's heuristic, not a guarantee. If your context is bad, the scores will be too. Hit a star, if you found this useful.
Try now: pip install hallx