After struggling to understand why our reasoning models would sometimes produce flawless reasoning or go completely off track - we updated Klarity to get instant insights into reasoning uncertainty and concrete suggestions for dataset and prompt optimization. Just point it at your model to save testing time.
Key new features:
- Identify where your model's reasoning goes off track with step-by-step entropy analysis
- Get actionable scores for coherence and confidence at each reasoning step
- Training data insights: Identify which reasoning data lead to high-quality outputs
Structured JSON output with step-by-step analysis:
- steps: array of {step_number, content, entropy_score, semantic_score, top_tokens[]}
- quality_metrics: array of {step, coherence, relevance, confidence}
- reasoning_insights: array of {step, type, pattern, suggestions[]}
- training_targets: array of {aspect, current_issue, improvement}
Example use cases:
- Debug why your model's reasoning edge cases
- Identify which types of reasoning steps contribute to better outcomes
- Optimize your RL datasets by focusing on high-quality reasoning patterns
Currently supports Hugging Face transformers and Together AI API, we tested the library with DeepSeek R1 distilled series (Qwen-1.5b, Qwen-7b etc)
Installation: `pip install git+https://github.com/klara-research/klarity.git`
We are building OS interpretability/explainability tools to debug generative models behaviors. What insights would actually help you debug these black box systems?
Links:
- Repo: https://github.com/klara-research/klarity
- Our website: [https://klaralabs.com](https://klaralabs.com/)
- Discord: https://discord.gg/wCnTRzBE
Key new features:
- Identify where your model's reasoning goes off track with step-by-step entropy analysis - Get actionable scores for coherence and confidence at each reasoning step - Training data insights: Identify which reasoning data lead to high-quality outputs
Structured JSON output with step-by-step analysis:
- steps: array of {step_number, content, entropy_score, semantic_score, top_tokens[]} - quality_metrics: array of {step, coherence, relevance, confidence} - reasoning_insights: array of {step, type, pattern, suggestions[]} - training_targets: array of {aspect, current_issue, improvement}
Example use cases:
- Debug why your model's reasoning edge cases - Identify which types of reasoning steps contribute to better outcomes - Optimize your RL datasets by focusing on high-quality reasoning patterns
Currently supports Hugging Face transformers and Together AI API, we tested the library with DeepSeek R1 distilled series (Qwen-1.5b, Qwen-7b etc)
Installation: `pip install git+https://github.com/klara-research/klarity.git`
We are building OS interpretability/explainability tools to debug generative models behaviors. What insights would actually help you debug these black box systems?
Links:
- Repo: https://github.com/klara-research/klarity - Our website: [https://klaralabs.com](https://klaralabs.com/) - Discord: https://discord.gg/wCnTRzBE