Convert your vector embeddings into a set of questions and their ideal responses. Use this dataset to test your LLM and catch failures caused by prompt or RAG updates.
Get started in 3 lines of code:
```
pip3 install fiddlecube
```
```
from fiddlecube import FiddleCube
fc = FiddleCube(api_key="<api-key>")
dataset = fc.generate(
[
"The cat did not want to be petted.",
"The cat was not happy with the owner's behavior.",
],
10,
)
dataset
```
Generate your API key: https://dashboard.fiddlecube.ai/api-key
# Ideal QnA datasets for testing, eval and training LLMs
Testing, evaluation or training LLMs requires an ideal QnA dataset aka the golden dataset.
This dataset needs to be diverse, covering a wide range of queries with accurate responses.
Creating such a dataset takes significant manual effort.
As the prompt or RAG contexts are updated, which is nearly all the time for early applications, the dataset needs to be updated to match.
# FiddleCube generates ideal QnA from vector embeddings
- The questions cover the entire RAG knowledge corpus.
- Complex reasoning, safety alignment and 5 other question types are generated.
- Filtered for correctness, context relevance and style.
- Auto-updated with prompt and RAG updates.
Get started in 3 lines of code:
```
pip3 install fiddlecube
```
```
from fiddlecube import FiddleCube
fc = FiddleCube(api_key="<api-key>") dataset = fc.generate( [ "The cat did not want to be petted.", "The cat was not happy with the owner's behavior.", ], 10, ) dataset
```
Generate your API key: https://dashboard.fiddlecube.ai/api-key
# Ideal QnA datasets for testing, eval and training LLMs
Testing, evaluation or training LLMs requires an ideal QnA dataset aka the golden dataset.
This dataset needs to be diverse, covering a wide range of queries with accurate responses.
Creating such a dataset takes significant manual effort.
As the prompt or RAG contexts are updated, which is nearly all the time for early applications, the dataset needs to be updated to match.
# FiddleCube generates ideal QnA from vector embeddings
- The questions cover the entire RAG knowledge corpus.
- Complex reasoning, safety alignment and 5 other question types are generated.
- Filtered for correctness, context relevance and style.
- Auto-updated with prompt and RAG updates.