Comment by Tostino

It depends on what you're doing... Just for reference, here is a small showcase of the capabilities that I've trained on a 13 billion parameter llama2 fine tune (done with qlora).

https://old.reddit.com/r/LocalLLaMA/comments/186qq92/comment...

Edit: Embed some of the content instead.

Inkbot can create knowledge graphs. The structure returned is proper YAML, and I got much better results with my fine-tune than using GPT4.

https://huggingface.co/Tostino/Inkbot-13B-8k-0.2

Simple prompt: https://gist.github.com/Tostino/c3541f3a01d420e771f66c62014e...

Complex prompt: https://gist.github.com/Tostino/44bbc6a6321df5df23ba5b400a01...

It also does chunked summarization.

Here is an example of chunking:

Part 1: chunked summarization - https://gist.github.com/Tostino/cacb1cecdf2eb7386baf565d157f...

Part 2: summary-of-summaries - https://gist.github.com/Tostino/81eeee9781e519044950332b4e64...

Here is an example of a single-shot document that fits entirely within context: https://gist.github.com/Tostino/4ba4e7e7988348134a7256fd1cbb...