Comment by nlightcho

Make sure you are using the instruction tuned model. The base model will be difficult to prompt.

It works in 8-bit with about 12GB of VRAM usage. Here's sample code:

https://gist.github.com/AlexanderDzhoganov/a1d1ebdb018e2e573...