Comment by binarymax

I have a 3090 w/ 24G VRAM, it supports flash attention and bf16, but is a bit old. I dont run generative models on it as they would be slow and inaccurate, but I do run embedding models, OCR, and system one (decision) models on it.