Comment by bashtoni

My experience running quantised models is that they look the same at first but are all noticeably worse once the context size gets beyond anything trivial.