I tested your model quickly and it seems that the model does not do what is claimed here (that it can do the same as jev). It relies on an LLM, while jev does not need an LLM. While it is interesting and novel work it is not a replacement for expensive LLM calls.
For anyone interested in playing around with it, it requires python 3.11, else it will segfault.
According to the code the conversation turn index is a signal that enters so it is judging on something that does not transfer to other tasks. In the sampling it moved the score more than the actual content.
For anyone interested in playing around with it, it requires python 3.11, else it will segfault.
According to the code the conversation turn index is a signal that enters so it is judging on something that does not transfer to other tasks. In the sampling it moved the score more than the actual content.