Comment by anuj0456

yes, most of them are similar. but implementation of GQA, MLA, mHC, Sliding Window changes the implementation drastically because of which the overall model effeciency changes.