I think some of the "Japanese has a ton of grammar points" is an effect of how the Japanese-as-a-second-language teaching resources label things, where a lot of what you could classify as "sentence patterns" are described and taught as "grammar". For example, the Dictionary of Intermediate Japanese Grammar lists ~に比べると as one of its grammar points, but this (meaning "compared to ~") isn't really new grammar, it's just a specific usage of particle に, a particular verb and と for if/when (in the same way "compared to X" isn't new English grammar but is a pattern of use of a particular verb).
My experience is that Japanese grammar isn't particularly complicated, it's just that it works backwards from Indo-European languages. Vocab is a pain because there's no common root of word origins to help the way there is between say English and French, but that's true for Chinese too I suppose. The writing system is kind of silly but it is what it is (and of course it doesn't matter at all for conversational fluency).
My experience is that Japanese grammar isn't particularly complicated, it's just that it works backwards from Indo-European languages. Vocab is a pain because there's no common root of word origins to help the way there is between say English and French, but that's true for Chinese too I suppose. The writing system is kind of silly but it is what it is (and of course it doesn't matter at all for conversational fluency).