Comment by jimbokun

But is there is something inherent to Lisp that makes it "clunky" to work with matrices?

Lisp can be at least as high level and expressive as any other language.