> Also taking the inverse of a matrix isn't always easy. For example for very large matrices.
Yeah, I know. I am saying that I think there is probably an undiscovered solution to doing faster inverse matrices, based on occam's razor (which I'm taking to mean that "complex and ugly" suggests there is probably undiscovered "simpler and more elegant"... which I've seen over and over again in other areas... gradient descent being the "complex and ugly" here)
> This gives a good guess for the solution at each timestep(t>0) and improving that with gradient descent can be more performant than completely solving for a matrix inverse each time.
That's a good point, but perhaps the hypothetical matrix inversion I'm assuming exists also allows you to do computationally efficient differential inversions based on small changes to the matrix.
I admit this was just a "gut feeling," but when I first learned about gradient descent vs. matrix inversion I had a very strong gut feeling that there was a better way out there.
My career is programming (where I have followed this "gut feeling" many times, usually to success) and I did papers on matrices and determinants in high school, this may be informing my "gut".
Not sure if any mathy people are reading this, but I think there's gold to be had here, if you look hard enough
Yeah, I know. I am saying that I think there is probably an undiscovered solution to doing faster inverse matrices, based on occam's razor (which I'm taking to mean that "complex and ugly" suggests there is probably undiscovered "simpler and more elegant"... which I've seen over and over again in other areas... gradient descent being the "complex and ugly" here)
> This gives a good guess for the solution at each timestep(t>0) and improving that with gradient descent can be more performant than completely solving for a matrix inverse each time.
That's a good point, but perhaps the hypothetical matrix inversion I'm assuming exists also allows you to do computationally efficient differential inversions based on small changes to the matrix.
I admit this was just a "gut feeling," but when I first learned about gradient descent vs. matrix inversion I had a very strong gut feeling that there was a better way out there.
My career is programming (where I have followed this "gut feeling" many times, usually to success) and I did papers on matrices and determinants in high school, this may be informing my "gut".
Not sure if any mathy people are reading this, but I think there's gold to be had here, if you look hard enough