autograd

It's easy to "trivially know" because it's really expensive to disprove the statement, so you continue believing in your original one.

I am not in game development, but was recently looking in some deep learning networks. A lot of compute, bazillion layers of abstractions (erm python), but at the end of the day, 98% of the time was really spent calculating the matmul. I had luck that it was easy to check it and estimate the abstraction cost (not into great detail), and disprove my original thought.

Had i not checked it, I'd still trivially know that far more perf loss happened because of abstractions.

It is unclear to me whether it's archived because they've done everything they wanted or they just stopped all the efforts?

It's pretty refreshing that people have courage and start ambitious projects like this one :).

I hope projects like this get some traction and prove that we can move higher level than where we are atm with the frontend/frontend-backend code.

Thanks a lot for the reply. I'll include it some time soon in the article update

Thank you :)

It's just about backward compatibility. If you don't run the exact version of python that's running in production, how do you know that you're not using some method that does not exist in production yet (because you run older version there)?

Also, libraries with binary component often have to be compiled against specific version of python.

Docker is definitely an interesting tool for that, but my biggest problems is that I have to teach them Docker, which is a totally new layer of abstraction they haven't seen before.

How do you approach this? How technical are people you prepare Docker images for?

Does it pin versions of 2nd degree dependencies too? Like pip freeze would do? Also, when you remove a package, does it know to clear packages that were its deps and are not needed anymore?

Thanks for mentioning it! How would you compare it to the other tools from the original post?

I think this is not a problem specific to python packages, but a general problem of how we compile C/C++ software. There is no concept of packages and compiling one thing often requires installing a -dev package of some other library.

The issue is that lack of packaging C/C++ world spreads to all other communities that depend on them.

Hey xapata, thanks for pointing this out.

Any chance you could give me some reference so I can fix it in the original article?

I think that works when you use Python cli tools, but not when you're working on 5 different projects, each running different python version.

How come other tools don't just replicate successful models?

Unless you're in data science. Then you'll be split between conda and pip, which is worse than npm and yarn. You can always swap npm and yarn, but can't do that with conda and pip.

That's very true. Starting fresh definitely helps.

I would not say JS solved it, but yarn (and even npm these days) seem to be superior to all python dependency management tools before pipenv. None of them had a proper lockfile, for example.