nikonyrh
Joined 8 years ago
Joined 8 years ago
Making something just CUDA-compatible is non-trivial, and since Nvidia decides its direction and new features then the alternatives would always be lagging behind. Currently there are also major hardware differences between Nvidia and AMD, which may make highly optimized CUDA code inefficient or even buggy.
[1] https://github.com/ROCm/composable_kernel?tab=readme-ov-file#composable-kernel
[2] https://github.com/triton-lang/triton?tab=readme-ov-file#triton
[3] https://github.com/ROCm/HIP?tab=readme-ov-file#what-is-this-repository-for> The goal of the Hutter Prize is to encourage research in artificial intelligence (AI). The organizers believe that text compression and AI are equivalent problems.
Isn't this a standard on REPLs as well? You can select the code you wish to run, and press Ctrl+Enter or what ever. I must admit, I've programmed Python for about 10 years in Spyder and VS Code now, but I haven't used notebooks at any point. Just either ad-hoc scripts or actual source files.
My definition of a "notebook" is an ad-hoc script, split into individual "cells" which are typically run as a whole. On my workflow, I just select the code I wish to run. Sometimes it is one expression, one line, 100 lines or 1000 lines depending what I've changed on the script.
An other point is that you don't need to write any CUDA code to be able to utilize GPU computing. If you need ML models, you have frameworks like PyTorch and Tensorflow. You just need to express your mathematical problem, and the framework will take care of the rest.
Even if you needed to write custom GPU code, you don't need to do it in C anymore! For example you can JIT Python, using Numba or Triton.
Usually writing custom code is only required when:
- You are doing something novel, like PhD level stuff
- You must optimize the ML project for performance and trough-put at interference time
- You need to brute-force solutions (be it crypto-hashes, passwords, NP-complete problems, ...)
My last point to you is that do you want to learn to use these pre-existing frameworks and libraries, or learn to develop them or maybe even create new ones? What ever your answer is, I'd say that the first option is a great stepping-stone to advance to the second one.
- In-memory database
- Data compression algorithm
- 2D or 3D physics engine / game
- Game engines in general