olokobayusuf
Joined 8 years ago
Karma
174
Submissions
68
Member since
Mar 18
Recent submissions
Comment · 8 months ago
Comment · 1 year ago
Congrats on the launch!
Comment · 1 year ago
We should collab! We prefer to be the underlying infrastructure behind the scenes, and have a pretty holistic approach towards hardware coverage and performance optimization.
Read more:
- https://blog.codingconfessions.com/p/compiling-python-to-run... - https://docs.muna.ai/predictors/ai#inference-backends
Comment · 1 year ago
We're building something closer to this at Muna: https://docs.muna.ai . Check us out and let me know what you think!
Comment · 1 year ago
This is super interesting! I'm the founder of Muna (https://docs.muna.ai) with much of the same underlying philosophy, but a different approach:
We're building a general purpose compiler for Python. Once compiled, developers can deploy across Android, iOS, Linux, macOS, Web (wasm), and Windows in as little as two lines of code.
Congrats on the launch!
Comment · 1 year ago
Yup that's true. We do benefit from massive efficiencies though, thanks to LLM codegen.
Comment · 1 year ago
Our primary use case is cross-platform AI inference (unsurprising), and for that use case we're already in production by startups to larger co's.
It's kind of funny: our compiler currently doesn't support classes, but we support many kinds of AI models (vision, text generation, TTS). This is mainly because math, tensor, and AI libraries are almost always written with a functional paradigm.
Business plan is simple: we charge per endpoint that downloads and executes the compiled binary. In the AI world, this removes a large multiplier in cost structure (paying per token). Beyond that, we help co's find, eval, deploy, and optimize models (more enterprise-y).
Comment · 1 year ago
I'm founding a company that is building an AOT compiler for Python (Python -> C++ -> object code) and it works by propagating type information through a Python function. That type propagation process is seeded by type hints on the function that gets compiled:
https://blog.codingconfessions.com/i/174257095/lowering-to-c...
Comment · 1 year ago
Function (https://fxn.ai) | Remote (US)
We're building native code generation for AI developers. We generate high-performance C++/Rust to power open-source and on-device AI for our customers. We have customers ranging from early stage startups to the Fortune 1000.
You'll be:
1. Writing open-source Python functions that run popular vision models and LLMs; or
2. Writing high-performance C++ and Rust code that targets different accelerators (CUDA, Metal, etc); or
3. Writing parts of our Python-to-C++ compiler in support of (1) and (2); or
4. Some combination thereof.
Join the party: Email us at [email protected] or apply at https://app.dover.com/jobs/fxn.
No recruiters; no visa sponsorship (yet). We prize demonstrated curiosity and impact over everything else.
Comment · 1 year ago
Link to original article that kickstarted all of this: https://medium.com/@eugeniyoz/powering-angular-with-rust-was...
We're working to make Qualcomm NPUs a first-class citizen for deployment from PyTorch. Devs can write a Python function that runs a PyTorch model, then use our `@compile` decorator to transpile the model to a Qcom-specific C++ implementation (DLC) which compiles to a self-contained shared library.
The Qualcomm NPUs are fast. 1.8x faster than ONNXRuntime. See the link above.