ahgamut

Joined 6 years ago

Karma

639

Submissions

Member since

Jan 3

I like programming with numbers.

I regularly use Python (numpy+, pytorch), C, some parts of C++, and now R. I would like to explore J further.

username at gmail dot com. I have the same username on Github.

View on Hacker News

Recent submissions

Comment · 1 year ago

My early contributions to https://github.com/jart/cosmopolitan were focused towards getting a single-file Python executable. I wanted my Python scripts to run on both Windows and Linux, and now they do. To try out Python, you can:

    wget https://cosmo.zip/pub/cosmos/bin/python -qO python.com
    chmod +x python.com
    ./python.com

Adding pure-Python libraries just means downloading the wheel and adding files to the binary using the zip command:

    ./python.com -m pip download Click
    mkdir -p Lib && cd Lib
    unzip ../click*.whl
    cd ..
    zip -qr ./python.com Lib/
    ./python.com # can now import click

Cosmopolitan Libc provides some nice APIs to load arguments at startup, like cosmo_args() [1], if you'd like to run the Python binary as a specific program. For example, you could set the startup arguments to `-m datasette`.

[1]: https://github.com/jart/cosmopolitan/commit/4e9566cd3328626d...

Comment · 2 years ago

most likely before v2 as well. The original post is from July 2021, which is a bit after Cosmopolitan Libc's v1 IIRC.

Comment · 2 years ago

My 4-core computer's probably building Cosmopolitan Libc in the background + the Python binary is 2.7 unoptimized. This blog post is originally from 2021.

Comment · 2 years ago

I've tried nuitka before, and a recent question that occurred to me was: does nuitka have an option to output just C files? Something like:

  python -m nuitka example.py --no-compile

Might be interesting to see if the above is possible. We could get things like a completely-statically-compiled Python stdlib within the APE.

Comment · 2 years ago

Hey, this is my post from 2021, when I was testing Python2.7 and Python3.6 with Cosmopolitan Libc on an old 4-thread Haswell. It's now a lot easier to build Python (and gcc, gnu coreutils, curl etc.), and the binaries are faster, multi-threaded, and quite convenient to use. There are lots of interesting directions to explore when it comes to building software with Cosmopolitan Libc.

Comment · 2 years ago

> new methods of communication with the compiler have been established.

From what I understand, this appears to a be separate binary from GCC/Clang that does static analysis and outputs C99.

Can this be a GCC plugin? I know we can write plugins that are activated when a specific macro is provided, and the GCC plugin event list allows intercepting the AST at every function declaration/definition. Unless you're rewriting the AST substantially, I feel this could be a compiler plugin. I'd like to know a bit more about what kinds of AST transformations/checks are run as part of Cake.

Comment · 2 years ago

> Instead, many have deeply underestimated LLMs, saying that after all they were nothing more than somewhat advanced Markov chains, capable, at most, of regurgitating extremely limited variations of what they had seen in the training set. Then this notion of the parrot, in the face of evidence, was almost universally retracted.

I'd like to see this evidence, and by that I don't mean someone just writing a blog post or tweeting "hey I asked an LLM to do this, and wow". Is there a numerical measurement, like training loss or perplexity, that quantifies "outside the training set"? Otherwise, I find it difficult to take statements like the above seriously.

LLMs can do some interesting things with text, no doubt. But these models are trained on terabytes of data. Can you really guarantee "there is no part of my query that is in the training set, not even reworded"? Perhaps we can grep through the training set every time one of these claims are made.

Comment · 3 years ago

What packages do you use in your app? If you're not using too many C extensions, it may be possible to build your app with Cosmopolitan.

Comment · 3 years ago

Try it out! We got Lua/LuaJIT, Python, PHP, and Rust building (the latter two are not fully automatic yet), so Ruby might be possible even now.

Comment · 3 years ago

https://github.com/ahgamut/rust-ape-example

My above repo contains example with the Rust standard library that build as fat executables with Cosmopolitan Libc.

I also got ripgrep to build https://github.com/ahgamut/ripgrep/tree/cosmopolitan, but it wasn't part of the cosmocc binaries because some crates that are part of the build require `#!feature(rustc_private)]` to be added to their source code. Same goes for bat and fd.

To summarize, Rust CLI tools can be built with Cosmopolitan Libc, but to build them like we can build gcc/emacs/git right now, we will need some support and Rust expertise.

Comment · 3 years ago

Yes, so pip works because the Python APE has OpenSSL built-in.

pip install requires modifying the APE, so I end up installing pure-Python libraries as follows:

    mkdir Lib
    ./python -m pip download httpx
    unzip ./*.whl -d ./Lib
    mv python python.com # in case the next step doesn't work
    zip -r ./python.com Lib
    mv python.com python

Installing CPython extensions like this is an unsolved problem, but I think there might be some interesting workarounds possible.

Comment · 3 years ago

The "/zip/" folder refers to the ZIP store inside the actually portable executable -- you can run `unzip -vl ./python` to see what's inside.

Also, to add new pure-python packages, you just need to unzip them into a local folder "./Lib", and add that folder to the APE.

Comment · 3 years ago

Thank you Gregory for writing this post. There have been a bunch of announcements about "setup.py has been deprecated", but few have clearly outlined how to move away from setup.py, and more importantly, fewer have outlined what a struggle it is to move away from setup.py.

I was sad to see setuptools officially deprecated, because it looks like another way in which Python packaging is being red-taped away for a non-expert. If someone like the OP (who has 10+ years programming Python) had to do so much for what appears to be a zstd CFFI/Rust wrapper, where does that leave the rest of us?

Here's a python package of mine that uses setup.py: https://github.com/ahgamut/cliquematch/blob/master/setup.py which I have not upgraded to the new tool(s) yet. I think I will need to upgrade it soon. If anyone has suggestions for a tool that will _fully replace_ setup.py, I would like to see tutorials with the following examples:

1. How would I build a package that has pure-Python files and data files? With setuptools I would use maybe MANIFEST.in or package_dir.

2. How would I build a package that has a CPython extension accessed via cffi? (this post points to the answer)

3. How would I build a package that has a CPython extension _without_ cffi, that just wraps some small C code I wrote with CPython's API? What about an extension that uses PyBind11? What about an extension that uses Rust?

4. How would I build a package that requires a "system" package like libblas-dev? Can something like numpy be built optimally without ever writing setup.py? What would a config for that look like? Last I remember numpy used their own patch of distutils to build, I wonder what it is now.

Comment · 3 years ago

One of the comments in the OP:

   I believe the problem is that bridge in the US (not true in some other countries) lacks a "critical mass" of 20 and 30 somethings.

I like to play bridge, but all the people I have asked in my age group (the 20 and 30 somethings) say bridge is a game their (grand)parents (used to) play. I am not sure how bridge can be introduced as a fun hobby without the face-to-face aspect, because compared to other online games it does seem a bit monotone.

Comment · 3 years ago

"Python 4, but not really", because we want to squeeze out more multithreading performance and be cool again. Some questions from reading the OP:

- How much does performance improve due to this No-GIL thing? Is it greater than 2x? For what workloads? - Do I have to compile two versions of every extension (gil/nogil)? I would prefer building extensions does not get any more complicated. - Can I automatically update my code to handle nogil? (a tool like lib2to3 or six)