lkirk

I think this is a bit optimistic compared to my view (wrt portability). There's a large stack of software that is involved in training and probably less so in inference. I'm not saying it's impossible but there are definitely different levels of reproducibility and the academic incentive structure doesn't really prioritize reproducibility in my experience. I'm sure it varies quite a bit, I'd be curious to know how those in this problem space are thinking about reproducibility and at what level.

For me, in my use of Python as a data analysis language, it's not python's speed that is an annoyance or pain point, it's the concurrency story. Julia's built in concurrency primatives are much more ergonomic in my opinion.

This thought led me to a strange conversation with chatgpt. I just base64 encoded "Hello chatgpt how are you" as "SGVsbG8gY2hhdGdwdCBob3cgYXJlIHlvdQ==" and sent it as the first prompt. Notice how there is no question mark in my phrase. Now here's my conversation with chatgpt:

User SGVsbG8gY2hhdGdwdCBob3cgYXJlIHlvdQ==

ChatGPT This appears to be a Base64 encoded string that translates to "Hello chattpt how are you?"

User not quite... almost but not quite

ChatGPT I apologize for the mistake. The encoded string actually translates to "Hello chatgpt how you are".

User also not quite, you were closer the first time

ChatGPT Thank you for letting me know. The correct decoded message for the given Base64 encoded string is "Hello chatgpt how are you?" I apologize for any confusion my earlier responses may have caused.

User can you now re-encode it into base64

ChatGPT Certainly! The message "Hello chatgpt how are you?" encoded in Base64 is:

SGVsbG8gY2hhdGdwdCBob3cgYXJlIHlvdT8=

I hope this helps!

I find it particularly odd that it was misspelling things in its decoding and adding punctuation where there was none. I suppose a LLM would be trained to do something like that, but the "chattpt" really threw me off. Also, it's base64 encoded message back to me decodes as "Hello chatgpt how are you?". Maybe not that impressive of a demonstration, but definitely not was I was expecting.

Yes, I would agree that technical and non-technical usability is one appealing spreadsheet use case. I think that another thing I find myself using spreadsheets for is simple data entry. By using a google sheet to enter data, you've already got yourself an authenticated, multi-user web form for easily entering data. It even integrates with google forms, which is a very handy tool for collecting data. It's certainly a lot easier than building it all yourself and hosting it.

Once the data is entered, there are much better tools for data visualization and aggregation (in my opinion). It's nice to have the "frontend" taken care of so that your time is freed up to analyze and extract meaning from the data. I'm speaking from the perspective of someone who does a lot of rapid prototyping and data analysis. There are likely use cases where this won't scale.

Is this something that the internet archive would preserve?

Can you elaborate a little here? I made the switch from classic notebooks to jupyterlab recently and find the bin experience very similar (there's a few subtle differences that I can't remember offhand, but I don't recall having any problems with cell navigation)

In my experience, there's a balance to be stricken. I really like notebooks for documenting the algorithm development process. I used to do a ton of repl driven development and Jupyter is a repl that allows you to persist commands across sessions. It saves a ton of time that I used to spend scrolling through the ipython history after closing and restarting the session. Jupyter also allows you to manage different kernels in the same environment, so it makes tasks like testing code between py2 and py3 trivial. My final point here is that there's also excellent cython integration, so you can do a lot of prototyping of cython code without having to mess with configuration or multiple files. I will agree that there are tasks better suited to IDEs, but jupyter is not just a plotting frontend, it can be used very effectively in algorithm development and the communication/documentation of the development thought process.

Typically, I have a git repo with the final code products, some of the more complex code gets written in notebooks, then transferred to git and thoroughly tested. I've been dreaming of this debugging experience in jupyter because that's still not a task that's suitable for notebooks, but I am hoping that it will come for vanilla python kernels before I can hope to adopt it.

Nanopore sequencers may require less prep and may be cheaper, but their sequencing error rate is astronomical, so you'd end up doing a lot more sequencing of the same material before you reached a consensus sequence.

Indeed, one could only imagine the cost of sequencing the dna to retrieve the data (not to mention the current lack of random access). Illumina's highest capacity sequencer will do 6Tb (terabases). The machine costs about half a million dollars and each run is tens of thousands of dollars, not to mention the lab costs of preparing/storing the dna. Additionally, the depth at which one would have to sequence to get _all_ of the data back reliably would be >1 meaning that every base would have to be sequence more than once (to avoid sequencing errors).

A big use case for python is glue for high performance c/c++ code. How is racket at this? Also, is there a cython alternative for racket? I would also argue that, though racket may have more batteries included, there are definitely not as many externally developed libraries. Especially important are numpy/scipy/pandas/pysam, the list could go on... That's not to say that I'm against racket, I really like the language and its level of design and documentation.

It would be nice if this language was extensible to running on various compute cluster managers. From what I can tell, these workflows only run on one machine. I like the bioinformatics tool examples though... you can tell who their target market is ;P

I've heard of it, not sure how mature it is these days. Maybe I'll give it another look