akbo

Joined 7 years ago

Karma
22
Submissions
14
Member since
Feb 21
Freelance Data Scientist - https://haveagreatdata.com/
View on Hacker News

Recent submissions

Comment · 6 years ago
Comment · 6 years ago
I have a different perspective. In a drug trial you have to prove that a new drug is safe before handing it out to a bigger number of people. The default case (if you don't test the drug) is that nobody gets it.

In this case, it is different. People will put pressure on the government to allow concerts again rather sooner than later and the default case is probably that there will be concerts again before the Corona situation is over. If you don't do the test, you have to make the decision whether / when to allow concerts again blindly.

So basically you have the situation where people will take the new drug anyway and you have the choice whether you want to test its safeness before they do.

If you do the test and it turns out to be unsafe, at least you have good arguments when you need to make the decision.

Comment · 6 years ago
Comment · 6 years ago
I love it! What I always missed in, e.g., matplotlib, is the interactivity. I need to be able to zoom in and out without changing code. Hover information is another great feature coming to mind.
Comment · 6 years ago
Comment · 6 years ago
Comment · 6 years ago
I see where you're coming from.

My use case is this: as a data scientist, I start new code bases all the time. Each project, simple experiment, data analysis, etc. needs its own cleanly separated dependency environment so I don't end up in dependency hell (I have 12 conda environments on my machine right now). Conda allows me to handle these environments with ease (one tool and a handful of commands -> as detailed in the article). With conda, I also have my data science Python cleanly separated from my system Python.

Of course there are other tools that can handle this use case. But pip alone won't do the trick. I don't like to have three separate tools for this (pip + venv + pyenv).

When I put something into production, I naturally want to keep using my conda environment.yml and have the same environment in dev and prod instead of switching to pip + requirements.txt, which might introduce inconsistencies.

Comment · 6 years ago
Comment · 6 years ago
Comment · 7 years ago
> Also, I'm not sure why you called CSV unsafe.

One example of it being unsafe that happened to me: I got a CSV file written by a program with a broken implementation of a CSV writer that didn't quote string fields when there was a newline in them (in my case only the first half of a newline: carriage return). Then I read the file with a broken implementation of a CSV reader that assumed that the carriage return meant a new record and filled both parts of the broken line with N/As instead of throwing an error. This way the data in the sink didn't match the data in the source. This is the loss of data integrity, which I would call unsafe. It doesn't happen if you have a file format that serializes your data safely.

Due to the format being underspecified, many people roll their own unsafe CSV writer or CSV reader, thus every CSV file (where you don't completely control the source) is potentially broken.

Edit: Browsing your Github account I found that you implemented a CSV parser in Rust. I didn't know that when I wrote the above comment, so I was definitely not trying to imply that your particular CSV parser is unsafe.

Comment · 7 years ago
Comment · 7 years ago

Live from the official Hacker News API

A modern reader for Hacker News · SvelteKit + Skeleton