gjreda
Joined 14 years ago
Joined 14 years ago
The easternmost portion of Northwestern's campus also has a nice walking/biking path along the lakeshore with a great view looking back towards the Chicago skyline.
I've mostly written technical, code-centric posts on Python, ML, and data science. Some of my early posts (2013) were wildly popular at the time and hit the top of HN and various subreddits.
I haven't written much recently, but I've been trying to branch outside of technical posts as I felt like my profession had started to become too much of my identity.
The post I'm most proud of:
- https://gregreda.com/2022/11/30/this-ones-for-me/ - Feeling pride and catharsis after years of bad health luck (leukemia, bad bike crash, cardiac arrest).
My most popular posts:
- https://gregreda.com/2013/03/03/web-scraping-101-with-python... - Web scraping tutorial using Python and beautifulsoup
- http://www.gregreda.com/2015/02/15/web-scraping-finding-the-... - Another web scraping tutorial with Python, but this time for sites that dynamically load content
- https://gregreda.com/2013/10/26/intro-to-pandas-data-structu... - The start of a series of posts on Python's pandas library
- https://gregreda.com/2013/07/15/unix-commands-for-data-scien... - Some useful unix commands for data processing
- https://gregreda.com/2015/08/23/cohort-analysis-with-python/ - Tutorial on doing cohort analysis using Python and pandas
- https://gregreda.com/2017/01/07/freelance-data-science-exper... - My experience as a freelance data scientist
- https://gregreda.com/2018/02/04/hiring-data-scientists/ - My approach to hiring data scientists (though my thoughts on this have evolved over the last five years).
Some context: I've been a jack-of-all-trades data scientist / machine learning engineer for the past 15 years (officially titled as an MLE the last four years).
I share that only because I think it plays a role in how I'm typically accustomed to working.
1. I found LangChain to be overkill for this use-case. While it might allow some to move more quickly when building, I found it to be cumbersome. My suspicion is this is largely because of my background - I understand how to build much of what's "under the hood" in LangChain. Because of this, I think it felt overly abstracted and I found the docs difficult to navigate and sometimes incomplete.
2. I used Qdrant via their docker image and it was simple to setup and start using. I didn't try to push the limits with it, so I can't say anything about performance. Because Qdrant runs as an http service, I found that it didn't fit well into my workflow - I'm accustomed to being able to visually inspect my data inside the interpreter, debugging, trying out commands, interacting and experimenting with my results, etc. Again, my suspicion is this is my own bias in how I typically work. Qdrant otherwise seemed very nice.
3. LanceDB felt powerful yet lightweight, and fit well into my workflow. It was far more intuitive for me. It was as if sqlite, the python data ecosystem, and a vector database had a child and named it LanceDB. Under the hood, it's built on Apache Arrow and integrates nicely with pandas, allowing me to seamlessly go from LanceDB table on disk, to pandas dataframe, and into some analysis or investigation of my LanceDB query results. This line [1] is a great example of why I liked it. This feels nicer to me than the world of API params and HTTP requests.
1. https://github.com/gjreda/scratch-pdf-bot/blob/main/gpt_pdf_...
I spent some time today exploring HuggingFace's Inference API but if the model is sufficiently large (> 10gb), HF requires you to use their commercial offerings.
I'm working on using it in a large project now.
Can you provide a source for this?
- https://www.cmegroup.com/trading/interest-rates/countdown-to...
I’d highly recommend the book The Philadelphia Chromosome if you’re interested in learning more.
> For restaurant inventory, we will rapidly expand our recent pilots of putting non-partnered restaurants on the platform. For reasons we’ve discussed many times, we believe non-partnered options are the wrong long-term answer for diners, restaurants and shareholders. It is expensive for everyone, a suboptimal diner experience and rife with operational challenges. With that said, it is extremely efficient and cheap to add non-partnered inventory to our platform and it can at least ensure that all of our current and potential new diners have the option to order from any of their favorite restaurants now, even if it’s not the best solution. By leveraging non-partnered options, we believe we can more than double the number of restaurants on our platform by the end of 2020.
> At the same time, because we know that partnered relationships are critical to the long-term success of this business, we will be investing aggressively in our independent restaurant sales organization to support converting as many of these non-partnered restaurants to partnered relationships as quickly as possible and to take advantage of other innovations in the restaurant space, like virtual restaurants.
https://s2.q4cdn.com/772508021/files/doc_financials/2019/q3/...