kianN

Immutability and stateless separate the nrepl from jupyter. Clojure is not strictly functional but it tends to be the most straightforward way to do most tasks in the language.

This looks really cool! I’ve been really appreciating the revival of old-school server side rendering and foundational web components, but with modern tools or approaches.

The data labeling objection baffles me. Even if you don’t need labels for training, how do you know your model is working if you’re not evaluating it?

My company specializes in statistical long document text classification, but nowadays we mainly work with audit trail requirements because we got tired of hearing complaints about our 5 example learning curve. Seems like the industry standard is telling an llm to label and telling an llm to eval, and crossing your fingers that it’s correct.

It really is - we use duckdb as our main workhorse in our entire stack.

As a side note, thank you for all the work you have done in the Clojure ecosystem! Techascent’s data science packages was a major tipping point in my company’s decision to build our data science ecosystem in Clojure and has been wonderful to work with.

Happy to chat more in depth if more details would be helpful. I think my contact info is accessible from my HN profile.

To clarify, I wasn't criticizing your approach or product, more responding to the people dismissing the problem you are solving.

Regarding my experience, I have done a fair amount of work in the contact center space with long calls. I used statistical Bayesian approaches which I found to be much more resilient especially on long documents than embeddings/transformers. It also provided a joint modeling foundation for classification with much lower label requirements than BERT or traditional ML.

I see a fair number of comments here advocating for either codex to hand-roll this themselves, or to simply punt to SQL. I do want to advocate for the difficulty of the problem, even if I can't speak to the company itself.

At the scale of a few hundred to a few thousand documents, especially short documents, there are a few out of the box methods that can yield reasonable results, whether it be embedding clustering or leveraging LLMs for tagging.

However as your (1) datasets gets larger (2) documents expand from tweets and text messages to 30+ minute conversations and (3) you build downstream analytics on top of the learned semantic units, you really start to feel the limitations of LLMs and embedding for reliable annotation. That doesn't even get into the nuances associated with taxonomy management, seasonality, and model drift.

TLDR; this problem solved effectively has a lot of value and is a lot harder than it seems.

The author really extracted the core tenants of exactly how my former research mentor and I ended up building our business.

We started with the second two points: our core technology was a sampler that enables arbitrary hierarchical Bayesian graph models for sparse data, our constraint was cpu bound tractable compute. The piece that took us the longest to discover was the fact that our end products need to be separate from our underlying technology.

We were given that advice in various words from many people even before we started but some lessons need to be lived to be learned.