mierz00

If I am being honest, the value came from doing evals and testing against different models.

Essentially all I needed was a way to upload a data set, run tests against that data set and spit out a percentage of pass fail.

Braintrust makes this pretty easy, but If I was to do it again I would vibecode the same functionality.

I highly rate Braintrust.

It wouldn’t be too difficult to build something like that for your own usage, but I found it pretty easy to get datasets set up.

Essentially a game changer in understanding if your prompts are working. Especially if you’re doing something which requires high levels of consistency.

In our case we would use LLM for classification which fits in perfectly with evals.

“I admit that I still disagreed with him after the exchange, but I had a new respect for him as a designer because he was able to articulate a rationale for his decision.”

Any competent designer gets really good at justifying their decisions. Everyone has an opinion about design and thinks that their taste is correct.

I’m glad I don’t have to deal with that on the software side.

Aren’t people researching the companies they are applying for?

Also, I don’t think I have ever applied to a fake job.

I’m sure the welfare of the Iranian people is a top priority for Trump.

Marginally related, I feel the same way about honesty, especially in a work context.

I’ve always prided myself in being an honest but considerate person.

A recent experience with a colleague who weaponised my honesty in an attempt to manipulate me has left a foul taste in my mouth. Luckily their contract ended and the problem resolved itself.

But I remember distinctly feeling that I will be professional and polite but I do not automatically owe anyone my honesty.

Talk to people.

There are an infinite amount of problems to solve.

Deciding whether they’re worth solving is the hard part.

We analyse thousands of lines from a csv using an LLM. The only thing that worked for us was to send each individual line and analyse it one by one.

I’m not sure if that would work in your use case, but you could classify each line into a value using an LLM then hard code the trends you are looking for.

For example if you’re analysing something like support tickets. Use an LLM to classify the sentiment, and you can plot the sentiment on a graph and see if it’s trending up or down.

I’m really not sure I follow this argument.

A lot of software has friction to get to the value. This is often because of constraints not choice.

To give a concrete example of this, in my company we had users upload files for analysis. To get the export for the file, it took many steps. Not hard, but a lot to get done.

We switched it to an integration and now it’s 3 clicks. We’ve gone from 10% of users onboarding to 100%.

It doesn’t mean we get people to stay, but the barrier to understanding if our tool provides value to them has completely disappeared.

I’m very curious though, what value did you strip away when trying to make your product easier to use?

This also goes the other way too, you don’t assume the original is incorrect.

I see this a lot with developers who come in and start to criticise before understanding.

There is always a reason for why something is as it is, and it’s unlikely that the people before you were just idiots.

How do you introduce any tool/change to a team of people?

You get buy in, start having conversations see what AI people have explored. Have they tried claude? Do they prefer other tools? If so why? What are the objections. Actually listen. I’d also showcase what you can do. I love to present what codex has found when debugging something, or a prototype I’ve put together.

If you have the budget pay for subscriptions so they can play around.

Also, you say that development velocity is a big problem, but I would dive into why that is. You may be disappointed when velocity remains the same with AI tools.

If you focus on the differences that’s all you’ll see.

The opposite applies too, if you try hard enough you’ll find plenty of similarities with an Eskimo. We’re all just people in the end.

I didn’t even realise that was a possibility with iOS. Thanks!

I’m curious have you ever been burnt or seen anyone burnt by copyright infringements in code?

Sometimes it’s super obvious because a game company steals code from a previous employer, but I have never seen this play out in entreprise software.