benrow

Have a look around for token biasing, or green lists. It's based on a nudge to the choice of the next token (which can always be drawn from a set of possibilities which are all probable enough).

At first I thought this approach was just the "LLM flavour" of writing, but it's way more subtle, especially as the bias is applied uniquely for each token position.

I've heard that this kind of watermarking process works by biassing the statistical sampling towards a partition of the set of possible next tokens (red set and green set), at each position. It might only be a slight nudge each time, but over a sequence of tokens, the likelihood of repeating the bias by chance is increasingly improbable.

The bias is different for each position and follows a defined RNG, seeded somehow predictably.

Can be either an open algorithm, or not. If not open, then an API could be provided to determine if text is watermarked or not.

How it applies to code - maybe it could be a subtle nudge to symbol names, etc, I'm just speculating (I only read about this in passing very recently).

You had keyboards? Luxury, back in my day we had to stamp punch cards with the tips of our fingers. If we made a mistake, the head operator would cut one of our fingers off with a bread knife. We didn't mind though, it made us tough, that and sleeping in the car park wrapped in tape from old backups just t'stay warm.

I didn't get very far into AoC this year as I ran out of time. Maybe I'll pick it up again later.

But my point is, I was surprised at how hard day 5, part 2 was. I didn't give up and solved it, but went away wondering whey I'd missed something obvious and overcomplicated it. So it brings some relief to know it was 'supposed" to be a bit challenging!

My first thought was that the pilots and crew would lose their jobs if they refused to fly.

But perhaps pilots would be less likely to get rated for Max aircraft if (hypothetically) there's a perceived broad issue with their safety? This could cause those who have a choice (stronger pilots) to go airbus.

Not saying it's the case, just pondering here.

I still prefer 200 with an empty list for the 'no results' case. The same client code works, rather than having to code for 204.

- If I query for an identifier which doesn't exist - Server replies 404, this should fall into my error handling as there's a data inconsistency

- If I query say like ?category=automotive&price=1 and I get a 200 containing a json body in which there's an empty list of matches, then my client code can handle 0 matches as well as 1 or 10 with no special handling.

I like the convention:

- 404 if resource requested by id

- 200 with empty list of results if it was a 'search' type request with params (not referring directly to an id)

I was looking for this comment. I'm very happy with the combo of IntelliJ features and vim movement commands.

The only inertia vim adds to my workflow is escaping into command mode. I have a 'jk' shortcut combo rather than escape, but if I'm hammering away I often mistime it and need to backspace out my jjkk or whatever.

We ran into a problem where we blew the direct memory limit recently.

To trace the problem, I added a breakpoint on ByteBuffer.allocateDirect() - this is how DM is normally allocated. But rather than suspending the process, we set it to print the stacktrace and continue.

This gave us handy output of possible causes. Focusing on the cases which were called from our own code (likely to be the root cause), we found a call to Files.readAllBytes() - which immediately sounds suspicious to cause a memory leak. Indeed this was the culprit, and explained why the problem was sporadic and hard to reproduce (there is some kind of per-thread caching behaviour of direct memory I never did understand properly).

Incidentally, Files.readAllBytes() uses direct memory because it uses an area of memory which both the OS can access (onto which to read the file contents) and which also Java can access (for the application code, rather than copying to the heap, which could be inefficient for large files).

Fortunately, this "by inspection" approach got us to the answer.

Others have stated profiling would work well. We'd have needed a local reproduction of the issue, which wasn't available at the time (could not reproduce without realistic workloads). Otherwise we could have used our SaaS APM software which runs in deployed envs, but that takes a sampling approach and doesn't really dig deep enough for this kind of stuff.

Regression is a good analogy of the problem here. If you found a line of best fit for some datapoints, how would you get back the original datapoints, from the line?

Now imagine terabytes worth of datapoints, and thousands of dimensions rather than two.

Could still happen though unfortunately, as many bought at ultra low interest rates. Those fixes will be coming to an end in a year or two. At least in this country we tend to have fixes of 1-5 years.

Also curious.

I'm a sample of 1 and also relatively inexperienced. But I felt I quickly reached the limits of what was possible when I tried doing sentence classification with OSS sentence embedding models. The issue was with negation. I'd attributed too much magic to embedding models - they don't really understand language.

Not to say there isn't very capable tech out there. Just to add a datapoint that "sentiment analysis"-like approaches in blogs don't always scale to your particular use-case.

Edit: conscious I've drifted from the topic of chatbot type models, but felt relevant somehow.

I wrote a subtractive synth in java for my masters project. It was a fascinating domain to learn. For example, how to manage the independent voices, wavetable interpolation, envelopes. I was probably in over my head, and if I recall correctly, my supervisor was warning me about the complexity, since it was a software engineering programme.

The filter was the weirdest, least intuitive part. I ended up porting some open source code to java. This eventually worked, but was essentially impossible to debug without some sort of software oscilloscope.

That was about 18 years ago (yikes), and I still think about DSP from time to time.

The best intuition I have of low pass filters is to imagine it like an averaging, smoothing function which operates on the sample values as a sliding window. But that's not really true - it's not a straight rolling average, instead the sinc function (sin x / x) is used to scale the sample values with respect to time.

The way FIR filters work assumes a finite amount of samples (otherwise it would be technically impossible to know all past and future samples), so instead you pick a time period over which to calculate the output. Since there are always past and future samples included, this leads to a delayed output. Eg for a 100 sample filter window, you'd need to first have 100 samples in order to calculate the first output sample. This is intuitive, since a filter is sort of a smoothing, averaging function.