scriptsmith

Has their reputation tanked so much that the alternatives get all the buzz? Or is it that non-enterprise users are priced out by the usage costs, so no free marketing?

Comment · 4 months ago

Definitely took longer than I was expecting, then after two weeks I thought we'd never get it.

Comment · 4 months ago

And now is this going to be a one-off, or routine with every new generation of models?

Comment · 5 months ago

It's based on a Gemma 3n model, and yeah it's not the best. But if you have a use case that needs constrained JSON output for example, it's pretty neat.

Maybe it would do better with the new Gemma 4 models, which the Chrome devs have been hinting at moving to. And why the API doesn't let you introspect / pick the model, I'm still not sure.

Comment · 5 months ago

I've got some demos of what the new Prompt API in Chrome that uses a local model can do: https://adsm.dev/posts/prompt-api/#what-could-you-build-with...

As OP says, it shines in constrained environments where the model is transforming user-owned data. Definitely less useful for anything more open-ended.

Comment · 5 months ago

Chromium mostly does not support this, because it doesn't have the binary blob required to run the inference. However, it does still download the model weights and expose the LanguageModel API, because that part is hooked up.

https://adsm.dev/posts/prompt-api/#which-browsers-support-th...

Packagers might eventually disable that but I tested this behaviour in chromium 148 a few hours ago, and it would download the weights but has trouble running them.

Comment · 5 months ago

> My understanding is that you’ll have to explicitly agree to download alternate models in the future, per the specification

I don't get that from the spec. Closest thing I could find in the Writing and Assistance APIs spec [1] (which the Prompt API refers to) says:

> [It] allows the user agent to prompt the user for permission

But that's talking more about initiating the 'download' process more than choosing the model.

I think Chrome wants the browser to be 'opinionated' about the model used however, rather than letting the webpage pick, at least for now.

[1] https://webmachinelearning.github.io/writing-assistance-apis...

Comment · 5 months ago

Depends on where you get it. By default the flags will be enabled, but some packagers may choose to disable them. I haven't seen a major distro release chromium 148 yet.

Weirdly though, chromium won't be able to actually use the model even though it can download it, because the inference engine is a closed-source blob.

https://adsm.dev/posts/prompt-api/#which-browsers-support-th...

Comment · 5 months ago

I wrote a more detailed blog post here:

https://news.ycombinator.com/item?id=48028662

Comment · 6 months ago

Chromium doesn't support this API because it needs a binary blob to run the inference, although in theory it may still be configured to download the weights:

https://adsm.dev/posts/prompt-api/#which-browsers-support-th...