keizo

Glad to see others dual wielding: “I used to think that the idea of having one model review the work of another was somewhat absurd—it felt weirdly superstitious. The problem is it really does work”

interesting read albeit over my head, but i spent half of yesterday comparing Gemini Live (websockets) vs gpt-realtime-2 and while gpt is super good, seemingly more robust. Gemini connects faster.

Interesting! Like less overall engagement? Was it fully mixed or in distinct columns. I feel like it makes a difference.

I would at least interact with the new content if it was on the page. Vs almost never now. I assume people that are active on the new page must be 1% of users or mostly those directly involved with the story.

Anyway, thanks for the response and keep the place sane.

Grugnotes.com still my personal project after a few years. And lately how to make agents useful for notes.

This. Lately for some harder problems, I'll open two sessions. One writes a draft spec to a file. Then in the 2nd, i ask it to analyze, critique, etc. Often feed that response back to the first. A few ping pongs later, get a pretty polished plan. Open a new session to execute.

gemini flash and groq are pretty fast, and that part is streamable. curiosity got the best of me so i had claude code write a quick test. given this test is simply is 20 requests, with 1 second delay between requests ran once. so take with a grain of salt, but interesting still. Extra half second in a search is super noticeable so google looking like a reasonable improvement.

  OpenAI Statistics:

  - Average: 0.360 seconds
  - Median: 0.292 seconds
  - Min: 0.211 seconds
  - Max: 0.779 seconds
  - Std Dev: 0.172 seconds

  Google Gemini Statistics:

  - Average: 0.304 seconds
  - Median: 0.273 seconds
  - Min: 0.250 seconds
  - Max: 0.445 seconds
  - Std Dev: 0.066 seconds

  The key insights from these numbers:
  - Google has much lower standard deviation (0.066 vs 0.172), meaning more consistent/predictable performance
  - Google's worst-case (max) is much better than OpenAI's (0.445s vs 0.779s)
  - OpenAI had a slightly better best-case (min) performance (0.211s vs 0.250s)
  - Google's performance is more tightly clustered around its average, while OpenAI has more variability

has anyone done some simple latency profiling of gemini embedding vs open ai embedding api? seem like that api call is one of the biggest chunks of time in a simple rag setup.