kflansburg
Joined 7 years ago
Karma
72
Submissions
26
Member since
Apr 1
Recent submissions
Comment · 5 days ago
At no point do we claim that it is a bug with the model. We specifically outline how it was a mis-configuration on Cloudflare's part and a bug in SGLang. We name the model in the title because our customers need to know which of our endpoints was affected. In fact, we say "the GLM-5.3 Workers AI model", not "GLM-5.3".
Comment · 1 month ago
> Our post-training utilizes label-smoothed cross-entropy for valid schema outputs paired with a Brier loss to refine probability calibration.
Comment · 1 month ago
Some users on X [1] were discussing the use of DiffusionGemma for Jev-like use cases. Similar to Jev, diffusion models generate output fully in parallel. We achieve similar estimation of relative confidence by looking at token log probabilities. Finally, by disabling reasoning we achieve low output tokens and similar latencies. This demo lets you compare the output to Jev for some canned prompts, or see the DiffusionGemma output for your own prompts.
[1] Original X post with vLLM PR https://x.com/mmastrac/status/2100626193943052784
Comment · 5 months ago
My point was not about our ranking specifically, but the methodology of taking a point-in-time sample.
Comment · 5 months ago
I would recommend tracking this data over time. I work on Cloudflare's KV cache for Kimi K2.6, and while there are periods where our cache rate is low, we are frequently in the 80-90% range. OpenRouter shows us at 87.3% at the time of this post. We observe cache rates change quite a bit from hour to hour.
Comment · 5 months ago
If you aren't already aware, Karpathy has several videos that could get you there in a few hours https://www.youtube.com/@AndrejKarpathy
Comment · 6 months ago
Cloudflare | Systems or ML Engineer, Workers AI | Austin, TX or London, UK or San Francisco, CA (Hybrid) | Full-Time | https://cloudflare.com
Cloudflare is building across the entire AI stack. Here are some exciting things that we have launched recently:
- Kimi k2.5: https://blog.cloudflare.com/workers-ai-large-models/
- Kimi Performance Improvements: https://blog.cloudflare.com/high-performance-llms/
- Kimi k2.6 Speculative Decoding and Shared KV Cache: https://x.com/kevin_flansburg/status/2050238819065299110
- Unweight Tensor Compression: https://blog.cloudflare.com/unweight-tensor-compression/
- Code Mode MCP Servers: https://blog.cloudflare.com/code-mode/
- Agents SDK: https://blog.cloudflare.com/building-agents-with-openai-and-...
- Agent Memory: https://blog.cloudflare.com/introducing-agent-memory/
- Internal AI Tooling: https://blog.cloudflare.com/internal-ai-engineering-stack/
- Release Cog v0.19: https://github.com/replicate/cog/releases/tag/v0.19.0
- Dynamic Worker Sandboxes: https://blog.cloudflare.com/dynamic-workers/
- Dynamic Workflows: https://blog.cloudflare.com/dynamic-workflows/
We are looking for systems and ML engineers to help build our edge inference platform:
- Senior Systems Engineer, Workers AI - https://job-boards.greenhouse.io/cloudflare/jobs/7764827?gh_...
- Senior / Principal Machine Learning Engineer, Workers AI- https://job-boards.greenhouse.io/cloudflare/jobs/6297179?gh_...
Comment · 7 months ago
Cloudflare | Systems or ML Engineer, Workers AI | Austin, TX or London, UK or San Francisco, CA (Hybrid) | Full-Time | https://cloudflare.com
Cloudflare is building across the entire AI stack. Here are some exciting things that we have launched recently:
- Release Cog v0.17: https://github.com/replicate/cog/releases/tag/v0.17.0
- Dynamic Worker Sandboxes: https://blog.cloudflare.com/dynamic-workers/
- Kimi K2.5: https://blog.cloudflare.com/workers-ai-large-models/
- Code Mode MCP Servers: https://blog.cloudflare.com/code-mode/
- Agents SDK: https://blog.cloudflare.com/building-agents-with-openai-and-...
We are looking for systems and ML engineers to help build our edge inference platform:
- Senior Systems Engineer, Workers AI - https://job-boards.greenhouse.io/cloudflare/jobs/7764827?gh_...
- Senior / Principal Machine Learning Engineer, Workers AI- https://job-boards.greenhouse.io/cloudflare/jobs/6297179?gh_...
Comment · 1 year ago
I see now that this incident happened in September 2024 as well.
Comment · 1 year ago
Yes, I've already performed the upgrade for my projects, but since they hit this bug, I'm guessing they haven't.
Comment · 1 year ago
> an if let expression over an RWLock assumed (reasonably, but incorrectly) in its else branch that the lock had been released. Instant and virulently contagious deadlock.
I believe this behavior is changing in the 2024 edition: https://doc.rust-lang.org/edition-guide/rust-2024/temporary-...
Comment · 1 year ago
I'd love to see this comparison between Garmin and Oura. I ditched Whoop because it was too easy to get 100 sleep scores, but my Garmin watch is much harder to please. I do think it offers better signal, but I've never scored over a 90, so maybe it is too critical?
Comment · 1 year ago
All of the JS features are available in Rust, but some don’t have a first-class SDK API yet and you must use wasm-bindgen.
Formerly Rust @ Apple, Cloudflare Workers
kflansburg.com