aslushnikov
Joined 12 years ago
Karma
68
Submissions
25
Member since
Apr 7
Recent submissions
Comment · 16 days ago
We're computing sliding window aggregations to detect regressions on a history of commits in Flakiness.io.
Our current algorithm is suboptimal, so we had to limit the "lookback": a test failure is marked as a regression iff it had a perfect history for the last 10 observed commits.
While the current approach works reasonably well, we had a few requests to make lookback larger/configurable, which we now might be able to do with this algorithm.
Thank you for sharing!
Comment · 1 month ago
https://flakiness.io: test analytics for GitHub and GitLab.
The idea is to connect test results and artifacts with commit history and testing environment. This way test reports can show whether a failure in a pull request is a real regression or a known flake from a target branch.
The core of the service is a specialized analytics engine that makes it possible to store and query large amounts of test results [1]. Thanks to this engine, we've processed 2.5 billion test results to date.
Flakiness.io is used already by a few large open sources (nuxt/nuxt [2], wordpress/gutenberg [3]). There's a 1GB free plan which is enough for roughly 10M+ test results, and I'd be happy to increase this limit to 100GB for large open source projects upon request.
Let me know what you think about the project!
[1]: https://blog.flakiness.io/posts/2026/engine/
Comment · 2 months ago
I've been working on shard balancing for Playwright Test. The idea is to use the new API released in Playwright v1.62 together with historical test durations to balance tests across shards.
I ended up designing a specialized variant of LPT that works well for Playwright test suites. It's now used in WordPress's Gutenberg project, where it cut test runtime from 35 minutes to 23.
The algorithm is called SALT. Here's the detailed write-up if you're interested in job scheduling:
Comment · 3 months ago
Hi HN, I’m Andrey. I previously worked on Puppeteer and Playwright; for the last couple of years I’ve been building Flakiness.io, leading a small bootstrapped team.
Flakiness.io collects test results from CI, associates them with GitHub commits and pull requests, and cross-references them with Git history. This way, a test report can show whether a failure is new, whether a test has been flaky before, and which commit caused a test to start running 5x slower.
We built integrations for several test runners, along with an open JSON report format that we use to represent concepts shared across different runners: tests, tags, annotations, retries, steps, attachments, logs, and more.
https://github.com/flakiness/flakiness-report
Supporting large projects was an interesting engineering challenge: some repositories generate hundreds of thousands of test results per commit, but the data is very repetitive. Our experiments showed that it can be compressed and analyzed without decompression, so we ended up with a specialized analytics engine that can process billions of results on modest hardware while keeping the service affordable.
The report format, SDK, and runner integrations are MIT-licensed. The server is closed source, which is how we plan to make money. To avoid locking users in, our CLI [1] can export all of your data so you can process it however you want.
Here is a public project to explore:
https://flakiness.io/wordpress/gutenberg
We’d love your feedback! What's missing for Flakiness.io to fit your test setup?
Comment · 3 months ago
Thank you for pointing!
Comment · 3 months ago
This reminds me of https://mc-stan.org
Comment · 5 months ago
Working on https://flakiness.io: GitHub-native test analytics. I worked on Playwright before, and this project started as a natural continuation of that work. We’ve since expanded beyond Playwright to support many popular test runners.
The idea is to connect test results and artifacts with commit history. Test reports should know whether a failure is new, whether a test has been flaky before, and which commit made a test start running 5x slower.
If you maintain an open-source GitHub project with tests, please give it a try. The free plan is a good fit for OSS projects: 1GB of storage is enough for roughly 10M+ test results.
Comment · 6 months ago
I'm very surprised to see the rapid advancement in robotics these days. After all the fancy demos of Boston Dynamics and others from 10 years ago, and no real advancement beyond them, we kinda learned to treat robotics as "fancy toys".
Now, this feels to me very much like a Deep Blue moment in chess, when to everyone's surprise it won over Garry Kasparov 3.5 to 2.5. 20 years in, and no one even considers competing with chess engines.
This Ace robot won over table tennis professionals in 3 matches and lost in 2. Even the score is similar. I wonder what it'll all look like in 20 years from now.
Comment · 6 months ago
The setup I'd recommend:
- Velocidrone [1] flight sim
- Any FPV controller that connects to laptop. 2 solid choices are TBS Tango 2 or DJI FPV Remote Controller [3]
I spent ~20 hours in the sim before advancing to real drones; a few of my friends followed this path and successfully passed an improvised exam on real drone after just ~10 hours in the sim.
As for the drone itself, the easiest setup is probably DJI Avata [4], but it's less of the proper "FPV feel". I personally fly Flywoo Explorer [5] with DJI system: it's a small & nibmle long range drone, easy to travel with, and powerful enough to chase kiteboarders even in a strong wind.
P.S. Don't be discouraged with sim flying: it's actually very fun, feels similar to TrackMania Nations.
[1] https://www.velocidrone.com/
[2] https://www.team-blacksheep.com/products/prod:tbs_tango_2
[3] https://store.dji.com/product/dji-fpv-remote-controller-3?vi...
[4] https://www.dji.com/avata-2
[5] https://flywoo.net/products/explorer-lr-4-o4-pro-sub250-4k-1...
Comment · 6 months ago
I'm very much into niche hobbies: they usually have nice tight & friendly communities.
Below are some of my favorite I'd love to share:
- FPV drone flying: once you've spent 5-10 hours to get initial reflexes for the controls in the simulator, the first flight on a real machine outside feels magical.
- Electric unicycles: the "mind-controlled" PEV, and arguably the best way to get around in San Francisco.
- Foiling: the closest feeling to riding a hoverboard. You can kite-foil, pump-foil, sup-foil etc, but wing foiling is the easiest to get started.
- Knots: tying laces properly just makes life easier, and tying tucker's / voodoo hitches for the first few times feels like a magic trick.
- Cardistry: learning to do a proper riffle shuffle and a few artistic cuts adds some fun to the most boring part of any card game.
Comment · 2 years ago
This is awesome! I enjoyed typing out the 20 words available, and would love to type more! Why are the majority of the words in the word list [1] commented out?
Also, I would appreciate if the word & its description is automatically pronounced (or maybe there's a setting to do so). This way I would be able to train audio perception, which is oftentimes challenging with French.
Right now, clicking the "speaker" icon speaks out the word, not definition, and also steals focus from the input field.
1: https://github.com/agencyenterprise/Term-Typer-Words/blob/ma...
Founder @ Flakiness.io, former engineer @ Microsoft Playwright, former TL @ Google Puppeteer
[ my public key: https://keybase.io/aslushnikov; my proof: https://keybase.io/aslushnikov/sigs/nayDSNJ7x7GBsQ71hCvh1CVfGt1beCjHz474qDT8qx4 ]