mleonhard

Joined 18 years ago

Karma
4,067
Submissions
1,635
Member since
Feb 26
Michael Leonhard. San Francisco. Software engineer and entrepreneur. Interested in many things.

[email protected]

http://www.tamale.net/

View on Hacker News

Recent submissions

Comment · 20 days ago
https://cloudping.info/ measures latency to multiple major cloud providers. I built it after leaving AWS. I plan to keep it running for many years.

Like your gcping.com, I originally named mine awsping.com. Then someone on HN informed me that the name is forbidden by AWS's rules, so I renamed it [0].

[0] https://news.ycombinator.com/item?id=2288250

Comment · 3 months ago
The problem is that the system is RAM-constrained but the work scheduler doesn't use a RAM pressure signal.

We could solve this with a crate that rate-limits new tasks based on heap usage.

We could build this into the runtime and make it apply to all task spawns. That could be problematic since any spawn could block.

It would probably be better to make it opt-in and less granular, releasing groups of tasks at a time.

Comment · 3 months ago
It's hard to tell how much care was put into futures_orchestra crate. The docs are generated. I don't see any benchmark in the repo. Performance claims without measurements are a red flag.
Comment · 7 months ago
> Inexplicably, major advances in this area 15-20 years ago under the auspices of government programs never bled into the academic literature even though it materially improved the situation.

Would you please share some more info about this? Were the advances implemented in software and never written up and published? What are the names of the government programs?

Comment · 9 months ago
s/Prolificacy/Verbosity/
Comment · 9 months ago
Over-exaggeration also works well when learning to play stringed instruments like cello.
Comment · 11 months ago
Comment · 1 year ago
> // Start a background task that takes the lock and holds it for a few seconds.

Holding a lock while waiting for IO can destroy a system's performance. With async Rust, we can prevent this by making the MutexGuard !Send, so it cannot be held across an await. Specifically, because it is !Send, it cannot be stored in the Future [2], so it must be dropped immediately, freeing the lock. This also prevents Futurelock deadlock.

This is how I wrote safina::sync::Mutex [0]. I did try to make it Send, like Tokio's MutexGuard, but stopped when I realized that it would become very complicated or require unsafe.

> You could imagine an unfair Mutex that always woke up all waiters and let them race to grab the lock again. That would not suffer from risk of futurelock, but it would have the thundering herd problem plus all the liveness issues associated with unfair synchronization primitives.

Thundering herd is when clients overload servers. This simple Mutex has O(n^2) runtime: every task must acquire and release the mutex, which adds all waiting tasks to the scheduler queue. In practice, scheduling a task is very fast (~600ns). As long as polling the lock-mutex-future is fast and you have <500 waiting tasks, then the O(n^2) runtime is fine.

Performance is hard to predict. I wrote Safina using the simplest possible implementations and assumed they would be slow. Then I wrote some micro-benchmarks and found that some parts (like the async Mutex) actually outperform Tokio's complicated versions [1]. I spent days coding optimizations that did not improve performance (work stealing) or even reduced performance (thread affinity). Now I'm hesitant to believe assumptions and predictions about performance, even if they are based on profiling data.

[0] https://docs.rs/safina/latest/safina/sync/struct.MutexGuard....

[1] https://docs.rs/safina/latest/safina/index.html#benchmark

[2] Multi-threaded async executors require futures to be Send.

Comment · 1 year ago
I spent some years working for a large NGO (Opportunity International) and living with people who work for NGOs.

NGOs must constantly raise money to fund their operations. The money that an NGO spends on fund-raising & administration is called "overhead". The percentage of annual revenue spent on overhead is the overhead percentage. Most NGOs publish this metric.

When a big donor stops contributing, the NGO must cut pay or lay off people and cut projects. I've never heard of an NGO "succumbing to excessive staff costs" like a startup running out of money. Financial mismanagement does occasionally happen and boards do replace CEOs. Board members are mostly donors, so they tend to donate more to help the NGO recover from mismanagement, instead of walking away.

NGOs pay less than other organizations, so they mostly attract workers who care about the NGO's mission. These are people with intrinsic motivation to make the NGO succeed in its mission. Financial incentives are a small part of their motivations. For example, my supervisor at Opportunity International refused several raises.

> So they go around addressing individual problems, taking sad pictures, and avoid addressing systemic problems.

Work on individual problems is valuable. For example, the Carter Center has prevented many millions of people from going blind from onchocerciasis and trachoma [0].

The Carter Center is not directly addressing the systemic problems of poverty and ineffective government health programs. That would take different expertise and different kinds of donors.

The world is extremely complicated and interconnected. The Carter Center's work preventing blindness directly supports worker productivity in many poor countries. Productivity helps economic growth and reduces poverty. And with more resources, government health programs run better.

Being effective in charity work requires humility and diligence to understand what can be done now, with the available resources. And then it requires tenacity to work in dangerous and backward places. It's an extremely hard job. People burn out. And we are all better off because of the work they do.

When we ignore the value of work on individual problems, because it doesn't address systemic problems, we practice binary thinking [1]. It's good to avoid binary thinking.

[0] https://en.wikipedia.org/wiki/Carter_Center#Implementing_dis...

[1] https://en.wikipedia.org/wiki/Splitting_(psychology)

Comment · 1 year ago
When I used AWS startup credits in 2019, the AWS console made it very difficult to estimate the bill after the credits ran out. I lost a lot of trust in AWS. Also, there were buried mines in the APIs, like the risk of bad logging running up a $70,000/day bill with CloudWatch Logs.

If I could go back and do it again, I would rent a single machine and deploy with ssh (git pull & docker-compose up) and backup to my laptop.

Comment · 1 year ago
I think that async in Rust has a significant devex/velocity cost. Unfortunately, nearly all of the effort in Rust libraries has gone into async code, so the async libraries have outpaced the threaded libraries.

There was only one threaded web server, https://lib.rs/crates/rouille . It has 1.1M lines of code (including deps). Its hello-world example reaches only 26Krps on my machine (Apple M4 Pro). It also has a bug that makes it problematic to use in production: https://github.com/tiny-http/tiny-http/issues/221 .

I wrote https://lib.rs/crates/servlin threaded web server. It uses async internally. It has 221K lines of code. Its hello-world example reaches 102Krps on my machine.

https://lib.rs/crates/ehttpd is another one but it has no tests and it seems abandoned. It does an impressive 113Krps without async, using only 8K lines of code.

For comparison, the popular Axum async web server has 4.3M lines of code and its hello-world example reaches 190Krps on my machine.

The popular threaded Postgres client uses Tokio internally and has 1M lines of code: http://lib.rs/postgres .

Recently a threaded Postgres client was released. It has 500K lines of code: https://lib.rs/crates/postgres_sync .

There was no ergonomic way to signal cancellation to threads, so I wrote one: https://crates.io/crates/permit .

Rust's threaded libraries are starting to catch up to the async libraries!

---

I measured lines of code with `rm -rf deps.filtered && cargo vendor-filterer --platform=aarch64-apple-darwin --exclude-crate-path='*#tests' deps.filtered && tokei deps.filtered`.

I ran web servers with `cargo run --release --example hello-world` and measured throughput with `rewrk -c 1000 -d 10s -h http://127.0.0.1:3000/`.

Live from the official Hacker News API

A modern reader for Hacker News · SvelteKit + Skeleton