hyc_symas

> A power outage interrupted Garage mid-write on the LMDB metadata database. LMDB uses mmap’d I/O and, by default, operates with MDB_NOMETASYNC and MDB_NOSYNC (Garage sets metadata_fsync = false by default)

No, Garage disables LMDB's safety mechanisms by default. By default, LMDB performs whatever necessary syncs and is 100% crash proof, even in the face of power outages. Garage's problems are of their own making, not because LMDB has unsafe defaults.

The txnID was added to the page header to enable support for incremental backup. As a consequence, it's sufficient to compare a page's txnID to the current txnID to know if it's dirty or not, and spilled pages don't need a cleanup pass to clear their dirty bit on commit so commits of large txns are faster now.

write/fsync can be faster in a large dataset because writes let the filesystem know an explicit list of dirty pages, so fsync only needs to deal with them.

mmap/msync gives no hints about which pages are dirty (unless the app tracks them itself and msyncs them individually, which would completely defeat any reduced syscall advantage of using a writable mmap in the first place) so the entire map must be scanned for dirty pages.

In practice, the expected performance advantages of using a writable mmap just aren't there, and coupled with the ease of silent corruption, it's best to never use that approach.

By the way, you're wrong on both points - the cache of page mallocs is not infinite, and it does spill dirty pages to disk when necessary. And the latter is what bounds the number of malloc'd pages.

You're dreaming. None of your explicit memory control operations mean anything in practice, because today everything runs in VMs with no actual control of the underlying hardware. Probably co-resident with an unknown number of other tenants.

As for what you claim the paper's authors were saying - I quoted their text verbatim. Your interpretation is not what they said.

They claimed using mmap safely is impossible, and using it correctly requires more complexity than a traditional DB design. The safety claim was already disproven by multiple researchers. To prove their second claim they would have had to produce a DB that did traditional buffer management and was simpler and more performant than using mmap. They never did any such thing, nor could they.

In the time it takes for your query optimizer to dissect a query and "look ahead" LMDB would have already answered a million queries. You think your magical "future oracle" is zero cost? How many KLOCs is it? LMDB's hot paths fit entirely inside a CPU's L1 cache.

Indeed, there's no need to use the doc website. There's nothing there that isn't embedded in the LMDB source code. All of the docs are generated from doxygen comments in the source.

If you're downloading binaries from a plaintext documentation site, I think that's on you.

Nonsense. The best you will ever do, even with full application knowledge and complete control of the machine, is an LRU cache replacement algorithm. But when you do it yourself you have to juggle the fine details of which indices to prioritize, and you will never get it perfect. If you're not running a dedicated machine, as soon as any other processes run all your careful tuning goes out the window.

Since LMDB manages multiple tables as a tree of trees, no fine tuning is needed. The internal paths to every hot page automatically take priority, regardless of which index or how large each index is. So a simpleminded LRU always makes optimal use of available cache, regardless of access pattern or other load on the system.

Irrelevant, since LMDB doesn't dirty the pages in the mmap. The mmap is read-only.

LMDB 1.0 no longer uses a P_DIRTY flag, it no longer has to explicitly mark pages as clean.

> The price is intended to decrease.

No. Monero's tail emission rate was specifically chosen to be less than the rate of global gold production. Do you claim the price of gold is intended to decrease?

A continuous emission like Monero's doesn't equate to inflation/devaluation. It allows its userbase to expand organically, without artificial scarcity pumping its price.

I don't know what the current versions do, it's been a while since I touched that code.

I have no reason to lie, I'm not selling anything. Bitmain is selling mining hardware, take a look at their claims. They've had 7 years to try to crack it.

FPGAs have no particular advantage. You could dedicate a chunk of their resources to implement a softcore CPU but it'd be several times slower than a real CPU.

The random programs change too quickly to just implement them directly on an FPGA. Reprogramming the entire chip like that takes too much time.