sgarland
Joined 6 years ago
Joined 6 years ago
I suspect that the average Cisco customer expects a bit more stability than $RANDOM_SAAS.
I don’t know how people who’ve worked with Python and been frustrated with its performance don’t know about Numba.
In contrast (I know this is apples to oranges), Python’s stdlib is incredibly broad. colorsys lets you convert between color spaces. plistlib lets you read and write Apple’s property list files. graphlib has a topological sorter, which can lead to hilariously short interview questions when the interviewer isn’t familiar with Python’s stdlib.
> Additionally, think resource constrained devices like mobile phones.
SQLite will saturate the phone’s I/O before you saturate SQLite.
> Or an aggressive cost reduction mentality where each cpu cycle shouldn't be unnecessarily wasted without feeling guilty.
> Our database Dip stores raw binary as well as json as first class citizens.
These two statements are in conflict. If you want to minimize CPU cycles, parsing JSON is a terrible idea, and you should be pushing everything into bitfields, operating on it with bitwise manipulations.
> If you come from a systems engineering background like that, you wouldn't be much impressed with many traditional databases at all. I am generally not much interested in SQL databases anyway.
I would love to see a Jepsen analysis of your DB. Then a benchmark of it vs. Postgres, or MySQL, or SQLite.
IMO, if you read the later post on Docstore [0], you can see that they spent an enormous amount of time building back everything that they gave up with Schemaless - the enforcement of a schema (the irony...), transactions, serializability, etc.
They needed to create secondary indices as tables because they were storing everything in JSON blobs, which is difficult to directly index now, let alone in the 2010s. You can of course argue that they needed to use JSON blobs because they kept needing schema changes, and online DDL back then was difficult. I would counter that if you keep needing schema changes, you haven't done a good job modeling.
Everywhere I have worked that has DBs they claim are too large, I have been able to prove (typically by cloning one and remodeling it) that the reason they're suffering from performance issues is because they've made a series of decisions which were fighting against their RDBMS: UUIDv4 PKs, strings for low-cardinality data, excessive secondary indices, poor query plans from the ORM, massive JSON blobs that's duplicating already-present data, etc. At my last job, I showed a 30% speedup purely from changing to a PK that made sense for the table's usage.
I am not asking for hand-holding, I’m asking for one single publicly-available example of this being done, because I cannot fathom it being necessary or beneficial, and you have not provided concrete proof.
You should, of course, test it on your own data before deciding.
Same with locks / contention: non-locking reads don’t cause contention, or at least, none that wouldn’t also occur with a separate table acting as the index.
Do you have any examples of this being used at scale, preferably with an accompanying blog post or other form of documentation?
email: <stephan@{username}.dev>
meet.hn/city/us-Charlotte
Socials: - github.com/stephanGarland
Interests: DevOps, Hardware, Databases, Open Source, Linux
---