ivansmf
Joined 12 years ago
Karma
53
Submissions
18
Member since
Mar 20
Recent submissions
Comment · 2 days ago
Whatever you think of the original discussion, I want to share two lessons I learned, in the hopes they'll help someone: (1) never wrestle a pig. You both end up covered in mud and the pig likes it; and (2) if you're trying to teach or share, argue for the audience. If you try to level with the antagonistic person, people watching from outside will have a hard time differentiating you from them. If the response is not about helping others, well, my sarcastic side will advise you to argue as much as possible with strangers on the Internet. That's what life is for.
Comment · 4 months ago
The technology is amazing. The marketing around it is a decade ahead of its capability, and the pushiness to make LLMs do what they still can't is just irritating. The question to me is "who is seeing the goal posts?" And the answer is "the marketing department of whoever sells it".
Comment · 5 months ago
The article severely underestimates deployment times for large, world wide services. Usually the strategy is to have a smaller "blast radius" for deployments and going in stages that are also usually time bound ("let it bake"). It also does not account for outages and fixing things you only find in deployment. Programming languages like Python it using injection in Java (e.g. using Guice) either need pristine testing, and all test teams were converted to dev 20 years ago, or have a magical way to destroy all the help compilers and static analysis can give you. So yeah, you take the 4 weeks of development from your 6 month deployment, then add 6 weeks of debugging and retries by using AI. You're welcome that will be 3 million tokens, of which you wrote 1k, the rest was system prompts and "reasoning", which you do not control. This whole AI space is highly fixable, but requires investment no one seems to be willing to do, particularly in areas that were mistakes from the past.
Comment · 6 months ago
I suspect the entire industry uses "auto-raters", where an agent instance is used to scores the agent's output. The idea is similar in intent as using adversarial networks to train image generation, minus the human labelers. Raising the scores of the auto-rater then becomes the metric teams optimize, and it is no wonder the end result is that the agent scores its own generated content the highest.
Comment · 9 months ago
The existence of a kill switch plus tracking in legislation very likely means the manufacturers wanted to track and sell user data and needed a scapegoat to avoid customer backlash. I would profoundly surprised if we don't find a lobbyist at the bottom of this.
Comment · 1 year ago
I have not read the books, so I guess I'm the target audience. It was very hard to watch. The actors were fine, actually better than fine, but the writing was painful. There are lots of standard adventure and fantasy arcs that are just impossible to carry forward with the type of "modern" they wanted. For instance, you cannot have the most diverse village in the history of villages anywhere, then later the Orc (?) stares at one of the kids once and goes "you are not from this village, you are certainly from this other village because of how you look". How? Aura color? At least change it so the kid has a tattoo or some birth mark then. I could go down a long list of dumb stuff like this that makes me come back to reality instead of allowing me to stay with the flow and enjoy the fantasy.
Comment · 2 years ago
Calling failing to show up or nominate a legal representative to a judicial hearing in a country that offers a ridiculous number of opportunities to appeal as "feud" shows some bias in reporting.
Comment · 3 years ago
You. don't. say. Really?!? OMG, what an insight! What are they going to say next? That startups also use the good will of the internet to create their IP to then take it private as a way to get free labor? I'm shocked! And because this is also the internet, that was sarcasm.
Comment · 3 years ago
Allegedly Costco has a policy of making their white collar folks work on the floor of their stores and rotate positions before getting to the office, same with McDonalds. I would love to hear from someone with first hand experience to confirm or correct my impression. I did confirm that Costco gives good benefits and decent pay for all their automotive section employees, and since then I only buy tires there (since 2000) for that very reason. Every time I visit I reconfirm that is the case. It is not much in the great scheme of things, but it is an honest effort to buy from good employers.
Comment · 8 years ago
Yes! I still can work on my open floor plan. Let's open the windows and connect to the awesome traffic jam outside.
Comment · 11 years ago
I wrote last year's benchmark. The clusters are completely different, and so is the workload.
Last year's cluster had 300 VMs, which was a much higher price point, and the workload was write only.
This benchmark uses YCSB workloads A and B, which we though matches the usage we'll have on BigTable. The cluster is much smaller as well.
I shared my scripts from last year, it is pretty easy (although a bit expensive) to repro the numbers. Let me check if we can share this year's benchmark scripts as well.
Comment · 12 years ago
Hi,
The cost we published includes the time to setup the whole cluster, warm up the data nodes, and run for 5 minutes at 1M per second.
Our run rate is $281 per hour, which is the same as AWS a couple of years back. What changed is that we are using quorum commit, the data is encrypted at rest, we have very low tail latency, and we look at all samples when computing that.
Computing our price is easier because we do not charge per access.
Here is the formula for our run rate:
30 loaders (n1-highcpu-8) at $0.522 per hour: 248.7
300 nodes (n1-standard-8) at $0.829 per hour: 15.66
300 1TB PDs that run at 0.055555556/hour: 16.67 Total: 281.03
But keep an eye on us. This is for today prices.
Comment · 12 years ago
Hi - I figured you know what you're doing based on the details you picked on. I suspect you'd have way more fun reading the raw logs, but that would make for a post that is incredibly hard to parse. I will try to figure out a venue for them.
It was 100M keys per loader, so 30x100M for the low latency cluster and 45x100M for the high latency cluster. I sized to keep running long enough to go over half dozen major compactions, not necessarily to eat storage space.
Most of the IO is sequential, which really makes no difference for our backend (sorry, I have to leave it at that). Read performance ends up affecting the performance when the system reaches its limits of pending compactions, which I limited in the config. I could probably get a higher number without it, but it looked more realistic to set limits and I'd not deploy a server without the limits.
I had to make the fsyncs more frequent, not less, to reduce the odds of an operation sitting behind a long flush. This was a counter-intuitive finding for me, but it makes sense considering Persistent Disks throttle both IO sizes and IOPS. We do that to make sure a noisy neighbouring VM won't affect yours. So I turned on trickle_fsync and tuned the size of the maximum syncs to never have a slow flush, but a lot of smaller flushes.
Tuning the Java GC was for the same reason - I did not want long GC cycles. The settings I used were based on the guidance from Java memory ergonomics. The DataStax distribution has tips and hints in the cassandra-env.sh file itself, which I read it and followed through. I also read more about Java memory than I ever want to read again from various vendors. As the joke goes "I did even the things that contradicted the other things" before getting to the settings you saw.
I hope you appreciate the fact I set limits on some of the most dangerous knobs, for instance limiting the RPC server and using HSHA as opposed to unbounded sync.
I understand the limitations of the benchmark I used, and the limitations of the post. I tried to stay within recommended settings for all subsystems, have safe limits for everything I found dangerous, and I picked the workload because that is what our customers do. I don't advocate one storage solution over another, we love everyone that buys our solutions :)
BTW, while we do not sell Cassandra as a service, we do sell Cloud SQL (https://developers.google.com/cloud-sql/). Maybe the sales guys will give me a cut :)
Sharing the config was more work, but I suspected I was going to learn something by doing it. Thank you for the feedback!
PS: sorry for the delay replying - Y Combinator says I am posting too much. I am not quite sure at what point I will hit my daily quota of replies.
Comment · 12 years ago
Network average utilization was low by design. Keeping it steady was more important than low, though, and harder too.
Latency spikes come from Cassandra flushing data to disk (large sequential IO), Java garbage collection and heap resize, and page faults during compactions (random reads).
What I did to even traffic out was to enable trickle_fsync and size the flushes, set Java's max and min heap sizes, as well as to tune the Java heap ergonomics. I treated random reads as a fact of life - I did nothing to tune that.
Comment · 12 years ago
Here is the GIST link: https://gist.github.com/ivansmf/6ec2197b69d1b7b26153