x0xMaximus

We run Dask distributed for contents in what their chart covers (10TB+ with 4+ physical hosts) and it's a problem that can't really be discussed without well defined drive read or network latency limitations (if non-local storage). We've found defining recordsize, raid and other filesystem patterns to have an equal or larger impact in itself. Pandas is great because it allows us to validate on a sampling or smaller time window dataset, and then the same engineers can use a very similar implementation for Dask distributed through their scheduler tool.

While throwing hardware at problems isn't always a reasonable solution, it's trivialized to discuss this without IOPS or other read limitations.. and the nvme enterprise market is rapidly developing making this change quickly.

Amazon took 20% of all workers earnings for doing essentially nothing for years and years.. of course I take great joy in giving AWS as little money as possible.

Do you know how much inn-n-out you can buy when saving ~$50 a day and still be in the positive! It came out of building our own ledger system and abandoning any Decimal or float operations early on.. then it sparks "I wonder how they do it and if it's correct??"

Finally you get it. Only exception is that we run our own exchange now to do task bidding so we don’t need to deal exclusively with other companies to middleman. Core business is what’s called yield management (akin to DSP in adtech) where the best survey (is the user qualified for it, does it have the best pay, etc) is selected for traffic in <100ms. I’d only argue the shady companies are the ones paying proxies (like cint.com) and pushing paid user acquisition instead of surveys. We actively fund ontology development for better profiling targeting. but yes, big enterprises/consultancies can use and interpret the collected data however they want, not our responsibility and we have no legal rights over it anyway

Users/respondents lie, and many buyers/researchers are neo-luddites; most C level still come from the telemarketing days. A fun example: mobile targeting is terrible when off wifi because all survey platforms uniq identify users based off their IPv4, so any T-Mobile LTE users in the same city going through the same CGNAT often share profiling data that ends up conflicting, which ends up meaning they don't get sent into the best survey(s). The issue is even worse in heavy IPv6 countries like India+France.

Buyer A is presented with the following rate card:

< 10min targeting gen pop: $3 10 <= 15min targeting gen pop: $5

Of course they'll bid saying their 14 min survey only takes 9 min to complete. Buyers try to cheat pricing strategies of exchanges just as much as respondents try to cheat buyers on survey platforms. Both parties can't be trusted and have adverse incentives.

Wish we didn’t even need one

1

Obviously, a problem not even fully addressed by L2’s datasets. Sure, but the use of proxies to masquerade the identity of users being sold to researcher buyers that paid for a different service is fraud

As AMT's largest requester for the past 10 years, this news was relayed to requesters at the same time as respondents. It's also worth noting that our lead contact, the Sr Program Manager at AWS leading AMT, transitioned to Amazon Bedrock and SageMaker Model Evaluations a ~2-3 years ago.. Leaving behind essential zero team managing the project after they migrated over the stored value accounts to native AWS billing.

aerc is the best program I've ever used. tmux from anywhere. May TUIs never die.

We had last bought 24TB+ (4x 24 racks, and some miscellaneous) of DDR5 4800 and 6400 last year while finishing a move off of AWS entirely. I tried to be responsible and prioritize a cold and warm spares of all our equipment and over allocation to ensure proper redundancy capacity. At the time it was a hard pill to swallow looking at our budget; but a few months ago we've started to offload some DIMMs on various channels. What went from a cost analysis between a cloud provider, to having to manage transit, circuits, equipment, a IP blocks, etc etc and all the operational obligations that come with it.. it ultimately covering our hosting costs entirely for the next ~18months *at least* (while "only" coming at the cost of having slightly less cold spares sitting in the bottom of our cages).

Given we're all 9004/9005 now, and that (from what I understand) Venice will support 6400. Will hopefully buy back in a glut comes ready for the next bubble. It's been wild, I even posted a small 32GB DDR5-4800 that was not getting recognized for $100 on eBay and it was literally purchased within ~15min of being live.