mikeckennedy
Joined 11 years ago
Karma
529
Submissions
200
Member since
Apr 1
Recent submissions
Comment · 8 months ago
Comment · 9 months ago
Disappearing from AI results won't make AI go away.
Comment · 10 months ago
I didn't tell anyone to optimize anything. I just posted numbers. It's not my fault some people are wired that way. Anytime I suggested some sort of recommendation it was to NOT optimize.
For example, from the post "Maybe we don’t have to optimize it out of the test condition on a while loop looping 100 times after all."
Comment · 10 months ago
That's a fair point @esafak. I updated the article with something akin to the doubling chart of numbers in the original article from 2012.
Comment · 10 months ago
It is open source, you could just look. :) But here is a summary for you. It's not just one run and take the number:
Benchmark Iteration Process
Core Approach:
- Warmup Phase: 100 iterations to prepare the operation (default)
- Timing Runs: 5 repeated runs (default), each executing the operation a specified number of times
- Result: Median time per operation across the 5 runs
Iteration Counts by Operation Speed: - Very fast ops (arithmetic): 100,000 iterations per run
- Fast ops (dict/list access): 10,000 iterations per run
- Medium ops (list membership): 1,000 iterations per run
- Slower ops (database, file I/O): 1,000-5,000 iterations per run
Quality Controls:
- Garbage collection is disabled during timing to prevent interference
- Warmup runs prevent cold-start bias
- Median of 5 runs reduces noise from outliers
- Results are captured to prevent compiler optimization elimination
Total Executions: For a typical benchmark with 1,000 iterations and 5 repeats, each operation runs 5,100 times (100 warmup + 5×1,000 timed) before reporting the median result.
A bit of feedback I've already gotten since this is a Python-leaning article is:
1. What are you crazy: Why not Pydantic? Pydantic is great for many things. I'm a bit fan. I'm just not convinced it needs to be the access layer for my DB. Most apps do many more reads than writes. You pay the cost of validation on every read, not just writes. Alternatives include using dataclass-wizard (https://github.com/rnag/dataclass-wizard) and cattrs (https://catt.rs) for validation on the way in for dataclasses without revalidating on every read.
2. Data classes don't provide runtime type checking. No they don't. If that is paramount for you, choose Pydantic (see #1for why I chose data classes ;) ). But data classes do provide type-checking time type safety. When paired with tools like cattrs and dataclass-wizard, inbound data can be type safe too.
3. This uses MongoDB examples. Fair. The pattern applies to SQL too (psycopg3 + parameterized queries + dataclasses). The core argument, that AI handles vanilla SQL better than SQLAlchemy ORM code, holds even more strongly there given how universal SQL is.
Benchmarks and runnable code are in the repo if you want to check it out.