tybug
Joined 4 years ago
Karma
76
Submissions
29
Member since
Jan 5
Recent submissions
Comment · 6 months ago
We plan to rewrite hypothesis in rust and expose an FFI through that! This is a medium term plan (months, not weeks or years), but we're acutely aware that relying on a python component is not a long-term solution.
Comment · 6 months ago
Yes! I just wrote up documentation for the protocol earlier this week: https://hegel.dev/reference/protocol.
In reality, we hope to provide more guidance than this to people who want to write their own language frontend. This protocol reference doesn't talk about the realities of [hegel-core](https://github.com/hegeldev/hegel-core) and how to invoke it, for example.
We intend to write a "How to write your own Hegel library" how-to guide. You can subscribe to this issue to get notified when we write that: https://github.com/hegeldev/website/issues/3.
If you're eager, pointing your favorite LLM at https://hegel.dev/reference/protocol + https://github.com/hegeldev/hegel-rust and asking it to write you one for your language of choice should be enough to get you started!
Comment · 6 months ago
I didn't expect to see Hegel when opening up HN today! Feel free to ask any questions about it. We released hegel-go earlier this week, and plan to release hegel-cpp sometime next week, so look forward to that :)
Comment · 7 months ago
To put it on the record: my position is current models can't get us there, and neither can the next iteration of models, but in two model iterations this will be worth doing. There's a lot of fiddly details in Hypothesis that are critical to get right. You can get a plausible 80% port with agents today but find they've structured it in a way to make it impossible to get to 100%.
Comment · 7 months ago
`from_type` just supports a bunch more things than rust ever can due to the flexibility of python's type system. `from_type(object)` is amazing, for example, and not something we can write in rust.
Comment · 7 months ago
Yep, `#[derive(DefaultGenerator)]` and `generators::default<T>()` are the right tools here.
This is one of the areas we've dogfooded the least, so we'd definitely be happy to get feedback on any sharp corners here!
I think `from_type` is one of Hypothesis's most powerful and ergonomic strategies, and that while we probably can't get quite to that level in rust, we can still get something that's pretty great.
Comment · 7 months ago
I actually think there's another angle here where PBT helps, which wasn't explored in the blog post.
That angle is legibility. How do you know your AI-written slop software is doing the right thing? One would normally read all the code. Bad news: that's not much less labor intensive as not using AI at all.
But, if one has comprehensive property-based tests, they can instead read only the property-based tests to convince themselves the software is doing the right thing.
By analogy: one doesn't need to see the machine-checked proof to know the claim is correct. One only needs to check the theorem statement is saying the right thing.
Comment · 7 months ago
As possibly the one community on earth where it's actually better to post the code than the blog post: TL;DR this is a universal property-based testing protocol (https://github.com/hegeldev/hegel-core) and family of libraries (https://github.com/hegeldev/hegel-rust, more to come later).
I've talked with lots of people in the PBT world who have always seen something like this as the end goal of the PBT ecosystem. It seemed like a thing that would happen eventually, someone just had to do it. I'm super excited to actually be doing it and bringing great PBT to every and any language.
It doesn't hurt that this is coming right as great PBT in every language is suddenly a lot more important thanks to AI code!
Comment · 1 year ago
Appreciate the thoughts <3. I do think there might be stronger examples we could choose. Possibly json encode/decode..
Comment · 1 year ago
(Hypothesis maintainer here) If you have recommendations for a better example on the front page, I'd love to hear them! (I mean this entirely genuinely and non-sarcastically; I agree sorting can give misleading ideas, but it is also concise and well understood by every reader).
Comment · 1 year ago
(Hypothesis maintainer here)
Yup, a standard test suite just doesn't run for long enough for coverage guidance to be worthwhile by default.
That said, coverage-guided fuzzing can be a really valuable and effective form of testing (see eg https://hypofuzz.com/).
Comment · 1 year ago
Nice! "testing your test code" is particularly important when dealing with PBT distributions, especially when your generator gets more complicated.
Tyche [0] is another cool tool for addressing the same problem, visualizing the PBT distribution but not making assertions about it.
https://github.com/liam-devoe