evmunro

You're right, it's likely that almost nobody is using `fmt.Sprintf` to build SQL queries in production.

Templating and `fmt.Sprintf` are essentially the same thing in this context - `Sprintf` just gets the point across in fewer lines of code, and allows people to come up with realistic scenarios themselves.

This is a great resource on fuzz testing algorithms & internals. I find myself coming back to it occasionally when building new fuzzing techniques at Fuzzbuzz.

OP and Founder of Fuzzbuzz here - let me know if you have any questions about fuzz testing, especially any particularly tricky cases you’ve run into.

I noticed that as well - most fuzzers will have a maximum duration or number of iterations they're allowed to attempt when minimizing so as not to starve out actual inputs. It could be that the fuzzer hit that limit, or potentially prioritizes readable inputs over small inputs.

Thanks for the questions & feedback! Concise docs are really important so this is all super useful. To answer your questions one by one:

1) The BrokenMethods are simple examples of programs that crash on buffer overflows/index out of range errors. If you were to pass "FUZ" into the Go method, it would check Data[3], thus causing a panic since there are only 3 elements in the string.

ninja edit: that python method in your comment IS a valid method with no error - a bit of a brain fart on my end when writing out the docs. It's been changed :)

2) In general a failure is any non-zero exit. We do this to be flexible in the way you report bugs. For C/C++ and Python this is usually with assertions, and in Go you can achieve something similar with:

  if !x {
    panic("Error")
  }

We also have other checkers or "sanitizers" that run with your code to look for certain bugs. For C and C++ code we support tools like Address Sanitizer, which report memory bugs like Heap Buffer Overflows and UAFs, and for Golang you can choose to fuzz your code with a race condition checker. These are just some of the examples of more advanced fuzzing methods we support, and we'll be making nicer tutorials/screencasts to showcase those over the coming week.

3) Thanks for the fixes - much appreciated. And yeah, we know GitBook is pretty slow, and we're in the process of moving to another docs provider.

If you've got any more questions please let me know!

Sure! Some of the classes of bugs that remain low-hanging fruit for languages like Python include slowness, hangs, panics, race conditions, assert failures, excessive resource consumption and other Denial of Service attacks.

Other use cases include using fuzzing to compare implementations of libs that require the same functionality, detecting invariant violations, testing implementations that are meant to work together (i.e. serialize(deserialize(x)) == x).

In general fuzzing C/C++ libraries for memory bugs is the most commonly described use-case, but I think there are tons of fuzzing use cases that haven't been thoroughly explored yet.

Radamsa is awesome! Definitely agree, and one of the goals for Fuzzbuzz is to be able to hot-swap between fuzzing backends without any interface changes (or to use all backends at the same time, to account for differences in findings).

re: pricing, we do offer infinite scalability in terms of CPUs, but that might not be as clear as we'd like it from our pricing page. Or maybe I'm misunderstanding you. Either way, if you have any more thoughts/suggestions on pricing I'd love to hear it.

Thanks for the link! We've been looking at all the current AFL-like/AFL wrappers for Java as we decide how best to implement Java fuzzing in Fuzzbuzz, and yours looks pretty nice.

Definitely going to play around with this :)

Memory security issues have been the main focus of fuzzing, but it's really useful for other use cases as well, such as: slowness/hangs, assert failures, panics, excessive resource consumption and DOS attacks. We've also done some work with Go to detect race conditions while fuzzing.

You can also do differential fuzzing to compare 2 different implementations that solve the same problem, or fuzz for invariant violations/assertion failures. I think the possibilities extend far beyond just memory safety, and I'm really looking forward to finding other areas in which fuzzing is applicable.