Binary Coverage the Wrong Way

Way back in the age of the dinosaurs, if you wrote a program and wanted to test that it was correct, you mostly had two choices: you could either manually construct malformed inputs and try them in succession, or hook up a program which would randomly generate inputs and try them automatically 1 . Those programs were called fuzzers, or sometimes generators, and this was “fuzz testing”. It also for quite a long time wasn’t very good: the generator would have no feedback from the program under test except “did this random input crash or not”. This feedback, although useful, doesn’t allow the fuzzer to tell if it is making progress relative to the last random input or doing a good job, and so your fuzzer would just repeatedly hit the same shallow code. You’d leave it running for a week and then when you come back realize it was never generating inputs that would hit interesting parts of your program, and they were all immediately being thrown out or never progressing past a if(packet_header == 0xdeadbeef) check.