🐈‍⬛ Mood: Petting my friendly neighborhood black cat named Moon Dragon on the way to the corner market.

🎵 Soundtrack: The Lumineers Radio

📚 Reading Github Issues

Only 8 new github issues this morning.

🧶 Back to Go Fuzzing

The “interesting” inputs issue caught my eye, so I decided to set aside compiling for today. Before I claimed it I wanted to make sure I could reproduce and fix the issue. (I wouldn’t want to have to do something embarassing like ask for help or anything.)

My first question was: what are “interesting” inputs???

When I was testing go fuzz for the first time on day 8 the example I used had 0 interesting inputs.

$ gotip test -fuzz=FuzzParse

found a crash, minimizing...
fuzzing, elapsed: 0.1s, execs: 27 (218/sec), workers: 4, interesting: 0
--- FAIL: FuzzParse (0.12s)
        panic: oh no! bad size!
        goroutine 11 [running]:
        runtime/debug.Stack()
        ...

I didn’t find much information in the draft design. I grep-ed through the code and found this comment.

// minimizeResponse contains results from workerServer.minimize.
type minimizeResponse struct {
	// Success is true if the worker found a smaller input, stored in shared
	// memory, that was 👉"interesting"👈 for the same reason as the original input.
	// If minimizeArgs.KeepCoverage was set, the minimized input preserved at
	// least one coverage bit and did not cause an error. Otherwise, the
	// minimized input caused some error, recorded in Err.
	Success bool
  ...

I tried writing an example that would create “interesting” inputs, but I kept getting 0. My examples were very simplistic. I didn’t feel like I was getting anywhere, so I decided to replicate an old bug. I looked through the trophies in the go-fuzz (with a hyphen) repo and I chose a bug from 2015 where invalid XML comments would cause marshaling to fail.

Initially I tried to reproduce this bug by going back to go1.5 when this bug was still there and then running a fuzzing test against it. Ha, no. That did not work. It is hard enough to run go tests using the go test source code I am developing, let alone run a new gotip feature against an old local version of go1.5. The commit fixing the bug was pretty simple, so in the end I ended up manually reverting the fix on my local master branch of golang/go.

Turns out this example was much more “interesting” to go fuzz (no hyphen).

Here is my fuzz test:

func FuzzXML(f *testing.F) {
	// Seed the corpus
	f.Add([]byte("<z><!-- my - - - -comment --><b>foo</b></z>"))

	// Run the fuzz test
	f.Fuzz(func(t *testing.T, a []byte) {
		t.Parallel()
		v := new(X)
		if Unmarshal(a, v) != nil {
			t.Skip() // only test unmarshal-able things
		}
		if _, err := Marshal(v); err != nil {
			panic(err)
		}

	})
}

And here is the first few seconds of output:

$../bin/go test -fuzz=FuzzXML
fuzz: elapsed: 0s, gathering baseline coverage: 0/2 completed
fuzz: elapsed: 0s, gathering baseline coverage: 2/2 completed, now fuzzing with 4 workers
fuzz: elapsed: 3s, execs: 184966 (61597/sec), interesting: 23
fuzz: elapsed: 6s, execs: 384296 (64005/sec), interesting: 26
fuzz: elapsed: 9s, execs: 627750 (69729/sec), interesting: 30
^Cfuzz: elapsed: 10s, execs: 728539 (71595/sec), interesting: 31
PASS
ok  	encoding/xml	10.281s

But after I ctl-Ced there was no testdata dir! On the one hand this makes sense because I didn’t let the test run long enough to find a crasher. On the other hand: where were these “interesting” things being stored?

After digging through the code I found out that they are stored in the $GOCACHE.

$ ../bin/go env GOCACHE
/Users/adowns/Library/Caches/go-build

When I looked at the cache for this specific test it was full of files with guid names that had inputs in them:

$ ls ~/Library/Caches/go-build/fuzz/encoding/xml/FuzzXML

11d583bd9e786f3b85c64107dbcf460c7908784d0103656619450d012ef041cb
16e03fe84250b0ab157cc1e6de3da3c755868a3b7dba714cc897f0417b821801
...


$ cat ~/Library/Caches/go-build/fuzz/encoding/xml/FuzzXML/*
go test fuzz v1
[]byte("oooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooo")
go test fuzz v1
[]byte("><")
go test fuzz v1
[]byte("\xc0\x00:ڔ.Q\u007f\xafh\xb0\xbez\xf2p\x9f\xecG\xc4\xe5A\x9en|\x10,H\x92\xff\x00\xf9")
go test fuzz v1
[]byte("<!D=\xce\x18\xa0\xec\x15\xa1--d HE")
go test fuzz v1
[]byte("\n\n\n\n\n\n\n")
go test fuzz v1
[]byte("\n")
go test fuzz v1
[]byte("\r\r\r\r\r\r\r\r")
go test fuzz v1
[]byte("d")
go test fuzz v1
[]byte(">;;;;\xf8\xe8\x03\x00\x00\xb1sssss\xc2\xff\x12\xf8\xff\xb1\u008f.\xb1\xc2\xff\xf0\xf8!")
go test fuzz v1
[]byte("<!--")
go test fuzz v1
[]byte("\r")
go test fuzz v1
[]byte("[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[")
...

Okay! Now I can reproduce the issue!

Adding the code to fix it wasn’t too much work after that. I did have a sneaky off-by-one error that I realized from from the seed value. I submitted my work and got a code review by Katie Hockman (!!!) back just 30 minutes later. Hopefully she likes my fixes and it will be merged tomorrow. Another contribution in the books!