Skip to content

Optimize JSON unmarshalling of DevTools messages

Problem

Chromium DevTools sends DAST thousands of JSON messages to describe a page that is loading in the browser. Unmarshalling each of these messages into an object that can be worked on in DAST takes time. Optimization of this process could lead to reductions in scan time.

Evidence

Flamegraph

Purple highlight is github.com/json-iterator/go.(*frozenConfig).Unmarshal

Spike benchmark results

Difference between the old and new style is a 2.5x improvement on the example JSON message used in the benchmark. Lower ns/op is better.

goos: darwin
goarch: arm64
pkg: github.com/wirepair/gcd/v2
BenchmarkUnmarshalComputedStyleOld
BenchmarkUnmarshalComputedStyleOld-10    	   44449	     26598 ns/op	   10384 B/op	     551 allocs/op
BenchmarkUnmarshalComputedStyleNew
BenchmarkUnmarshalComputedStyleNew-10    	  111758	     10212 ns/op	    9760 B/op	     110 allocs/op

Proposal

  • Update the GCD library to use go-mock instead of jsoniter
  • Inline error messages so that each JSON message only needs to be unmarshalled once

References

GCD GitHub pull request https://github.com/wirepair/gcd/pull/71

Edited by Cameron Swords