Trace Improvements

Description

Trace logging is currently quite slow, and has a couple of security edge cases where secrets are revealed (#16705 (closed), #4625 (closed), !1541 (comment 364168496)).

Proposal

  • Improve on these benchmarks:

    BenchmarkBuffer10k-16              46	 257056975 ns/op  13.54 MB/s	 1851786 B/op	  230045 allocs/op
BenchmarkBuffer10kWithURLScrub-16  30	 395656171 ns/op   8.80 MB/s	20065181 B/op	  330147 allocs/op

  • Remove io.Pipe/goroutine setup from existing trace logging implementation (#27070 (closed), !2464 (merged))

    new benchmarks:

    BenchmarkBuffer10k-16              66	 180558325 ns/op  19.27 MB/s	15927150 B/op	  240037 allocs/op
BenchmarkBuffer10kWithURLScrub-16  34	 356988114 ns/op   9.75 MB/s	35047596 B/op	  340087 allocs/op

  • Improve trace secret masking with x/text/transform and fix one security edge case (#16705 (closed), !2863 (merged))

    new benchmarks:

    BenchmarkBuffer10k-16              289   41393249 ns/op	  84.07 MB/s	    9416 B/op	      24 allocs/op
BenchmarkBuffer10kWithURLScrub-16  60    201807342 ns/op  17.24 MB/s	17749466 B/op	  100052 allocs/op

  • Add URL scrub text transformer to mask URL secrets and fix the other security edge case (#4625 (closed).

    This was worked on in !2679 (merged), but was then reverted in !2950 (merged) because it increased the rate a bug was triggered inside another transform chain.

    Now being worked on: !3404 (merged)

    BenchmarkBuffer10kWithURLScrub has been removed, as URL scrubbing is now built-in.

    new benchmarks:

    BenchmarkBuffer10k-16  237	  51358202 ns/op   67.76 MB/s	   13608 B/op	      26 allocs/op

  • Buffer writes, reduce read allocations (#27853 (closed), !2886 (closed))

Links to related issues and merge requests / references

!2400 (closed) layed out the initial work in one go, before being broken down into smaller MRs.

Edited by Arran Walker