Fix data race in Duo Workflow HTTP transport
What does this MR do and why?
Fixes a data race in the Duo Workflow ndjson HTTP transport (package internal/ai_assist/duoworkflow), caught by the Go race detector in TestBuildHTTP_TearsDownTheDwsStreamWhenTheCallerGoesAway.
Sample failing job: https://gitlab.com/gitlab-org/gitlab/-/jobs/16436965399
When the HTTP caller hangs up, the runner's Execute call returns as soon as the first goroutine reports an error, so the HTTP handler returns too. But a runner goroutine that forwards actions from Duo Workflow Service can still be writing to the response inside ndjsonTransport.flushLineLocked. Once the handler returns, net/http finishes the response on its own goroutine, and both goroutines touch the same response buffer at the same time. runner.Close only waits for that goroutine when a stop was requested, and the caller-gone path never requests a stop.
The fix adds a "done" flag to ndjsonTransport, guarded by its existing writeMu, and a finish() method that takes the lock, sets done, and returns. flushLineLocked returns early once done is set. The handler calls transport.finish() after the runner finishes and before the handler returns. Because finish() blocks on writeMu, it waits for any write in flight, and every later write becomes a no-op. Once the handler returns, no runner goroutine can touch the response, and net/http finishes it alone.
This keeps the fix local to the HTTP transport, with no change to the shared runner shutdown handshake. It covers all teardown paths: caller gone, keepalive failure, normal end, lock contention, and server shutdown. It also avoids a deadlock that would happen if we instead waited on the forwarding goroutine inside Execute, since a workflow that never started leaves that goroutine blocked waiting to receive.
How to test
The existing test already reproduces the race under the race detector, so it serves as the regression guard. Run it many times to confirm the race is gone:
cd workhorse
go test ./internal/ai_assist/duoworkflow -run TestBuildHTTP -race -count=200