Can't read duplicate URLs from STDIN
I'm using wget to test a web server, specifically reused connections, keep-alive, timeouts and so on. My test case is to request the same URL many times. Here is the command line I'm having trouble with (note this produces a LOT of output on STDERR):
$ yes "http://127.0.0.1:4001/ping" | ./wget2 --no-http2 --debug -O - -i -
This only makes one request and produces many copies of this:
20.155151.926 *url =
20.155151.926 *3 http://127.0.0.1:4001/ping
20.155151.926 [0] action=1 pending=0 host=0x0
20.155151.926 not requesting 'http://127.0.0.1:4001/ping'. (Already Seen)
20.155151.926 queue_size: qsize=0
And qsize is always 0. There is no other output until I ctrl-c and it prints the stats line:
Downloaded: 1 files, 5 bytes, 0 redirects, 0 errors
For contrast, if I use /dev/stdin
, it works fine. Output includes:
20.155659.970 *url =
20.155659.970 *3 localhost:4001/ping
20.155659.970 not requesting 'http://localhost:4001/ping'. (Already Seen)
20.155659.970 host_add_job: job fname (null)
20.155659.970 host_add_job: 0x561891cbb440 http://localhost:4001/ping
20.155659.970 host_add_job: qsize 6 host-qsize=6
With qsize incrementing as expected, interspersed with messages about actually connecting and getting data.
Edit: I'm haing a hard time reproducing this with recent changes to the master branch. In fact it seems backwards now? Using -
works fine, which means I no longer have an issue. But using /dev/stdin
doesn't work, which might indicate other problems with named pipes and so on.