Fix sftp aio read with ProxyJump (issue #319)

Eshan Kelkarrequested to merge
eshankelkar/libssh-mirror-eshan:aio_read_proxyjump_fix into master

Tries to fix #319 (closed) (tested with the reproducer linked in the issue)

The reproducer provided in the issue description had a model as follows (with one jump host): fd_1---(socket_pair)---fd_2---(connector)----channel(fd_3)-----server

Via debugging, it was noticed that the channel connected directly to the server stored a lot of unbuffered data (received from the server) that wasn't being written to fd_2 via the connector API.

(Here on, channel refers to the channel(fd_3) in the diagram connected directly to the server)

Consider the situation, where after a bit of progress in the transfer, the server has sent all the requested data (requested via outstanding requests) and all of that data is stored in channel->stdout_buffer. Say this data is 10,000 bytes.

At this point, all the client (fd_1) is doing is waiting for all outstanding requests. (and processing thei responses)

  • POLLOUT event callback gets generated indicating that fd_2 is available for writing.

  • ssh_connector_fd_out_cb() gets called to handle the POLLOUT.

  • Assuming connector->in_available was true, 4096 (CHUNKSIZE) bytes get read from the channel. (really channel->stdout_buffer) leaving 10,000 - 4096 = 5904 bytes unread in the channel.

  • The read bytes are sent via fd_2 (so that fd_1 can recv them)

  • After this, the callback sets connector->in_available to 0 and connector->out_wontblock to 0.

  • Since out_wontblock has been set to 0 ssh_connector_reset_pollevents() (called after the callback returns) will consider POLLOUT events on the connector output.

  • (Based on assumption before) Since the client (fd_1) is eagerly awaiting responses and processing them, the received data gets processed quickly and fd_2 is available for sending/writing.

  • POLLOUT event gets generated for fd_2 indicating that its available for writing/sending to fd_1

  • ssh_connector_fd_out_cb() gets called to handle the POLLOUT

  • Since connector->in_available is 0 (and ssh_connector_channel_data_cb() has not been trigerred in between as we have assumed before that all the data has already been received on the channel and is stored in the channel->stdout_buffer), ssh_connector_fd_out_cb() does nothing besides setting connector->out_wontblock to 1.

  • Since out_wontblock has been set to 1 ssh_connector_reset_pollevents() (called after the callback returns) will IGNORE POLLOUT events on the connector output.

  • So, at this point, the channel->buffer contains 5706 bytes and the fd_2 is available for writing/sending (out_wontblock is 1), but nothing happens and the transfer gets stalled/hanged.

In my opinion, this hanging occurs because connector->in_available was incorrectly set to 0 despite the channel buffer having 5706 bytes in it.

This commit changes that code to consider the data available to read on the channel (includes buffered data as well as polled data on channel's internal fd) and taking that into consideration to set in_available appropriately. (Instead of unconditionally setting it to 0 as the current code does) so that the next time POLLOUT gets received on fd_2 the ssh_connector_fd_out_cb() does read from the channel and write to fd_2 (as the connector->in_available flag would be set).

Checklist

  • Commits have Signed-off-by: with name/author being identical to the commit author
  • Code modified for feature
  • Test suite updated with functionality tests
  • Test suite updated with negative tests
  • Documentation updated

Reviewer's checklist:

  • Any issues marked for closing are addressed
  • There is a test suite reasonably covering new functionality or modifications
  • Function naming, parameters, return values, types, etc., are consistent and according to CONTRIBUTING.md
  • This feature/change has adequate documentation added
  • No obvious mistakes in the code

Merge request reports