Skip to content

sanei: Support capture and replay of USB data for testing

This pull request is a continuation of the idea of having integration tests that I raised on mailing list a couple of months ago: https://alioth-lists.debian.net/pipermail/sane-devel/2019-February/036577.html.

In short, we could have integration tests by capturing USB transfers, saving it somewhere, then running a mock scanning session that would use the captured data to check whether SANE still sends the same commands.

This pull request implements the above using the following approach:

  • sanei_usb can be configured in several testing modes:
    • "record mode": USB data is captured to an XML file without changing behavior
    • "replay mode": all USB transfers issued by SANE are compared to stored data in an XML file. Any differences are reported. The responses to the transfers are read from the file and supplied to SANE as if they come from a scanner.
    • "development mode": same as "replay mode" except that an attempt is made to proceed the scanning session as far as possible in the presence of differences between captured data and the transfers that SANE issues. The scanning session is aborted when SANE issues an USB IN transfer for which there's no reasonable response stored. This mode is useful for development of new backends and also to spot the reasons of test failures if the structure of the scanning session didn't change.
  • the configuration of sanei_usb is completely transparent to the backends and possibly to the frontend tools. The testing mode and the path to an XML file are both given via the --device-name option.

There were disagreements during the mailing list discussion over what approach to choose and I went in a different direction than what was suggested by Olaf, because my primary focus was in implementation of support of a new scanner and not tests, so I was fine with having something that could potentially never be merged to SANE. Now we have something concrete to talk about and I hope the code will be useful at least for that.

I believe the primary difference between the approach implemented in the PR and what was discussed on the mailing list is the selection of the storage format. It was suggested that pcapng data files are used for that and I argued that something like XML would be a better solution. After having some experience using the new functionality I continue to think that XML or similar format is best suited for the task on the following grounds:

  • XML is human readable and editable which makes working with the capture files much easier than compared to binary format.
  • XML is extensible and generic which allows us to write data at right abstraction level. pcap format for example encodes the USB packets themselves. We are more interested in logical transfers which is at a higher level. In addition to that we can save the USB descriptors to the XML file. If we used pcap directly, we would either extend the format which would make much harder to work with the data files, or do some work that is currently performed by libusb ourselves.
  • It's easier to process XML as there's plethora of tools for working with it.

The PR is not ready for merging yet, but it's sufficient enough for a discussion. I've used the functionality for the past month without large issues.

Edited by Povilas Kanapickas

Merge request reports