Trying to make a distinction between file directory operations for scans and scanning code

Julian Stirlingrequested to merge
split-scan-dir-operations into v3

This is built upon !289 (closed) which would need to be merged first

Rather than merging !289 (closed) with tests it was far easier to test this now it was split out.

This started as a small change and spiralled out of control, but it now has unit tests.

Generally this moves most file operations out of the SmartScanThing this isn't 100% compete. Dealing with the json file should be moved to ScanDirectory.

This has not reimplemented the "promoting" of files to outside the directory. This should be done by creating a fixed output directory.

This introduces some testing for SmartScanThing as well as the ScanDirectory. The Thing is being tested directly, rather than through a server/TestClient. This way there is no dependency injection so the class remains isolated from other Things. Any Things used for methods will need to be mocked.

Closes #328 (closed)

Notes:

  • Directory related functions moved to new python file, in two classes ScanDirectoryManager and ScanDirectory
  • The ScanInfo model was moved to the same python file.
  • Checking the disk space is also now in this new file
  • Scans now iterate in numerical order (i.e. if a scan_0001, scan_0002, and scan_0004 exits the next is scan_0005 (not scan_0003)
  • New file has 100% test coverage (there is a bug in displaying the test coverage in GitLab showing 4-5 lines as not covered), but I am sure could do with more detailed tests eventually.
  • Only update to WebApp on changing date formats (no visible UI change):
    • Previously: The scan date was calculated as a unix timestamp (s), loaded into a pydantic model that converts into a python datetime, which then formats it into a ISO 8601 string in json that is sent to the javascript where it is converted back into a unix timestamp (ms) (as it needs to be sorted) before finally being formatted into a human readable string for the UI?
    • Now: The scan date was calculated as a unix timestamp (s), that is sent to the javascript (via pydnatic), before finally being formatted into a human readable string for the UI?
  • Further tests (that use mock Things) have been created to test some basic smart scan functionality.

Merge checklist:

  • All new/changed functions have up to date typehints and docstrings
  • Any UI updates have a before and after screenshot in the MR description
  • Unit testing of new functionality has similar or better coverage that similar existing code
  • This code has been tested on a microscope
    • With webapp has been recompiled (if changed)
    • With updated settings JSON (if changed)
    • SmartScan and live stitching work
Edited by Julian Stirling

Merge request reports