Skip to content

Dataset input for analysis class

Adriaan requested to merge dataset_input_#181 into develop

Explanation of changes

This addresses #181 (closed). (@kel85uk it was assigned to you, I hope you don't mind my weekend coding). N.B. this MR also includes the work from !155 (merged) and as such will be blocked by that MR once review is complete. The reason is that I had to add documentation changes and wanted to avoid conflicts. The MR also contains several misc comments and codacy fixes.

I have implemented the following desired behavior (from #181 (closed))

  • Accept dataset as input. Bypass extract_dataset
  • Happens based on TUID of dataset if no TUID present, a new TUID will be generated based on the current timestamp I have additionally implemented the following behavior.
  • If a dataset is provided that is not present in the datadir, a copy of the dataset will be stored in the right location based on the TUID.

I have not implemented

  • Data conversion Quantify dataset will be resolved in a separate issue (MR opened by Pieter)
  • Disabling of disk writing will be an option added to the Analysis class. (Use options passing framework: i.e. the global settings similar to the dpi etc...)

I would propose moving the disabling of disk writing to a separate MR.

It should be noted that this analysis will create new files in the test directory upon executing and potentially creates a memory effect in the tests. Resolving that (tests should not write to memory, or at least clean up properly) is out of scope of this MR. See #180 (closed).

I create a custom dataset using the following snippet and give it as an input argument for the Basic1DAnalysis.

x0 = np.linspace(0, 2 * np.pi, 31)
y0 = np.cos(x0)
x0r = xr.DataArray(
    x0, name="x0", attrs={"name": "t", "long_name": "Time", "units": "s"}
)
y0r = xr.DataArray(
    y0, name="y0", attrs={"name": "A", "long_name": "Amplitude", "units": "V"}
)

dset = xr.Dataset(
    {"x0": x0r, "y0": y0r},
    attrs={"name": "custom cosine", "tuid": "20210417-191934-749-41de74"},
)
dset = dset.set_coords(["x0"])

Motivation of changes

The chosen solution is based on the discussion of #181 (closed) .


Merge checklist

See also merge request guidelines

  • Merge request has been reviewed and approved by a project maintainer.
  • Merge request contains a clear description of the proposed changes and the issue it addresses.
  • Merge request made onto appropriate branch (develop for most MRs).
  • New code is fully tested.
  • New code is documented and docstrings use numpydoc format.
  • Changelog has been updated (when applicable).
  • CI pipelines pass
    • black code-formatting passes (gitlab-ci),
    • test suite passes (gitlab-ci),
    • no degradation in code-coverage (codacy),
    • no (serious) new pylint code quality issues introduced (codacy),
    • documentation builds successfully (CI and readthedocs).
Edited by Kelvin Loh

Merge request reports