Signed-off-by: Jiri Denemark <jdenemar@redhat.com>

After the switch of 'my_clean' to a simple Command, the 'clean' command
has no more bits for options, resulting in distutils (either external
or embedded in setuptools) complaining about it:

  distutils.errors.DistutilsClassError: command class <class '__main__.my_clean'> must provide 'user_options' attribute (a list of tuples)

To overcome that, provide all the standard bits from options, i.e. the
'user_options' list, and the 'initialize_options' & 'finalize_options'
methods. In addition, add a dummy 'all' option, as distutils wants it:

  error: error in [...]/.pydistutils.cfg: command 'my_clean' has no such option 'all'

Fixes commit a965c91c

Signed-off-by: Pino Toscano <ptoscano@redhat.com>

Add classifiers that indicate we intend to support python versions
3.9 and 3.10.
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>

In Python 3.10 the setDaemon method was deprecated. It is redundant
since the 'daemon' parameter can be given when creating the thread,
or the 'daemon' attribute can be set after it was created.
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>

Add a test for one more usage scenario that was possible in the past,
whereby libvirt events are registered before starting the asyncio
loop, but we let libvirt find the loop associated with the current
thread.

Skip the test relies on auto-creating an event loop with Python >= 3.10
since it now triggers a deprecation warning which will soon turn into a
RuntimeError.
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>

We currently have to run each of the test_aio.py test cases in a
separate process, because libvirt.virEventRegisterImpl can only be
called once per process. This leads to quite unpleasant console
output when running tests.

By introducing a mock for libvirt.virEventRegisterImpl we can
regain the ability to run everything in a single process. The only
caveat is that it relies on tests to fully cleanup, but in practice
this is ok for our current tests.
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>

Chris Gunn authored Feb 10, 2022 and Daniel P. Berrangé committed Jun 08, 2022

Signed-off-by: Chris Gunn <chrisgun@microsoft.com>

Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>

The drain method uses an asyncio.Event object to be notified when other
coroutines have removed all registered callbacks. The Event object needs
to be associated with the coroutine that the event loop is running with
and currently this is achieved by passing in the 'loop' parameter.

Unfortunately Python 3.10 has removed the 'loop' parameter and now the
object is associated implicitly with the current thread's event loop.
At the time the virEventAsyncIOImpl constructor is called, however,
there is no guarantee that an event loop has been set for the thread.
The explicitly passed in 'loop' parameter would handle this scenario.

For portability with Python >= 3.10 we need to delay creation of the
Event object until we have a guarantee that there is a loop associated
with the current thread. This is achieved by lazily creating the Event
object inside the 'drain' method, which is expected to be invoked from
coroutine context and thus ensure a loop is associated.
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>

Chris Gunn authored May 16, 2022 and Daniel P. Berrangé committed Jun 08, 2022

The 'async' keyword is new in Python 3.5, as a way to declare that a
method is a coroutine. This replaces the '@asyncio.coroutine' decorator
that is deprecated since 3.8 and scheduled to be removed in 3.11

The 'await' keyword has to be used instead of 'yield' from any
coroutines declared with 'async'.
Signed-off-by: Chris Gunn <chrisgun@microsoft.com>
[DB: Split off from a larger patch mixing multiple changes]
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>

Chris Gunn authored May 16, 2022 and Daniel P. Berrangé committed Jun 08, 2022

setup.py ensures we have python >= 3.5, so there is no need to do
back compat with the 'asyncio.ensure_future' method, which was new
in 3.4.4
Signed-off-by: Chris Gunn <chrisgun@microsoft.com>
[DB: Split off from a larger patch mixing multiple changes]
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>

Signed-off-by: Jiri Denemark <jdenemar@redhat.com>

Signed-off-by: Michal Privoznik <mprivozn@redhat.com>

Signed-off-by: Jiri Denemark <jdenemar@redhat.com>

The API coverage test is no longer a special case.
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>

The python lxml registers some global callbacks with libxml2. As a
result when another user of libxml2 calls APIs, it can trigger the
python callbacks that lxml previously registered. Execution of the
python callbacks in this case is often unsafe and leads to SEGVs.

This hasn't been a problem since the sanitytest.py test has been
a standalone program we execute. When it gets turned into a real
python unit test, it will run in the same process as all the other
tests and trigger the crash.

A mitigation was added in lxml 4.5.2 which is good enough to let
us continuing using lxml.
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>

The sanitytest.py file is now using the normal python unittest
pattern, though we invoke the one test explicitly for now.
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>

This is a step towards turning the sanitytest.py file into a normal
python unittest.

Best viewed with the '-b' flag to diff.
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>

This is a step towards turning the sanitytest.py file into a normal
python unittest.

Best viewed with the '-b' flag to diff.
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>

This is a step towards turning the sanitytest.py file into a normal
python unittest.

Best viewed with the '-b' flag to diff.
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>

This is a step towards turning the sanitytest.py file into a normal
python unittest.

Best viewed with the '-b' flag to diff.
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>

This is a step towards turning the sanitytest.py file into a normal
python unittest.

Best viewed with the '-b' flag to diff.
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>

This is a step towards turning the sanitytest.py file into a normal
python unittest.

Best viewed with the '-b' flag to diff.
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>

This is a step towards turning the sanitytest.py file into a normal
python unittest.

Best viewed with the '-b' flag to diff.
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>

We want to move over to make sanitytest.py operate like a more normal
test script, which means making it self contained.

The setup.py already sets the PYTHONPATH thanks to changes introduced
in:

  commit eaded7bd


  Author: Daniel P. Berrange <berrange@redhat.com>
  Date:   Tue Mar 18 11:11:48 2014 +0000

    Add support for running unit tests with nose

so passing the python module path into sanitytest.py is redundant.
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>

We want to move over to make sanitytest.py operate like a more normal
test script, which means making it self contained.

The test already knows how to find the libvirt API XML path using
pkg-config and if an override location is required, this can be done
by pointing $PKG_CONFIG_PATH to a suitable place.
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>

The distutils package is deprecated and targetted for deletion in Python
3.12, so we need to switch to setuptools. Thanks to all the preceeding
changes this is no more difficult than changing the import statements.

Closes #1

Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>

The 'get_platform' function is used to determine the platform specific
component of the build output directory containing the loadable
modules and python code. There is no nice replacement for this
function, but we can achieve our goal by simply scaning for the desired
subdirectory, which should exist by the time 'test' runs.
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>

We override the 'build' command to invoke the code generator before the
extensions are compiled. The 'build' command, however, is merely a
wrapper around several other commands. It is possible for the user to
directly invoke those commands, in which case our code generator won't
get a chance to run:

$ python setup.py build_ext
running build_ext
building 'libvirtmod' extension
creating build/temp.linux-x86_64-3.10
creating build/temp.linux-x86_64-3.10/build
gcc ..snip... -c build/libvirt.c -o build/temp.linux-x86_64-3.10/build/libvirt.o
cc1: fatal error: build/libvirt.c: No such file or directory
compilation terminated.
error: command '/usr/lib64/ccache/gcc' failed with exit code 1

To solve this we instead override 'build_ext' and 'build_py'. This in
turn means we call the generator to emit C code separately from Python
code.
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>

The distutils.dir_util.remove_tree method has no compelling benefit
over using the standard python shutils module.
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>

The distutils.spawn method has no compelling benefit over using the
standard python subprocess module.
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>

The distutils.spawn.find_executable method has no compelling benefit
over using the standard python shutils module.
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>

Instead of searching for the pkg-config binary manually, just try to run
it and catch the exception raised if it isn't found. Use the --version
flag as a way to validate that it is at least somewhat functional.
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>

The default 'clean' command impl deletes only intermediate files from
the 'build' directory. We've overridden it to delete everything. There
is no benefit in inheriting from the default impl, given our subclass
will delete everything.
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>

This duplicates funtionality already provided by the 'bdist_rpm'
command.
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>

Currently we always generate both the C code and Python code at the same
time. To cope with following changes to the build process, we want to be
able to generate C and Python separately.
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>

The names make it clearer exactly what is being generated by each.
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>

The python generator needs to know whether certain functions were
skipped in the C generator. This is achieved by the C generator
deleting skipped functions as it runs. This is an unhelpful side
effect as it makes it impossible to run the python generator
without first running the C generator.

This refactors buildStubs to get rid of the global side effects
it has, by providing some helper functions for buildWrappers
to use.
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>

As a side effect of generating the C code, the buildStubs methods
checks for various unsupported types and reports errors. This is
an undesirable side effect, if we want to skip C code generation.

Splitting function type validation out into a separate method
allows better reuse.
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>

The buildStubs method has a side effect of loading and parsing the API
XML files, which the buildWrappers method then relies on.

Splitting API loading into a separate method will facilitate running
only the buildWrappers method in future.
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>