Tags give the ability to mark specific points in history as being important
  • python-pull-request
    Python patches
    
    A few fixes to the Python CI tests, a few fixes to the (async) QMP
    library, and a set of patches that begin to shift us towards using the
    new qmp lib.
    
  • ide-pull-request
    Pull request
    
  • python-async-qmp-aqmp-v1-errata
  • python-async-qmp-aqmp-v1
    python: introduce Asynchronous QMP package
    
    GitLab: https://gitlab.com/jsnow/qemu/-/commits/python-async-qmp-aqmp
    CI: https://gitlab.com/jsnow/qemu/-/pipelines/330003554
    Docs: https://people.redhat.com/~jsnow/sphinx/html/qemu.aqmp.html
    Based-on: <20210701020921.1679468-1-jsnow@redhat.com>
              [PULL 00/15] Python patches
    
    Hi!
    
    This patch series adds an Asynchronous QMP package to the Python
    library. It offers a few improvements over the previous library:
    
    - out-of-band support
    - true asynchronous event support
    - avoids undocumented interfaces abusing non-blocking sockets
    
    This library serves as the basis for a new qmp-shell program that will
    offer improved reconnection support, true asynchronous display of
    events, VM and job status update notifiers, and so on.
    
    My intent is to eventually publish this library directly to PyPI as a
    standalone package. I would like to phase out our usage of the old QMP
    library over time; eventually replacing it entirely with this one.
    
    This series looks big by line count, but it's *mostly*
    docstrings. Seriously!
    
    This package has *no* external dependencies whatsoever.
    
    Notes & Design
    ==============
    
    Here are some notes on the design of how the library works, to serve as
    a primer for review; however I also **highly recommend** browsing the
    generated Sphinx documentation for this series.
    
    Here's that link again:
    https://people.redhat.com/~jsnow/sphinx/html/qemu.aqmp.html
    
    The core machinery is split between the AsyncProtocol and QMP
    classes. AsyncProtocol provides the generic machinery, while QMP
    provides the QMP-specific details.
    
    The design uses two independent coroutines that act as the "bottom
    half", a writer task and a reader task. These tasks run for the duration
    of the connection and independently send and receive messages,
    respectively.
    
    A third task, disconnect, is scheduled asynchronously whenever an
    unrecoverable error occurs and facilitates coalescing of the other two
    tasks.
    
    This diagram for how execute() operates may be helpful for understanding
    how AsyncProtocol is laid out. The arrows indicate the direction of a
    QMP message; the long horizontal dash indicates the separation between
    the upper and lower half of the event loop. The queue mechanisms between
    both dashes serve as the intermediaries between the upper and lower
    half.
    
                           +---------+
                           | caller  |
                           +---------+
                               ^ |
                               | v
                           +---------+
         +---------------> |execute()| -----------+
         |                 +---------+            |
         |                                        |
    [-----------------------------------------------------------]
         |                                        |
         |                                        v
    +----+------+    +----------------+    +------+-------+
    | ExecQueue |    | EventListeners |    |Outbound Queue|
    +----+------+    +----+-----------+    +------+-------+
         ^                ^                       |
         |                |                       |
    [-----------------------------------------------------------]
         |                |                       |
         |                |                       v
      +--+----------------+---+       +-----------+-----------+
      | Reader Task/Coroutine |       | Writer Task/Coroutine |
      +-----------+-----------+       +-----------+-----------+
                  ^                               |
                  |                               v
            +-----+------+                  +-----+------+
            |StreamReader|                  |StreamWriter|
            +------------+                  +------------+
    
    The caller will invoke execute(), which in turn will deposit a message
    in the outbound send queue. This will wake up the writer task, which
    well send the message over the wire.
    
    The execute() method will then yield to wait for a reply delivered to an
    execution queue created solely for that execute statement.
    
    When a message arrives, the Reader task will unblock and route the
    message either to the EventListener subsystem, or place it in the
    appropriate pending execution queue.
    
    Once a message is placed in the pending execution queue, execute() will
    unblock and the execution will conclude, returning the result of the RPC
    call to the caller.
    
    Ugly Bits
    =========
    
    - MultiException is ... wonky. I am still working out how to avoid needing it.
      See patch 04/20 for details here, or see
      https://people.redhat.com/~jsnow/sphinx/html/qemu.aqmp.error.html
    
      Patch 06/20 also goes into details of the ugliness; see
      AsyncProtocol._results or view the same information here:
      https://people.redhat.com/~jsnow/sphinx/html/_modules/qemu/aqmp/protocol.html#AsyncProtocol._results
    
    - There are quite a few lingering questions I have over the design of the
      EventListener subsystem; I wrote about those ugly bits in excruciating detail
      in patch 14/20.
    
      You can view them formatted nicely here:
      https://people.redhat.com/~jsnow/sphinx/html/qemu.aqmp.events.html#experimental-interfaces-design-issues
    
    Patch Layout
    ============
    
    Patches 1-2 are tiny pylint configuration changes.
    Patches 3-5 begin to check in Async QMP components, they are small.
    Patches 6-11 add a generic async message-based protocol class,
                 AsyncProto. They are split as small as I could
                 reasonably split them.
    Patches 12-14 check in more QMP-specific components.
    Patches 15-18 add qmp_protocol.py, the new 'QMP' class. They're split as
                  far down as I could, I hope they're easy to review.
    Patches 19-20 add a few finishing touches, they are small patches.
    
    Future Work
    ===========
    
    These items are in progress:
    
    - A Synchronous QMP wrapper that allows this library to be easily used
      from non-async code; this will also allow me to prove it works well by
      demoing its replacement throughout iotests.
    
    - A QMP server class; to facilitate writing of unit tests.
    
    - Unit tests. Real unit tests.
    
    If you made it this far in the cover letter, congrats :)
  • aqmp-draft-v7
  • floppy-pull-request
    FDC Pull request
    
  • aqmp-draft-v6
  • aqmp-draft-v5
  • v6.0.0
    v6.0.0 release
    
  • v6.0.0-rc5
    v6.0.0-rc5 release
    
  • v6.0.0-rc4
    v6.0.0-rc4 release
    
  • v6.0.0-rc3
    v6.0.0-rc3 release
    
  • v6.0.0-rc2
    v6.0.0-rc2 release
    
  • v6.0.0-rc1
    v6.0.0-rc1 release
    
  • v6.0.0-rc0
    v6.0.0-rc0 release
    
  • v5.2.0
    v5.2.0 release
    
  • v5.2.0-rc4
    v5.2.0-rc4 release
    
  • v5.2.0-rc3
    v5.2.0-rc3 release
    
  • v5.2.0-rc2
    v5.2.0-rc2 release
    
  • v5.2.0-rc1
    v5.2.0-rc1 release