Misleading DeviceProxy green_mode when using MultiTestDeviceContext with Async server device
Summary
Came across this when attempting to test an Async device with async DeviceProxys, using MultiDeviceTestContext
with process=False
. In this case, a device proxy created using DeviceProxy(test_context.get_device_access(...), green_mode=GreenMode.Asyncio)
will, despite reporting an Asyncio
green_mode, will actually silently run all code synchronously. This is caused by the in_executor_context()
check in the AbstractExecutor
here.
Expected behaviour
Ideally it would be nice for async proxies to work with an async server, but I get why that might be quite a tough technical challenge across multiple threads. At the very least I expected an error to be thrown, or maybe a warning to be logged which would have saved me a few hours of tracking down why my device proxy was synchronous.
Reproduction
Here is some pytest code illustrating the issue:
import asyncio
import pytest
from tango import GreenMode, get_device_proxy
from tango.green import get_executor
from tango.server import Device, attribute, command
from tango.test_context import MultiDeviceTestContext
class AsyncDevice(Device):
green_mode = GreenMode.Asyncio
def init_device(self):
self._value = 0
@attribute
def value(self):
return self._value
@command(dtype_out="DevShort")
async def increment(self):
asyncio.sleep(0.1)
self._value += 1
return self._value
config = ({"class": AsyncDevice, "devices": [{"name": "test/device/1"}]},)
@pytest.mark.asyncio
async def test_multi_device_test_context_limitations():
with MultiDeviceTestContext(config, process=False) as context:
fq_name = context.get_device_access("test/device/1")
# This should be awaitable since we passed green_mode=Asyncio
proxy = get_device_proxy(fq_name, green_mode=GreenMode.Asyncio)
assert proxy.get_green_mode() == GreenMode.Asyncio
# This should be an async call, but it isn't
value = proxy.increment()
assert value == 1
# Same here
attr = proxy.read_attribute("value")
assert attr.value == 1
# And finally, if this were async, the shorthand syntax shouldn't work
assert proxy.value == 1
def test_executor_is_not_in_right_context():
with MultiDeviceTestContext(config, process=False):
executor = get_executor(GreenMode.Asyncio)
# As our server device has a green_mode of Asyncio, the Asyncio
# executor gets initialised in the MultiDeviceTestContext thread
# which causes this in_executor_context() call (in the test thread)
# to return false.
assert not executor.in_executor_context()
More context
One way around this is to use process=True
for the MultiDeviceTestContext
, however our server device actually needs to communicate with other tango devices in the test context. To manage this we have a device factory with a optional static reference to the test context (which gets set by the tests once the context is initialised). If we run this in a process, then that reference is not maintained across the processes once the process has forked. We also cannot set the reference prior to the fork, since the server port is not known until the server starts... after it has forked.
Happy to clarify etc, this is a fiddly issue with many moving parts and I'm sure I've probably missed some crucial context somewhere.