Mixed sync-async queue to interoperate between asyncio tasks and classic threads
Verified details
These details have been verified by PyPIProject links
Owner
GitHub Statistics
Maintainers
Project description
Mixed sync-async queue, supposed to be used for communicating between classic synchronous (threaded) code and asynchronous (in terms of asyncio) one.
Like Janus god the queue object from the library has two faces: synchronous and asynchronous interface.
Synchronous is fully compatible with standard queue, asynchronous one follows asyncio queue design.
Usage
Three queues are available:
Queue
LifoQueue
PriorityQueue
Each has two properties: sync_q and async_q.
Use the first to get synchronous interface and the second to get asynchronous one.
Example
import asyncio
import janus
def threaded(sync_q: janus.SyncQueue[int]) -> None:
for i in range(100):
sync_q.put(i)
sync_q.join()
async def async_coro(async_q: janus.AsyncQueue[int]) -> None:
for i in range(100):
val = await async_q.get()
assert val == i
async_q.task_done()
async def main() -> None:
queue: janus.Queue[int] = janus.Queue()
loop = asyncio.get_running_loop()
fut = loop.run_in_executor(None, threaded, queue.sync_q)
await async_coro(queue.async_q)
await fut
await queue.aclose()
asyncio.run(main())Limitations
This library is built using a classic thread-safe design. The design is time-tested, but has some limitations.
Once you are done working with a queue, you must properly close it using aclose(). This is because this library creates new tasks to notify other threads. If you do not properly close the queue, asyncio may generate error messages.
The library has quite good performance only when used as intended, that is, for communication between synchronous code and asynchronous one. For sync-only and async-only cases, use queues from queue and asyncio queue modules, otherwise the slowdown can be significant.
You cannot use queues for communicating between two different event loops because, like all asyncio primitives, they bind to the current one.
Development status is production/stable. The janus library is maintained to support the latest versions of Python and fixes, but no major changes will be made. If your application is performance-sensitive, or if you need any new features such as anyio support, try the experimental culsans library as an alternative.
Communication channels
GitHub Discussions: https://github.com/aio-libs/janus/discussions
Feel free to post your questions and ideas here.
gitter chat https://gitter.im/aio-libs/Lobby
License
janus library is offered under Apache 2 license.
Thanks
The library development is sponsored by DataRobot (https://datarobot.com)
