Does asyncio supports asynchronous I/O for file operations?
PythonPython 3.xPython AsyncioPython 3.5Python Problem Overview
Does asyncio supports asynchronous I/O for file operations? If yes, how I can use this in Python 3.5 with async/await syntax code?
Python Solutions
Solution 1 - Python
Most operating systems don't support asynchronous file operations.
That's why asyncio doesn't support them either.
See the asyncio wiki for further explanation.
Solution 2 - Python
That depends on what library you use.
curio
curio offers this functionality, see https://curio.readthedocs.io/en/latest/reference.html#module-curio.file
asyncio
Update 2021: aiofile ~2 and ~3 (current) supports true asynchronous IO on Linux >= 4.18 via https://github.com/mosquito/caio and falls back to threaded implementations otherwise.
Plain asyncio doesn't, although there are 3rd party libraries, e.g. aiofiles (where synchronous file access is isolated in threads) and aiofile (note the spelling) (where synchronous file access is in threads in other circumstances than the above paragraph)
Modern operating systems do provide asynchronous file primitives, but these are varied, thus each would need own implementation. Please compare:
- http://man7.org/linux/man-pages/man7/aio.7.html
- https://msdn.microsoft.com/en-us/library/windows/desktop/aa365683(v=vs.85).aspx
- https://developer.apple.com/library/content/documentation/FileManagement/Conceptual/FileSystemProgrammingGuide/TechniquesforReadingandWritingCustomFiles/TechniquesforReadingandWritingCustomFiles.html
I suspect someone will soon rip out underlying async io from node.js and make a decent Python library, or perhaps someone already has.
Specifically for Linux, there are low-level bindings in https://pypi.org/project/liburing/
For a solid overview of asynchronous IO APIs in Linux, circa 2020, see https://www.scylladb.com/2020/05/05/how-io_uring-and-ebpf-will-revolutionize-programming-in-linux/
Solution 3 - Python
asyncio does not have support for this. However, aiofiles supports just this. Please have a look.
Solution 4 - Python
As per Python 3.9 this is possible to do with asyncio. https://docs.python.org/3.9/library/asyncio-task.html#asyncio.to_thread
await asyncio.to_thread(shutil.copyfile, "a", "b")