What's the deal with boost.asio and file i/o?

I've noticed that boost.asio has a lot of examples involving sockets, serial ports, and all sorts of non-file examples. Google hasn't really turned up a lot for me that mentions if asio is a good or valid approach for doing asynchronous file i/o.

I've got gobs of data i'd like to write to disk asynchronously. This can be done with native overlapped io in Windows (my platform), but I'd prefer to have a platform independent solution.

I'm curious if

1. boost.asio has any kind of file support
2. boost.asio file support is mature enough for everyday file i/o
3. Will file support ever be added? Whats the outlook for this?

Has boost.asio any kind of file support?

Starting with (I think) Boost 1.36 (which contains Asio 1.2.0) you can use [boost::asio::]windows::stream_handle or windows::random_access_handle to wrap a HANDLE and perform asynchronous read and write methods on it that use the OVERLAPPED structure internally.

User Lazin also mentions boost::asio::windows::random_access_handle that can be used for async operations (e.g. named pipes, but also files).

Is boost.asio file support mature enough for everyday file i/o?

As Boost.Asio in itself is widely used by now, and the implementation uses overlapped IO internally, I would say yes.

Will file support ever be added? Whats the outlook for this?

As there's no roadmap found on the Asio website, I would say that there will be no new additions to Boost.Asio for this feature. Although there's always the chance of contributors adding code and classes to Boost.Asio. Maybe you can even contribute the missing parts yourself! :-)

boost::asio file i/o on Linux

On Linux, asio uses the epoll mechanism to detect if a socket/file descriptor is ready for reading/writing. If you attempt to use vanilla asio on a regular file on Linux you'll get an "operation not permitted" exception because epoll does not support regular files on Linux.

The workaround is to configure asio to use the select mechanism on Linux. You can do this by defining BOOST_ASIO_DISABLE_EPOLL. The trade-off here being select tends to be slower than epoll if you're working with a large number of open sockets. Open a file regularly using open() and then pass the file descriptor to a boost::asio::posix::stream_descriptor.

boost::asio file i/o on Windows

On Windows you can use boost::asio::windows::object_handle to wrap a Handle that was created from a file operation. See example.

boost::asio::windows::random_access_handle is the easiest way to do this, if you need something advanced, for example asynchronous LockFileEx or something else, you might extend asio, add your own asynchronous events. [example][1]

[1]: http://sourceforge.net/mailarchive/forum.php?thread_name=49222BF7.2080202%40synchromesh.com&forum_name=asio-users "asio-users mailing list"

ASIO supports overlapped I/O on Windows where support is good. On Unixes this idea has stagnated due to:

• Files are often located on the same physical device, accessing them sequentially is preferable.
• File requests often complete very rapidly because they are physically closeby.
• Files are often critical to complete the basic operation of a program (e.g. reading in its configuration file must be done before initializing further)

The one common exception is serving files directly to sockets. This is such a common special-case that Linux has a kernel function that handles this for you. Again, negating the reason to use asynchronous file I/O.

In Short: ASIO appears to reflect the underlying OS design philosophy, overlapped I/O being ignored by most Unix developers, so it is not supported on that platform.

io_uring has changed everything.
asio now support async file read/write.
See the releases notes:
asio 1.21.0 releases notes

Linux has an asio Library that is no harder to use than Windows APIs for this job (I've used it). Both sets of operating systems implement the same conceptual architecture. They differ in details that are relevant to writing a good library, but not to the point that you cannot have a common interface for both OS platforms (I've used one).

Basically, all flavors of Async File I/O follow the "Fry Cook" architecture. Here's what I mean in the context of a Read op: I (processing thread) go up to a fast food counter (OS) and ask for a cheeseburger (some data). It gives me a copy of my order ticket (some data structure) and issues a ticket in the back to the cook (the Kernel & file system) to cook my burger. I then go sit down or read my phone (do other work). Later, somebody announces that my burger is ready (a signal to the processing thread) and I collect my food (the read buffer).