Yep, pretty much. Right now filesystem^ sources are finite, scanning the target path at operator startup time and processing all matching files. This processing is done by opening an asynchronous reader, courtesy of the object_store crate.
^We call these Filesystem Sources/Sinks to match terminology present in other streaming systems, but I'm not in love with it.
^We call these Filesystem Sources/Sinks to match terminology present in other streaming systems, but I'm not in love with it.