elBoberido

iceoryx2 maintainer here. Of course everybody has different requirements when it comes to IPC and if a rich set of supported languages is the highest priority, then protocols like JSONRPC are a good option. But using such protocols for local IPC also has downsides, especially when the amount of data increases and the available resources are limited. This is where iceoryx2 has its strengths by utilizing shared memory for communication. We also split up the data flow from the control flow and one could send data to other processes without a context switch, in a way similar to what io_uring is doing. There is an upcoming blog post where we describe the possibilities shared memory enables compared to using sockets. I'll post it here once it is published.

Like another poster already commented, while iceoryx2 is implemented in Rust, it already support other languages, like C, C++, C# and Python. We'd love to add even more languages to the list, with Go being on top of our wishlist, but like always with OSS, someone needs to do the work or needs to step up and fund the work.

I'm one of the iceoryx maintainer :)

Yes, there is some overlap but iceoryx and threadprocs could complement each other. Currently, we achieve true zero copy only with data types that are shared memory compatible, i.e. no heap, no self references, etc., basically the data type must be memcopyable. Cap'n Proto or the Apache Arrow columnar format would also work. For other data, e.g one that uses the heap, there would still a serialization step be required but at least there is no further copy between the processes.

With threadprocs, it would be possible to also send data structures that use the heap or are self-referential. iceoryx2 already works in the same way as the proposed solution for the deallocation. There is a submission channel to provide the data and a completion channel where the pointer are sent back to the producer. Currently, we do not call any d'tor, since it is not necessary, but that should not be too difficult to implement.

Btw, with the next release iceoryx2 will have Python bindings. They are already on main and we will make it available via PIP. This should make it easier to use with Pytorch.

Thanks, really appreciate it.

Request-response is on our todo list and will be introduced in an upcoming release :)

What are you needing request-response for?

Indeed. That's our goal :)

Sweet. Can we link to your benchmark from the main iceoryx2 readme?

> In my experience shared memory is really hard to implement well and manage:

I second that. It took us quite some time to get the correct architecture. After all, iceoryx2 is the third incarnation of this piece of software, with elfepiff an me working on the last two.

> 1. Unless you're using either fixed sized or specially allocated structures, you end up paying for serialization anyhow (zero copy is actually one copy).

Indeed, we are using fixed size structures with a bucket allocator. We have ideas on how to enable the usage on types which support custom allocators and even with raw pointers but that is just a crazy idea which might not pan out to work.

> 2. There's no way to reference count the shared memory - if a reader crashes, it holds on to the memory it was reading. You can get around this with some form of watchdog process, or by other schemes with a side channel, but it's not "easy". > > 3. Similar to 2, if a writer crashes, it will leave behind junk in whatever filesystem you are using to hold the shared memory.

Indeed, this is a complicated topic and support from the OS would be appreciated. We found a few ways on how to make this feasible, though.

The origins of iceoryx are in automotive and there it is required to split functionality up into multiple processes. When one process goes down, the system can still operate in a degraded mode or just restart the faulty process. With this, one needs an efficient and low-latency solution else the CPU is spending more time on copying data than on doing real work.

Of course there are issues like the producer mutating data after delivery, but here are also solutions for this. It will of course affect the latency but should still be better than using e.g. unix domain sockets.

Fun fact. For iceoryx1 we supported only 4GB memory chunks and some time ago someone came and asked if we could lift this limitation since he wanted to transfer a 92GB large language model via shared memory.

Besides being written in Rust, the big difference is the decentralized approach. With iceoryx1 a central daemon is required but with iceoryx2 this in not the case anymore. Furthermore, more fine grained control over the resources like memory and endpoints like publisher. Overall the architecture is more modular and it should be easier to port iceoryx2 to even more platforms and customize it with 3rd party extension.

With this release we have initial support for C and C++. Not all features of the Rust version are supported yet, but the plan is to finish the bindings with the next release. Furthermore, with an upcoming release we will make it trivial to communicate between Rust, C and C++ applications and all the other language bindings we are going to provide, with Python being probably the next one.

Don't worry. It's great to have other projects in this field, exploring different routes and you created a great piece of software. The best thing, it's all open source after all :)

Reading your response is almost as you've been at our coffee chats. Quite some of your ideas are also either already implemented in iceoryx2 or on our todo list. It seems we just put our focus on different things. Here and there you also added the cherry on top. This motivates us to improve on some areas we neglected the last years. We can learn from each other and improve our projects thanks to the beauty of open source.

Keep up the good work

We are waiting with some salt & vinegar crisps ;)

Hehe, we are also using fancy-pointer in some places :)

We started with mapping to the shm to the same address but soon noticed that it was not a good idea. It works until some application already mapped something to the same address. It's good that you did not went that route.

I hoped you had an epiphany and found a nice solution for the raw-pointer problem without the need to change them and we could borrow that idea :) Replacing the raw-pointer with fancy-pointer is indeed much simpler than replacing the whole logic.

Since the raw-pointer need to be replaced by fancy-pointer, how do you handle STL container? Is there a way to replace the pointer type or some other magic?

Hehe, we have something called 'relative_ptr' which also tracks the segment ID + offset. It is a struct of two uint64_t though. Later on, we needed to condense it to 64 bit to prevent torn writes in our lock-free queue exchange. We went the same route and encoded the segment ID in the upper 16 bits since only 48 bits are used for addressing. It's kind of funny that other devs also converge to similar solutions. We also have something called 'relocatable_ptr'. This one tracks only the offset to itself and is nice to build relocatable structures which can be memcopied as long as the offset points to a place withing the copied memory. It's essentially the 'boost::offset_ptr'.

Btw, when you use jemalloc, do you free the memory from a different process than from which you allocate? We did the same for iceory1 but moved to a submission-queue/completion-queue architecture to reduce complexity in the allocator and free the memory in the same process that did the allocation. With iceoryx2 we also plan to be more dynamic and have ideas to implement multiple allocators with different characteristics. Funnily, jmalloc is also on the table for use-cases where fragmentation is not a big problem. Maybe we can create a common library for shm allocating strategies which can be used for both projects.

Now I'm curious. It's seems you are not the father I'm still drinking beer with. This means there is only one person left that fits this attribute :) ... we should meet for some beer with the other father ;)