[question] How to pin iox-roudi process on isolated core

[ocpalo]

Hi,

I'm planning to use this library for IPC between two processes to achieve the lowest possible latency. I've thoroughly searched the API and documentation for information on pinning iox-roudi to isolated cores, but I couldn't find anything specific.

While it's possible to achieve this using shell commands, I'm unsure about the best way to configure iox-roudi to utilize 100% of the CPU for minimal latency.

Is this functionality supported by the library for my specific use case?

Thanks,
Berke

[ZhangWeiZePy]

You can use taskset cmd, such as taskset -c iox-roudi .

[ocpalo]

Hi, I am aware of that. However, iox-roudi is not polling the messages, this will not be effective.
Can we configure iox-roudi to polling so it can utilizes 100% cpu which isolated on.

[elfenpiff]

@ocpalo This should be feasible with some effort.

Currently, RouDi starts 2 background threads to handle

1. the incoming IPC messages like a new process is registered, deregistered or requests some kind of resource
2. discovery, connecting new publishers with subscribers, deliver the history etc.

We could add roudi command line options to define the CPU affinity of each thread with the corresponding priority and the strategy (push notifications or polling in busy loop).

Merging those two threads into one thread could be hard since this would require the iceoryx waitset to be able to listen to file descriptor events since thread 1. waits on a unix domain socket and 2. waits on a semaphore inside the current waitset and one cannot wait in one call on a unix domain socket and semaphore at the same time.

But for polling and a busy loop it wouldn't be an actual problem ... except for some refactoring.

[ocpalo]

Hi @elfenpiff

Do you have this feature on your milestones?

If you guide me a little, i can try to implement that feature. I am new to this repo but it seems promising.
It should not be that hard to refactor and implement that feature as a option.

[elfenpiff]

@ocpalo

Do you have this feature on your milestones?

No.

If you guide me a little, i can try to implement that feature. I am new to this repo but it seems promising.
It should not be that hard to refactor and implement that feature as a option.

I am happy to guide you through it. The interesting files would be

• iceoryx_posh/include/iceoryx_posh/internal/roudi/roudi.hpp
• iceoryx_posh/source/roudi/roudi.cpp

You have here two threads m_monitoringAndDiscoveryThread and m_handleRuntimeMessageThread. So the first task would be to cleanup the code so that:

1. There is an option to run the tasks in a busy loop
2. To create a function that runs that busy loop and put this into one thread
3. Add CPU affinity and priority to our existing thread abstraction in iceoryx_hoofs/posix/sync/include/iox/thread.hpp
4. Use this thread abstraction to spawn the new thread with pinned CPU and priority.

Implementing this is some effort and before you start I want to hint you to iceoryx2. Currently, our main focus is on the next generation of iceoryx2 which has some significant improvements:

• no longer requires a central broker like roudi
• is up to 10 times faster than iceoryx1
• no longer requires a compiled in memory configuration
• supports dynamic data in publishers
• has a language bindings for C & C++
• completely decentralized and much more robust
• much more extensible and will in the future also support GPUs, FPGAs etc
• will soon have an iceoryx2_rmw binding for ROS2
• will soon have gateways to zenoh/dds

The only feature missing is request response but this will be implemented until the end of this year.

If you want to have a future proof solution and also want to benefit from all the new features we are currently developing (including tooling) I would recommend iceoryx2.
But we are also here for you if you choose to use good old iceoryx