Cool! I did this a while ago for some CPU-bound stuff, and ran into a bunch of performance bottlenecks.
Some things that helped me scale ptrace-interception up:
- SECCOMP_BPF filter (getting these right matters a lot)
- moving all of your intercept work to a single side (enter or exit)
- ensure affinity between the traced and tracing processes
- nuke vdso
- remove vdso from the aux vector (otherwise good libc's will find it again)
At the end of the day unfortunately the better solution would have been to write kernel support for what I wanted to do, but it's a fun exercise in learning about system calls.
Some things that helped me scale ptrace-interception up:
- SECCOMP_BPF filter (getting these right matters a lot)
- moving all of your intercept work to a single side (enter or exit)
- ensure affinity between the traced and tracing processes
- nuke vdso
- remove vdso from the aux vector (otherwise good libc's will find it again)
At the end of the day unfortunately the better solution would have been to write kernel support for what I wanted to do, but it's a fun exercise in learning about system calls.