Depends on your setup but we've had success with building custom harnesses and processing metrics via OpenTelemetry + Prometheus. This is running on a massive K8s cluster spinning up pods per N agents.
If you are using a proprietary harness like CC or Codex then look into automatically ingesting their memory logs and internal datastores to track progress and then instrument the host to send back metrics
If you are using a proprietary harness like CC or Codex then look into automatically ingesting their memory logs and internal datastores to track progress and then instrument the host to send back metrics