Refactor: Record Per-Message Latency Data from the Receiver Process

[dustinblack]

To provide detailed analysis, the benchmark must be capable of recording the latency of every single message.
With the implementation of the multi-process model and end-to-end latency measurement, the correct latency is
now calculated within the child (receiver) process.

This task is to correctly channel this per-message data into the streaming output files only when explicitly
requested by the user via the --streaming-output-csv or --streaming-output-json flags.

Proposed Solution

The logic for handling per-message latency data needs to be conditional and managed carefully between the
parent and child processes.

1. Conditional Logic in Parent:

   • The parent process will check if either --streaming-output-csv or --streaming-output-json was provided.
   • If a streaming flag is present: The parent will open the user-specified file and pass its file handle (or
     path) to the child process.
   • If no streaming flag is present: The parent will create a temporary, intermediate file (e.g., in /tmp) and
     pass that file's handle to the child. This file will be invisible to the user.

2. Child Process (Receiver) Logic:

   • The child process receives a file handle from the parent. It does not need to know whether this is a
     user-requested file or a temporary one.
   • In its main receive loop, after calculating the end-to-end latency for each message, the child process
     will write the result as a new record to the provided file handle.

3. Parent Process (Finalization) Logic:

   • After the child process completes, the parent needs to calculate the final summary statistics (min, max,
     average, percentiles).
   • It will do this by reading the raw data from the file that the child wrote to (either the user's streaming
     file or the temporary file).
   • If a temporary file was used, the parent process will delete it immediately after the summary statistics
     have been calculated.

This approach ensures that:

• Streaming files are only created when the user explicitly requests them.
• The final summary is always calculated correctly, regardless of whether streaming is enabled.
• Raw per-message data is never displayed on stdout.

Acceptance Criteria

• When no streaming flags are used, no streaming files are created or left in the user's directory.
• When a streaming flag is used, the child (receiver) process writes all per-message latency records to the
  specified file.
• The data written to the streaming files accurately reflects the end-to-end latency calculated by the
  receiver.
• The final summary report is correctly calculated and displayed in all cases (with or without streaming
  enabled).

Context

This is a high-priority task (P2) for ensuring the benchmark provides detailed and analyzable results without
creating unwanted side effects.

This task is dependent on the completion of:

1. The multi-process refactoring (Issue Refactor: Switch from Multi-Threaded to Multi-Process Model for All IPC Mechanisms #74)
2. The end-to-end latency measurement implementation (Issue Implement End-to-End Latency Measurement for All IPC Mechanisms #75)

[mcurrier2]

Asked if this is already fixed in main:

This feature is largely already implemented on main. Here's the breakdown:

Requirement Status Details
Per-message latency from receiver/child Yes Child calculates receive_time - send_timestamp and writes to file
--streaming-output-csv / --streaming-output-json flags Yes CLI flags exist in src/cli.rs, wired up in both async and blocking modes
Conditional parent logic (user file vs temp file) Partially Child always writes to a temp file (/tmp/ipc_benchmark_latencies_*.txt); parent reads it back and optionally streams to user's CSV/JSON. The child never writes directly to the user's streaming file.
Child writes per-message latency records Yes Writes one latency value (nanoseconds) per line to the temp file
Parent reads raw data, calculates stats, deletes temp file Yes Parent reads temp file, feeds into MetricsCollector, streams to CSV/JSON if enabled, then deletes the temp file

Architecture difference from the spec: The spec suggests the child should write directly to the user's streaming file. The current implementation uses an intermediate temp file that the child writes to, then the parent reads it back and handles streaming output. This is functionally equivalent but with an extra hop.

Potential issues found (worth verifying separately):

Async format mismatch — The async server writes just latency_ns per line, but the parser in src/benchmark.rs may expect wall_send_ns,latency_ns (two-column format). This could cause parsing failures for async one-way benchmarks. The blocking path uses simple line.parse::() which matches the single-value format.

Constructor signature mismatch — MessageLatencyRecord::new may be called with 6 arguments in benchmark.rs but defined with 5 in results.rs. This would be a compile error, but since tests pass on main, there may be an updated signature or the code path isn't reached.