Skip to content

Latest commit

 

History

History
92 lines (68 loc) · 7.23 KB

RESULTS.md

File metadata and controls

92 lines (68 loc) · 7.23 KB

Batch-LIO vs Point-LIO — A/B Results

Date: 2026-06-17 · Machine: 32-core x86_64, CPU-only, Docker osrf/ros:noetic-desktop-full · Data: HKU-MARS Avia bags (/livox/lidar CustomMsg + /livox/imu).

What Batch-LIO changes vs Point-LIO (innovation #1, Point-LIWO §3): group LiDAR points into 1 ms windows instead of per-timestamp; de-skew each point to the window's reference time using the EKF state's ω/v (eq 3.44–3.47); row-stack residuals of all valid points → one EKF update per window. Plus optional OpenMP on the per-point KNN+plane-fit loop. State model identical to Point-LIO (use_imu_as_input=0). No wheel-speed (innovation #2 out of scope).

Reproducibility: each config = run/run_lio.sh <launch> <bag> <out> <rate> <logsrc> "<args>". Metrics via run/compare_traj.py. Per-frame timing = the node's [ mapping ] running averages (wall-clock, rate-independent). Trajectory diff aligned per-frame vs the Point-LIO baseline.


1. quick-shack (50 s, 491 frames, gentle handheld), playback rate 1.0

Config total ms icp ms propag ms start→end drift traj len mean |Δpos| vs base
Point-LIO baseline (point-wise) 7.69 6.21 0.17 0.0715 m 11.84 m —
batch 1 ms, OMP off 6.72 5.89 0.12 0.0528 m 11.53 m 0.0591 m
batch 1 ms, OMP on 3.35 2.44 0.13 0.0528 m 11.53 m 0.0591 m
OMP on point-wise (Task 1) 10.43 — — — — bit-identical
batch 20 ms, deskew OFF 6.27 5.49 0.11 7.59 m ⚠ 21.6 m 2.99 m
batch 20 ms, deskew ON 6.96 6.17 0.11 0.51 m 13.2 m 0.36 m

Takeaways (quick-shack):

  • Efficiency: batch 1 ms + OMP is 2.3× faster than baseline (7.69 → 3.35 ms/frame). Batching alone is ~13% faster (fewer EKF updates: ~100 windows/frame vs many tiny per-timestamp groups).
  • OMP only pays off after batching. On Point-LIO's tiny per-timestamp groups, OMP is slower (10.43 vs 7.69 ms — fork/join overhead). At 1 ms the groups are large enough that OMP gives ~2× on top. This is exactly the paper's "batch enables parallelism" claim.
  • Accuracy: batch 1 ms start→end drift (0.053 m) is lower than baseline (0.072 m); trajectory length matches (11.5 vs 11.8 m). The 0.059 m mean per-frame diff vs baseline is just two valid LIO estimates differing (no ground truth on this bag), not degradation.
  • De-skew is correct (decisive probe): amplifying the window to 20 ms, de-skew OFF diverges (7.59 m drift, 21.6 m garbage trajectory) while de-skew ON stays sane (0.51 m drift) — an ~8× drift reduction. This confirms the transform's sign and frame (Tⱼ = R_Iᵀ·v·Δtⱼ, resolving the paper-frame Open Question). At 1 ms de-skew is a near-no-op because 1 ms of gentle motion is sub-mm — expected, not a bug.

2. HKU_MB main building (260 s, 2602 frames processed, traverse), playback rate 2.0

Config total ms icp ms propag ms traj len mean |Δpos| vs base max |Δpos|
Point-LIO baseline 16.21 13.07 0.31 103.0 m — —
batch 1 ms, OMP off 12.26 11.03 0.12 105.9 m 0.031 m 0.203 m
batch 1 ms, OMP on 4.56 3.14 0.13 105.9 m 0.031 m 0.203 m

All three processed 2602/2602 frames (no drops at rate 2.0) and emitted 2602 odom msgs; no NaN/divergence.

Takeaways (HKU_MB):

  • Efficiency: batch 1 ms + OMP is 3.6× faster than baseline (16.21 → 4.56 ms/frame) — an even bigger win than quick-shack, because this denser scene puts more points in each 1 ms window, so OMP parallelizes more effectively. Batching alone is ~24% faster (16.21 → 12.26).
  • Accuracy: Batch-LIO matches the baseline trajectory to mean 0.031 m over a 103 m path (~0.03%), max 0.20 m — essentially identical estimation at a fraction of the compute.
  • Note: HKU_MB is a traverse (ends ~63.6 m from start), not a loop, so "start→end drift" is the true displacement (all configs agree at 63.64 m), not an error metric. With no ground truth, trajectory-consistency-vs-baseline is the accuracy measure here.

Headline: Batch-LIO reproduces Point-LIWO innovation #1 — 2.3–3.6× lower compute per frame at equal-or-better agreement with the Point-LIO baseline (no ground truth), with the speedup growing on denser scenes, exactly as the "batch enables parallelism + fewer updates" thesis predicts.


3. Ablations

3a. Odometry rate vs publish policy (correcting the earlier "×90 bandwidth" framing)

Mode odom rate
Point-LIO per-point (publish per point-group) ~6.7 kHz (per-point ⇒ noisy micro-updates)
batch 1 ms (publish per window) ~913 Hz
Point-LIO frame-rate odom ~10 Hz

Correction: the earlier "10 Hz → ~913 Hz (~90×)" compared batch's per-window rate against Point-LIO's per-FRAME rate — apples to oranges, which hid Point-LIO's ~6.7 kHz per-point mode. Batching reduces the update count (windows ≪ points), so the raw max publish rate actually goes down (913 Hz < 6.7 kHz); the real win is the 2.3–3.6× compute reduction, not bandwidth.

3b. batch_dt sweep (quick-shack, OMP on, de-skew on; baseline 7.69 ms / drift 0.072 m)

batch_dt total ms start→end drift mean |Δpos| vs base
0.5 ms 4.47 0.092 m 0.056 m
1 ms 3.44 0.053 m 0.059 m
2 ms 2.72 0.066 m 0.059 m
5 ms 2.22 0.143 m 0.104 m
10 ms 1.99 3.67 m ⚠ 1.81 m
20 ms 1.85 0.509 m 0.360 m

Speed rises monotonically with window size; accuracy holds through ~2 ms then degrades (constant-velocity-in-window assumption breaks). 1–2 ms is the sweet spot — accuracy on par with / better than baseline at 2.3–2.8× speedup — empirically justifying the paper's 1 ms default.

3c. Aggressive bag outdoor_run_100Hz (63 s, 6375 frames @100 Hz, a return-loop)

Config total ms start→end drift (loop closure) traj len
baseline (Point-LIO) 1.98 0.073 m 92.1 m
batch 1 ms, de-skew ON 0.66 0.020 m 97.2 m
batch 1 ms, de-skew OFF 0.67 0.039 m 97.5 m

3× faster, and de-skew measurably helps at 1 ms under fast motion (loop-closure 0.020 vs 0.039 m, 2× better) — and beats baseline's 0.073 m. (On gentle quick-shack the 1 ms de-skew effect was sub-mm; its benefit scales with window × speed, as the 20 ms probe showed dramatically.)

4. Validation chain (incremental, each committed)

  1. Clone == baseline (batch_dt=0): bit-exact trajectory (mean Δpos = 0.000000).
  2. OMP numerically faithful: OMP on/off give bit-identical trajectories.
  3. De-skew unit-tested: zero-dt, zero-motion, pure-translation, pure-rotation, world-v-into-body cases all pass.
  4. Grouping degenerate path reproduces Point-LIO exactly.
  5. De-skew correctness proven via the 20 ms amplification probe (8× drift reduction).

Git history in Batch-LIO/: ce1841f scaffold → 51ce9b4 OMP → d671aeb deskew+test → a28c699 1ms grouping → 859c060 deskew integration.