ui: Show camera viewing ray when hovering a pixel in the image viewer - #4584
ui: Show camera viewing ray when hovering a pixel in the image viewer#4584behnamasadi wants to merge 1 commit into
Conversation
844f616 to
f43845f
Compare
|
@ahojnnes what do you say? |
ahojnnes
left a comment
There was a problem hiding this comment.
Thanks, it's a cool visualization to have but I am admittedly not very much convinced this adds a lot of practical value. Could you share a bit how you are using this in your workflow? Happy to be convinced if others find it useful.
Apart from these general concerns, it looks like you accidentally entangled a few other changes into this PR.
|
@ahojnnes Thanks for your reviews. I have rebased this on top of my other feature: #4579 |
f43845f to
2a0792c
Compare
Hovering the mouse over a pixel in the model viewer's image window now draws the back-projected viewing ray from that camera's center through the pixel into the 3D scene, as a visual check that a 2D observation lines up with the 3D geometry it should correspond to. The pixel is unprojected with Camera::CamFromImg and rotated into world coordinates; the ray is drawn with a dedicated LinePainter in the model viewer. The image viewer installs an event filter on its graphics view to track hover, and clears the ray when the cursor leaves the image. The per-image state needed for the back-projection (camera, camera center, rotation and ray length) is extracted once in ShowImageWithId() rather than on every mouse move. The ray length is the distance to the farthest 3D point observed by the image, so the ray stays visible at any model scale and does not degenerate when the camera sits near the world origin. Claude-Session: https://claude.ai/code/session_01JTjt8PHYuiZSJMFyPX16zB
2a0792c to
44d8463
Compare
|
@ahojnnes A rebase moved the line anchors, so the review threads above are marked outdated — all six points are addressed in the current head (
On the value question: I use it as a correspondence sanity check. Hovering a pixel and watching where its ray lands in the 3D view immediately shows whether the 2D observations and the triangulated geometry agree — a ray that misses the surface it visually points at means a bad pose, a wrong camera model, or a mis-registered image, and that is much faster to spot than reading reprojection-error numbers. It's the same idea as Rerun's ray picking. It's a debugging aid, not a workflow feature, and it costs nothing when unused: no state is built unless an image is open and the ray is only computed on hover. If you'd still rather not carry it in the GUI, I'm fine with closing — but the review comments are all resolved either way, so it's ready if you want it. |
|
@ahojnnes Gentle ping on this one. The six review threads are marked outdated after a rebase but all are addressed in |
Summary
Adds a rerun-style interaction to the model viewer: hovering a pixel in a
camera's image draws that pixel's back-projected viewing ray in the 3D view.
pixel into the 3D scene, updating live as the cursor moves and clearing when
it leaves the image.
highlights the corresponding 3D point (yellow); otherwise it extends across
the scene along the pixel's direction.
This makes the 2D image and the 3D reconstruction directly cross-referenceable:
point at anything in a photo and see where it lies in 3D.
Demo
How it works
its
QGraphicsViewviewport (mouse tracking on) and maps the cursor to imagepixel coordinates.
Camera::CamFromImg(pixel)gives the normalizedcamera-frame direction
(x, y, 1), transformed by the image pose(
Inverse(image.CamFromWorld())) into a world-space ray fromimage.ProjectionCenter().Point2Dwith apoint3D_idwithin ~1% of the image size is found; if present, the ray ends exactly at
that 3D point and highlights it.
LinePainter(ray) andPointPainter(highlight)render in
ModelViewerWidget::paintGL, using the samemodel_scale_ * (world + model_origin_)transform as the rest of the scene.Notes
model_viewer_widget.{h,cc}(SetHoverRay/ClearHoverRay + painters),image_viewer_widget.{h,cc}(event filter + unprojection).image_viewer_widget.hmoves
graphics_scene_/graphics_view_from private to protected so thedatabase image viewer can attach the hover filter.
-DGUI_ENABLED=ONand verified interactively; notcovered by the headless unit tests. The ray geometry was checked numerically
(computed pixel direction matches the camera→3D-point direction).
Note on the second commit (cherry-picked)
For convenience this branch also carries a second commit,
"ui: Add reprojection-error image colormap for camera frustums", cherry-picked
from #4579, so both GUI viewer features can be built and demoed from a single
branch. That colormap change is reviewed separately in #4579 — happy to drop
the commit here and keep this PR scoped to the hover-ray change alone if you
prefer.