Unscrambler is a playground for interactive experiments in reorganizing visual signals. The current live version captures camera frames, uses correlations between pixels to calculate a t-SNE embedding, and renders the rearranged pixels as live, color-filled Voronoi cells with OpenCV.
Unscrambler begins with a simple proposition: coherence is not necessarily given by proximity; it can be produced by shared change.
In an ordinary digital image, neighboring pixels are assumed to belong together because they occupy nearby positions. This work suspends that assumption. Instead, it asks which signals behave as though they belong together. Pixels that repeatedly change in similar ways are drawn toward one another, allowing a face, a hand, or an object to emerge through a history of movement rather than through a fixed spatial grid. This closely echoes the Gestalt principle of “common fate”: elements moving together are perceived as a group.
The project was conceived in response to conventional computer-vision systems that gather information through rigid, local operations such as pooling. Inspired by the human ability to group things that move together even when they are not spatially connected, Unscrambler uses t-SNE to create a form of dynamic routing: an ephemeral visual space organized by correlation rather than physical distance.
This produces a paradox. Stillness may appear more clear, yet for the system it removes the evidence needed to discover structure. A motionless pixel has almost no relational identity. Movement gives it a history, and that history allows it to find its neighbors. Motion does not simply alter an already visible object; it helps constitute the object as something discernible.
The resulting image is therefore not a conventional portrait. It is a map of relationships revealed by the viewer's movement; dense islands where the system finds strong evidence of belonging, and stretched forms where that evidence is uncertain. The viewer animates the image in order to become visible to it: the system watches the person, while the person's movement teaches the system how to see them.
We do not move through an already coherent world; through movement, the world becomes coherent.
Install the locked core dependencies into a local .venv:
uv syncInstall and run the live OpenCV version with:
uv sync --group opencv
uv run --group opencv python live_unscrambler_mp.pyThe application uses the default camera. Press b to toggle cell borders, r
to start or stop an MP4 recording, s to save a PNG snapshot, and q to exit.
Recordings and snapshots are written to the ignored captures/ directory. The
red recording indicator is shown in the live window but is not included in the
saved video. Recordings use a wall-clock-paced 30 FPS timeline by default, so
their playback duration remains correct even when the live renderer runs at a
different frame rate.
On systems with multiple cameras, list their names and indices, then select one explicitly:
uv run --group opencv python live_unscrambler_mp.py --list-cameras
uv run --group opencv python live_unscrambler_mp.py --camera 1Camera indices can change when devices such as an iPhone Continuity Camera appear or disappear, so check them again after changing the camera setup.
To start without borders, run:
uv run --group opencv python live_unscrambler_mp.py --no-bordersInstall the additional notebook dependencies and start JupyterLab:
uv sync --group notebooks
uv run --group notebooks jupyter labSelect the Python interpreter in .venv as the notebook kernel.
The older old_unscramler.py experiment additionally uses scikit-video:
uv sync --group legacyThat script is retained as an archival experiment and still uses removed NumPy, scikit-learn, and dataset APIs; it needs a separate modernization pass before it can run on the current environment.