phonopaper-rs

A Rust library and command-line tool for encoding and decoding audio in the PhonoPaper format.

PhonoPaper (invented by Alexander Zolotov / NightRadio) represents audio as a camera-readable grayscale spectrogram image that can be printed on paper and played back by sweeping a phone camera across it.

Independence notice: phonopaper-rs is an independent, clean-room implementation written from scratch by studying the publicly available PhonoPaper format specification. It is not affiliated with, endorsed by, or derived from Alexander Zolotov's original work in any way. The PhonoPaper name and concept belong to their respective author.

AI disclosure: Large language models were used heavily during the development of this library — for code drafting, test generation, and documentation — under strict human supervision and validation.

Repository layout

This repository contains a Cargo workspace plus an Android app:

Component	Path	Description
`phonopaper-rs`	`phonopaper-rs/`	Core library — encode, decode, render, vector output
`phonopaper-cli`	`phonopaper-cli/`	`phonopaper` binary — four subcommands
`phonopaper-android`	`phonopaper-android/`	JNI bridge crate that exposes Rust detection (neural network) and decoding to Android
Android app	`android-app/`	Kotlin UI with live camera preview, capture/manual scrubbing controls, and a Rust-backed decoder
ML tooling	`phonopaper-ml/`	Separate workspace: deterministic synthetic dataset generator and a burn trainer for a neural-network pattern detector — see `phonopaper-ml/README.md`

Format summary

Property	Value
X axis	Time, left → right
Y axis	Frequency, logarithmic musical scale (top = high, bottom = low)
Pixel brightness	White = silence, black = maximum amplitude
Frequency bins	384 (8 octaves × 12 semitones × 4 subdivisions)
Frequency range	≈ 66 Hz (C2) … ≈ 4186 Hz (C8); bins 0–95 above C8 are above human hearing
Marker bands	Alternating black/white stripes at top and bottom for auto-calibration

For a detailed technical description see PHONOPAPER_SPEC.md.

Command-line tool

Installation

From this repository:

cargo install --path phonopaper-cli

Or run directly from the workspace:

cargo run -p phonopaper-cli -- <subcommand> [options]

Or, if you want to install the released version:

cargo install phonopaper-cli

`decode` — image → WAV

Convert a PhonoPaper image to a WAV audio file.

phonopaper decode <INPUT> [OUTPUT] [OPTIONS]

Supported input formats: PNG, JPEG, SVG, PDF. Default output: <input-stem>.wav (mono, 16-bit PCM).

Option	Default	Description
`--samples-per-column <N>`	`353`	PCM samples synthesised per image column (353 ≈ Android's 44 100 ÷ 125)
`--sample-rate <HZ>`	`44100`	Output sample rate
`--gain <GAIN>`	`3.0`	Master output gain (linear)
`--gamma <G>`	`1.0`	Inverse gamma for pixel → amplitude; must match the `--gamma` used when encoding
`--min-db <DB>`	`-60.0`	Lower bound of the dB window used when the image was encoded
`--max-db <DB>`	`-10.0`	Upper bound of the dB window used when the image was encoded
`--amplitude-threshold <T>`	off	Binary threshold (e.g. `0.85` for Android-like fidelity); conflicts with `--linear`
`--linear`	off	Linear amplitude mode with no threshold; conflicts with `--amplitude-threshold`

Examples:

phonopaper decode code.png
phonopaper decode code.png output.wav --gain 2.0
phonopaper decode scan.pdf output.wav
phonopaper decode code.svg output.wav --amplitude-threshold 0.85

`encode` — WAV / MP3 → image

Convert an audio file to a PhonoPaper image.

phonopaper encode <INPUT> [OUTPUT] [OPTIONS]

Supported input formats: WAV (any bit depth, mono or stereo), MP3. Supported output formats: PNG, JPEG, SVG, PDF (inferred from extension). Default output: <input-stem>.png.

Option	Default	Description
`--fft-window <N>`	`4096`	FFT window size in samples (power of two)
`--hop-size <N>`	`353`	Hop size between FFT frames; controls columns/second
`--px-per-octave <PX>`	`90`	Pixels per octave (total data height = value × 8)
`--gamma <G>`	`1.0`	Gamma correction applied to amplitudes before writing pixels
`--min-db <DB>`	`-60.0`	Bins quieter than this map to white (silence)
`--max-db <DB>`	`-10.0`	Bins at this level or louder map to black (maximum)
`--octave-lines`	off	Draw light-gray octave separator lines (⚠ produces audible hum on decode)
`--pdf-page <SIZE>`	`a4-landscape`	PDF page size for centred output; use `pixel-perfect` for a 1 px = 1 pt page (PDF only)

Supported --pdf-page values: a4-landscape (default), a4-portrait, a3-landscape, a3-portrait, letter-landscape, letter-portrait, pixel-perfect.

Examples:

phonopaper encode input.wav
phonopaper encode input.mp3 code.png
phonopaper encode input.wav code.svg
phonopaper encode input.wav code.pdf --px-per-octave 120

⚠ JPEG output applies lossy compression that can corrupt amplitude data and degrade decode quality. Prefer PNG for any software round-trip; JPEG is acceptable only when the image will be scanned by a phone camera.

`robust-decode` — camera photo → WAV

Decode a PhonoPaper print photographed by a camera, tolerating keystone distortion, tilt, and paper curl. Uses per-column marker detection followed by linear interpolation of the data boundaries — no explicit de-warp step is required.

phonopaper robust-decode --input <FILE> --output <FILE> [OPTIONS]

Option	Default	Description
`-i, --input <FILE>`	required	Input image (PNG or JPEG)
`-o, --output <FILE>`	required	Output WAV file (mono, 16-bit PCM)
`--sample-columns <N>`	`50`	Number of evenly-spaced columns to sample for marker detection
`--gain <GAIN>`	`3.0`	Master output gain (linear)
`--sample-rate <HZ>`	`44100`	Output sample rate
`--samples-per-column <N>`	`353`	PCM samples synthesised per image column
`--min-db <DB>`	`-60.0`	Lower bound of the dB window
`--max-db <DB>`	`-10.0`	Upper bound of the dB window
`--debug-image <FILE>`	off	Save a PNG with detected `data_top`/`data_bottom` lines overlaid
`--rectified <FILE>`	off	Save a perspective-rectified PNG of the data area

Example:

phonopaper robust-decode --input photo.jpg --output decoded.wav
phonopaper robust-decode --input photo.jpg --output decoded.wav \
    --debug-image debug.png --rectified rectified.png

`blank` — generate a blank template PDF

Generate a ready-to-print PhonoPaper template: marker bands, blank white data area for hand-drawn music, octave separator lines, octave labels (C2–C8), and a footer.

phonopaper blank [OPTIONS]

Option	Default	Description
`-o, --output <FILE>`	`blank_phonopaper.pdf`	Output PDF path
`--paper <FORMAT>`	`a4`	Paper size: `a4`, `a3`, `letter`, `legal`
`--portrait`	off	Portrait orientation (default is landscape)

Example:

phonopaper blank
phonopaper blank --paper a3 --output template_a3.pdf
phonopaper blank --paper letter --portrait

Android application (experimental)

The repository also includes an experimental Android application in android-app/. It keeps the UI in Kotlin but performs the PhonoPaper decoding pipeline in the phonopaper-android Rust cdylib, which reuses phonopaper-rs directly.

Current app flow:

Development worktree warning: the checked-in detector weights predate the current stride-2 heat-map architecture and cannot be loaded. The app detector remains unusable until a new model is trained and embedded with phonopaper-ml/scripts/embed-model.sh; see the ML README for the exact flow.

open the live camera preview and outline the PhonoPaper sheet found by the neural-network detector (phonopaper_rs::decode::nn) in real time, whatever its rotation or perspective — the thicker edges of the outline are the marker bands
automatically decode and play the current camera frame when live auto-play is enabled
capture the current camera frame or pick a PhonoPaper image from Android's document picker
decode it in Rust: the detected quadrilateral is rectified to an upright image with a homography, then read with the robust per-column marker interpolation path (images that are already upright are read directly)
play the synthesized mono PCM audio with AudioTrack, either from the current seek position or by scrubbing directly on the image preview

The Android resources also include a dedicated adaptive launcher icon so installed builds appear with a branded app icon in the launcher.

Build an installable release APK locally with:

cd android-app
./gradlew assembleRelease

The default release build uses the standard Android debug keystore so it remains directly installable for local testing while still using the release optimization profile. Set ANDROID_SIGNING_MODE=unsigned for an unsigned APK, or ANDROID_SIGNING_MODE=release together with the release-key environment variables described below. Pull-request CI uploads a temporary debug-signed APK.

Install main snapshots with Obtainium

The merge-queue CI builds the final unsigned APK once, with its CI run number as the Android version code. When that exact tested commit reaches main, a privileged workflow finds the successful merge-group run by commit SHA, validates the artifact metadata and checksum, signs it with the permanent key, and publishes an immutable GitHub prerelease tagged android-main-<merge-group CI run number>. Each prerelease contains the universal phonopaper-android-main.apk and its SHA-256 checksum. Direct pushes to main have no merge-group artifact and are deliberately not published.

This avoids a third Android build: the normal path builds once for the pull request and once for the merge queue, then only signs and publishes after the merge. Signing secrets are never exposed to pull-request or merge-group code.

To follow this channel in Obtainium:

Add https://github.com/evenfurther/phonopaper-rs as a GitHub source.
Enable inclusion of prereleases.
Set the release asset regular expression to ^phonopaper-android-main\.apk$ if Obtainium does not select it automatically.

The application ID is com.evenfurther.phonopaper. Main snapshots track current development and do not have the stability guarantees of a normal release. Android requires every update to have the same signing identity, so a previously installed local or CI debug-signed APK must be uninstalled once before installing the first Obtainium snapshot. This removes that installation's app data.

Release signing is configured by ANDROID_KEYSTORE_PATH, ANDROID_KEYSTORE_PASSWORD, ANDROID_KEY_ALIAS, and ANDROID_KEY_PASSWORD. All four variables must be supplied together. The post-merge signing workflow restores these values from the ANDROID_RELEASE_KEYSTORE_BASE64, ANDROID_RELEASE_KEYSTORE_PASSWORD, ANDROID_RELEASE_KEY_ALIAS, and ANDROID_RELEASE_KEY_PASSWORD repository secrets. The keystore and credentials are irreplaceable: keep encrypted offline backups because losing either prevents in-place updates to existing installs.

Library usage

Add to your Cargo.toml:

[dependencies]
phonopaper-rs = "0.1.0"

The crate name on disk is phonopaper-rs; the Rust module name is phonopaper_rs.

Optional Cargo features:

Feature	Effect
`nn-detector`	Enables `phonopaper_rs::decode::nn`, a neural-network detector that finds a `PhonoPaper` sheet and its four corners in a camera frame. Pulls in burn (CPU inference only) and embeds ≈ 1.2 MB of trained weights.

Decode an image to a WAV file

use phonopaper_rs::decode::{decode_image_to_wav, SynthesisOptions};

decode_image_to_wav("code.png", "output.wav", SynthesisOptions::default())?;
# Ok::<(), phonopaper_rs::PhonoPaperError>(())

Encode an audio file to an image

use phonopaper_rs::encode::{encode_audio_to_image, AnalysisOptions};
use phonopaper_rs::render::RenderOptions;

encode_audio_to_image(
    "input.wav",
    "code.png",
    AnalysisOptions::default(),
    RenderOptions::default(),
)?;
# Ok::<(), phonopaper_rs::PhonoPaperError>(())

Real-time / column-by-column playback

Synthesizer maintains oscillator phase across calls, avoiding clicks at column boundaries — matching the PhonoPaper Android app's playback model.

use phonopaper_rs::decode::{
    Synthesizer, SynthesisOptions,
    column_amplitudes_from_image, detect_markers,
};

let image = image::open("code.png")?;
let bounds = detect_markers(&image)?;

let mut synth = Synthesizer::<353>::new(SynthesisOptions::default());
let mut pcm = [0.0_f32; 353];

for col_x in 0..image.width() {
    let amps = column_amplitudes_from_image(&image, Some(bounds), col_x)?;
    synth.synthesize_column(&amps, &mut pcm);
    // Send `pcm` to an audio device...
}
# Ok::<(), phonopaper_rs::PhonoPaperError>(())

Vector output (SVG / PDF)

use phonopaper_rs::encode::{AnalysisOptions, audio_to_spectrogram};
use phonopaper_rs::render::RenderOptions;
use phonopaper_rs::vector::{PdfPageLayout, page_size, spectrogram_to_pdf, spectrogram_to_svg};

let (mono, sample_rate) = phonopaper_rs::audio::read_audio_file("input.wav")?;
let spec = audio_to_spectrogram(&mono, sample_rate, &AnalysisOptions::default())?;
let render = RenderOptions::default();

std::fs::write("code.svg", spectrogram_to_svg(&spec, &render))?;
// A4 landscape, 10 mm margins, image centred on the page:
std::fs::write("code.pdf", spectrogram_to_pdf(&spec, &render, PdfPageLayout::FitToPage {
    page_width_pt:  page_size::A4_LANDSCAPE.0,
    page_height_pt: page_size::A4_LANDSCAPE.1,
    margin_pt: 28.35,
}))?;
# Ok::<(), Box<dyn std::error::Error>>(())

Find a pattern in a camera frame (`nn-detector` feature)

The hand-written detect_markers expects an upright, axis-aligned pattern. With the nn-detector feature, a small convolutional network trained on synthetic data (see phonopaper-ml/) tells whether a frame contains a pattern at all and where its four corners are, whatever its rotation, perspective or scale:

use phonopaper_rs::decode::nn::PatternDetector;

let detector = PatternDetector::new(); // loads the embedded weights; keep it around
let frame = image::open("photo.jpg")?;
if let Some(corners) = detector.find_corners(&frame, 0.5) {
    // `corners` are [TL, TR, BR, BL] in frame pixels, clockwise; the marker
    // bands run along TL→TR and BR→BL.  A sheet looks the same upside down,
    // so which of those two edges is the high-frequency end is not knowable
    // from the image alone.
    println!("pattern at {corners:?}");
}

The frame is converted to grayscale and stretched to the 128 × 128 network input internally. For upright or approximately rectified images, refine_pattern_corners can conservatively refine coarse network corners to the outer marker ink box and reports confidence/support; it returns None for rotated, strongly skewed, or weakly supported inputs so callers can retain the coarse result. Accuracy claims must be re-established after training the new stride-2 model.

Public API overview

Module	Key items
`phonopaper_rs::error`	`PhonoPaperError`, `Result`
`phonopaper_rs::format`	`TOTAL_BINS`, `OCTAVES`, `SAMPLE_RATE`, `index_to_freq`, `freq_to_index`
`phonopaper_rs::spectrogram`	`Spectrogram<S>`, `SpectrogramVec`, `SpectrogramBuf`, `SpectrogramBufMut`
`phonopaper_rs::render`	`RenderOptions`, `spectrogram_to_image`, `spectrogram_to_image_buf`
`phonopaper_rs::vector`	`spectrogram_to_svg`, `spectrogram_to_pdf`, `image_from_svg`, `image_from_pdf`, `PdfPageLayout`, `page_size`
`phonopaper_rs::audio`	`read_audio_file` (WAV + MP3 → mono `f32` + sample rate)
`phonopaper_rs::encode`	`AnalysisOptions`, `audio_to_spectrogram`, `encode_audio_to_image`
`phonopaper_rs::decode`	`SynthesisOptions`, `AmplitudeMode`, `Synthesizer<SPS>`, `DataBounds`, `CornerRefinement`, `detect_markers`, `detect_markers_at_column`, `refine_pattern_corners`, `column_amplitudes_from_image`, `spectrogram_to_audio`, `decode_image_to_wav`, `decode_image_to_wav_sps`
`phonopaper_rs::decode::nn` (`nn-detector` feature)	`PatternDetector`, `Detection`, `Detector`, `DetectorConfig`, `load`, `prepare_image`, `embedded_config`

Supported formats

Direction	Format	Notes
Input audio	WAV	Any bit depth (8/16/24/32-bit int, 32-bit float); mono or stereo
Input audio	MP3	Via symphonia; mono or stereo
Input image	PNG	Lossless; preferred for software round-trips
Input image	JPEG	Accepted; lossy compression may degrade decode quality
Input image	SVG	Embedded raster image extracted automatically
Input image	PDF	Embedded raster image extracted automatically
Output image	PNG	Lossless; strongly preferred
Output image	JPEG	Lossy; acceptable for camera-scanned use cases only
Output image	SVG	Vector; marker bands as `<rect>` elements + embedded PNG
Output image	PDF	Vector; marker bands as rectangles + deflate-compressed raster XObject
Output audio	WAV	Mono, 16-bit signed PCM

Development

The Rust workspace checks remain:

cargo fmt --check --all
cargo clippy --workspace --all-targets
cargo test --workspace
cargo bench -p phonopaper-rs

The optional neural-network detector is not built by default; check it with:

cargo clippy -p phonopaper-rs --all-targets --features nn-detector
cargo test -p phonopaper-rs --features nn-detector

When you touch the Android integration, also build the APK:

cd android-app
./gradlew assembleRelease

The produced release APK is intended for test installs, not store distribution.

The machine-learning tooling in phonopaper-ml/ is its own workspace and has the same formatting / clippy / test gates:

cd phonopaper-ml
cargo fmt --check --all
cargo clippy --workspace --all-targets
cargo test --workspace

Coverage (informational, not a hard gate):

cargo llvm-cov -p phonopaper-rs --tests \
    --ignore-filename-regex='(benches|examples)' --summary-only

License

Licensed under either of Apache License, Version 2.0 or MIT License at your option.

Navigation Menu

Repository navigation

About

Repository files navigation

phonopaper-rs

Repository layout

Format summary

Command-line tool

Installation

`decode` — image → WAV

`encode` — WAV / MP3 → image

`robust-decode` — camera photo → WAV

`blank` — generate a blank template PDF

Android application (experimental)

Install main snapshots with Obtainium

Library usage

Decode an image to a WAV file

Encode an audio file to an image

Real-time / column-by-column playback

Vector output (SVG / PDF)

Find a pattern in a camera frame (`nn-detector` feature)

Public API overview

Supported formats

Development

License

About

Releases

Packages

Contributors

Languages

Name		Name	Last commit message	Last commit date
.cargo		.cargo
.github		.github
android-app		android-app
phonopaper-android		phonopaper-android
phonopaper-cli		phonopaper-cli
phonopaper-ml		phonopaper-ml
phonopaper-rs		phonopaper-rs
.gitignore		.gitignore
AGENTS.md		AGENTS.md
Cargo.lock		Cargo.lock
Cargo.toml		Cargo.toml
PHONOPAPER_SPEC.md		PHONOPAPER_SPEC.md
README.md		README.md

Latest commit

History

Folders and files

Repository files navigation

phonopaper-rs

Repository layout

Format summary

Command-line tool

Installation

decode — image → WAV

encode — WAV / MP3 → image

robust-decode — camera photo → WAV

blank — generate a blank template PDF

Android application (experimental)

Install main snapshots with Obtainium

Library usage

Decode an image to a WAV file

Encode an audio file to an image

Real-time / column-by-column playback

Vector output (SVG / PDF)

Find a pattern in a camera frame (nn-detector feature)

Public API overview

Supported formats

Development

License

About

Resources

Stars

Watchers

Forks

Releases

Packages

Contributors

Languages

`decode` — image → WAV

`encode` — WAV / MP3 → image

`robust-decode` — camera photo → WAV

`blank` — generate a blank template PDF

Find a pattern in a camera frame (`nn-detector` feature)