A Rigorous Hardware-Validated Benchmarking Study of Forward-Only Algorithms
Forward-Forward, Cascaded Forward and Mono-Forward measured on an NVIDIA A100
against tuned backpropagation, with energy, power and memory read from the device.
-1f4e79.svg?style=flat-square){kind=icon}
Paper · Key Results · Reproduce · Quickstart · Citation
Paper: Energy-Efficient Deep Learning Without Backpropagation: A Rigorous Hardware-Validated Benchmarking Study of Forward-Only Algorithms, accepted at PPAM 2026 (Springer LNCS). The paper source and PDF are maintained separately from this code repository. An earlier preprint is arXiv:2511.01061.
Authors: Przemysław Spyra, Witold Dzwinel · AGH University of Krakow, Faculty of Computer Science
Contents
- Overview
- Key Results
- Follow-up studies
- Reproducing the Experiments
- Repository Structure
- Installation
- Data
- Quickstart
- Configuration
- Checkpoints
- Scope and Limitations
- Citation
- License and Acknowledgements
Backpropagation (BP) keeps every activation in memory until a global backward sweep consumes it. Forward-only algorithms replace that global gradient with local learning signals and are usually promoted as more efficient on the basis of FLOP counts. This repository tests the claim on hardware: across almost 1000 runs on MNIST, Fashion-MNIST, CIFAR-10 and CIFAR-100, every run is instrumented through NVML at 5 Hz for power, utilisation, clock and memory, and each algorithm is compared with a BP baseline tuned under the same budget on the same architecture.
| Algorithm | Backward pass | Learning signal | Studied on |
|---|---|---|---|
| BP | yes | global end-to-end loss | every architecture (baseline) |
| Forward-Forward (FF) | no | layer-local goodness | MLPs |
| Cascaded Forward (CaFo) | no | per-block local predictors | 3-block CNNs |
| Mono-Forward (MF) | no | layer-local cross-entropy through learned projections | MLPs |
In short. Removing the backward pass alone saves nothing: FF costs 4.1× to 6.3× BP's energy and CaFo 3.0× to 7.1×. Mono-Forward is different. Given the same number of data passes as BP, it uses 12.5% to 19.9% less energy and up to 27.5% less memory on the largest MLPs tested, and on CIFAR-100 its accuracy is statistically equivalent to BP's. Trained to its own convergence, MF exceeds BP's test accuracy on CIFAR-10 and CIFAR-100 by 3.90 and 5.04 percentage points, at 3.2× to 4.0× BP's energy because it needs 3.5× to 4.5× as many passes. The conference paper's ablation attributed the accuracy gain to gradient detachment. The pre-registered follow-up experiments (G1, W1) attribute it to MF's stage-by-stage training schedule instead: detachment alone lands at BP's accuracy, and letting the earlier layers keep training preserves the gain. Against a BP given the same compute and a tuned cosine schedule, MF's edge is 3.74 pp on CIFAR-100 and 0.78 pp on CIFAR-10 (see Follow-up studies).
Under matched passes, the accuracy costs shown below are −1.11 pp on CIFAR-10, −0.22 pp on MNIST, and −0.49 pp on Fashion-MNIST.
All numbers below are produced by the released analysis; none is transcribed by hand. The raw run
records are archived separately (TODO: Zenodo DOI).
Every run stops under one rule (validation loss, patience 20, cap 100 epochs per training unit),
and the paper reports two protocols because no single one answers both questions a practitioner
asks:
- Protocol A, own convergence: each algorithm trains until it stops improving.
- Protocol B, matched data passes: MF receives at least as many data passes as BP actually used, so the stopping rule cannot confound the cost comparison.
Memory is the per-process allocator peak, torch.cuda.max_memory_allocated, never the
device-wide NVML reading, which includes a ≈870 MiB CUDA context no algorithm controls.
MF has n = 20 seeds per configuration; Δ is MF relative to BP, so negative cost is cheaper.
| Configuration | Protocol | Δ accuracy (pp) | Δ energy | Δ memory | Δ power | Δ time |
|---|---|---|---|---|---|---|
| CIFAR-10 MLP 3×2000 | B, matched passes | −1.11 | −19.9% | −27.5% | −17.6% | −2.8% |
| CIFAR-100 MLP 3×2000 | B, matched passes | +0.02 (equivalent) | −12.5% | −24.6% | −12.8% | +0.4% |
| MNIST MLP 2×1000 | B, matched passes | −0.22 | −2.2% | +0.3% | −5.8% | +3.8% |
| Fashion-MNIST MLP 2×1000 | B, matched passes | −0.49 | −4.3% | +0.3% | −3.6% | −0.8% |
| CIFAR-10 MLP 3×2000 | A, own convergence | +3.90 | 3.20× | −27.5% | −19.1% | 3.96× |
| CIFAR-100 MLP 3×2000 | A, own convergence | +5.04 | 3.77× | −24.6% | −14.3% | 4.39× |
Read the two protocols together. Per data pass, MF is the cheaper algorithm, and the saving is a power saving at near-equal wall-clock time. Trained to convergence it is more expensive end to end, because it needs 3.5× to 4.5× as many passes. The pass multiplier, not the cost of a pass, is what stands between MF and an end-to-end energy advantage.
The ladder moves from BP to MF one structural change at a time:
BP → BP with deep supervision (BP-DS) → MF-Joint (MF's readout, global gradients) → MF (local
gradients). The analysis script, its tests, Holm correction and the ±0.25 pp equivalence margin
were committed before any ladder result was inspected
(scripts/analyze_ablation_ladder.py).
- Deep supervision alone makes BP worse on CIFAR-10 and CIFAR-100, by 1.65 and 1.06 pp.
- MF beats that deep supervision control by 5.56 and 6.10 pp (Holm-corrected p < 10⁻²⁰).
- The detach step, with the optimiser family held fixed, is the largest single gain (+4.10 and +3.18 pp), and it also carries essentially the whole energy increase. This step also switches from joint to stage-by-stage training; the follow-up experiments assign the gain to that schedule, not to detachment (see Follow-up studies).
- On the two 2×1000 MLPs, MF and the deep supervision control are equivalent within the margin.
- FF trails BP by 0.01 to 1.10 pp on its native MLPs while needing 4.1× to 7.4× BP's wall-clock time and 4.1× to 6.3× its energy, at 0.91× to 1.06× its allocator memory.
- CaFo costs 3.0× to 7.1× BP's energy. It lands within 0.4 pp of BP on the two MNIST-scale CNNs but loses 18.8 and 16.9 pp on CIFAR-10 and CIFAR-100; its DFA variant narrows the gap to 7.1 and 8.5 pp at a further 1.5× to 2.6× the energy.
The complete per-configuration FF and CaFo table belongs to the paper archive and is not included in this code-only checkout.
Three pre-registered experiments extend the conference paper. Each was frozen (code, configs,
seeds, hypotheses and decision rules) before its data were analysed, and each analysis was run
once. Protocols and dated amendment logs are in prereg/; the frozen outputs are in
analysis/. All runs use the 3×2000 MLP on CIFAR-10 and CIFAR-100 with paired seeds
42–61 on A100-40GB GPUs; CIFAR-100 is the confirmatory dataset and CIFAR-10 the consistency check.
G1: detachment or staging? (prereg/PREREG.md, 160 runs). MF as its
authors define it (every layer updated on every mini-batch, detached inputs; MF-Simul) lands at
BP's accuracy: −0.86 pp on CIFAR-10 and −0.27 pp on CIFAR-100. The stage-by-stage schedule used
in the conference paper (MF-Seq) keeps its +3.90 and +5.04 pp at 3.99× and 4.36× BP's passes.
The accuracy edge and its cost belong to the schedule, not to removing the backward pass.
W1: which part of the schedule? (prereg/PREREG-phase2.md,
440 runs, 11 arms).
- Letting the earlier layers keep training through backpropagation (Seq-Global) does not remove the edge: +0.90 pp [+0.69, +1.14] over MF-Seq on CIFAR-100 (−1.17 pp on CIFAR-10). The gain comes from stage-wise training, not from layer-local credit assignment.
- MF-Seq's first stage (a classifier on the raw input, M0) is overhead: removing it changes accuracy by −0.00 pp (CIFAR-100) and +0.06 pp (CIFAR-10) and saves 22–24% of passes and energy.
- Against BP with a tuned cosine schedule and MF-Seq's pass budget, MF-Seq leads by 3.74 pp on CIFAR-100 and 0.78 pp on CIFAR-10.
- At BP's own pass budget, MF-Seq matches BP on CIFAR-100 (+0.02 pp) with 0.857× its energy; on CIFAR-10 it trails by 0.81 pp at 0.912×.
G2: per-pass cost under load and an initialisation control
(prereg/PREREG-G2-DRAFT.md, 80 jobs). Energy is read from the NVML
energy counter, net of each job's idle power, with every arm of a repeat on the same GPU.
- With the production data loader the GPU is mostly idle: in gross energy an MF-Seq stage pass is 1.16–1.19× cheaper than a BP pass (W1 measured 1.16×), but net of idle it is 2.45–3.06× cheaper. With the input held on the GPU, the stages are 2.1–2.4× cheaper even in gross energy.
- MF-Simul's per-pass advantage grows with batch × width and reaches 1.44× [1.43, 1.46] at batch 8192 and width 8000; at the W1 size it is no cheaper than BP in gross energy.
- Combining these per-pass energies with W1's pass counts (a projection, not a measurement), staged MF without M0 would still use 1.3–1.6× BP's energy at its own convergence.
- BP's default initialisation differs from MF's (He-uniform, zero biases). Training BP with MF's initialisation lowers its accuracy by 0.81 pp (CIFAR-100) and 1.08 pp (CIFAR-10), so the BP-versus-MF comparisons above are not inflated in MF's favour by this difference.
Corrections to the conference paper. Its measured numbers stand; two readings change. (1) The ablation ladder credited the accuracy gain to gradient detachment; G1 and W1 assign it to the stage-by-stage schedule. (2) The +3.90 and +5.04 pp edge is relative to an early-stopped BP with a constant learning rate; against a scheduled BP with the same compute it is 0.78 and 3.74 pp. The Protocol B energy saving is a property of staged MF, not of detachment as such.
Reproducing the analyses (the per-run records are archived separately):
PYTHONPATH=. python scripts/jpdc/analyze_g1.py --results-dir <results> --out-dir <out>
PYTHONPATH=. python scripts/jpdc/analyze_w1.py --results-dir <results> --out-dir <out>
PYTHONPATH=. python scripts/jpdc/analyze_g2.py --sweep-dir <sweep> --bridge-dir <bridge> \
--results-dir <results> --out-dir <out>The code repository retains the tidy measurements and experiment-side analysis needed to inspect the released results. The LaTeX source, generated paper tables, and archived raw run traces are maintained separately.
| Artefact | Location |
|---|---|
| Tidy table of 19 944 measurements | Generated locally from run records |
| Builder of the tidy table from raw run records | scripts/build_tidy_table.py |
| Pre-specified ladder analysis | scripts/analyze_ablation_ladder.py |
| Bit-exactness check of the MF activation cache | scripts/check_mf_cache_equivalence.py |
| The per-run JSON records and the 1027 NVML power traces are archived separately and available from | |
the authors on request (TODO: Zenodo DOI). |
The configs/diagnostics/*_legacy_es.yaml files are stopping-policy sensitivity diagnostics, not
canonical result configurations; the paper does not report them as headline results.
Experiment configurations for each arm of the study:
| Arm | Configs |
|---|---|
| BP baselines | configs/bp_baselines/ |
| Mono-Forward | configs/mf/ |
| Ablation ladder controls | configs/bp_ds/, configs/mf_joint/ |
| Matched data-pass budgets | configs/diagnostics/ |
| Forward-Forward, CaFo | configs/ff/, configs/cafo/ |
Run the study in this order on an A100-40GB, using the seed lists in each config; the MF
comparisons use n = 20 seeds per configuration. Run outputs are written below results/, with
the main arms in results/phase4, ladder controls in results/runs, matched-pass diagnostics in
results/equal_epochs, and reproduction checks in results/reproduction.
# BP baselines
python scripts/run_local_array.py --config-dir configs/bp_baselines/
# MF, FF and CaFo
python scripts/run_local_array.py --config-dir configs/mf/
python scripts/run_local_array.py --config-dir configs/ff/
python scripts/run_local_array.py --config-dir configs/cafo/
# Ablation ladder controls
python scripts/run_local_array.py --config-dir configs/bp_ds/
python scripts/run_local_array.py --config-dir configs/mf_joint/
# Matched data-pass diagnostics
python scripts/run_local_array.py --config-dir configs/diagnostics/
# Build the tidy measurements and analyze the ladder
python scripts/build_tidy_table.py --out results/tidy
python scripts/analyze_ablation_ladder.py --results-dir results/runs --json results/ablation_ladder.jsonThe exact Athena GPU-hour budget is TODO: owner. The configs/reproduction/ files are the
published MF reproduction checks; this checkout does not retain the historical tuning driver or
its Optuna environment, so the tuned baseline values are consumed as config data.
.
|-- configs/
| |-- base.yaml # shared defaults: device, data root, logging, monitoring, tuning
| |-- bp_baselines/ # tuned BP baselines matching the FF, MF and CaFo architectures
| |-- bp_ds/, mf_joint/ # ablation ladder controls
| |-- diagnostics/ # matched data-pass (equal-epoch) budgets
| |-- ff/, cafo/, mf/ # final experiment configs per algorithm
| |-- reproduction/ # reproductions of the published MF setup
|-- analysis/ # frozen outputs of the G1/W1/G2 analyses, with SHA-256 manifests
|-- prereg/ # pre-registrations and amendment logs (G1, W1, G2)
|-- scripts/
| |-- run_experiment.py # single train-and-test entry point
| |-- run_local_array.py # local sequential batch runner
| |-- build_tidy_table.py # run records -> generated tidy table
| |-- analyze_ablation_ladder.py # pre-specified ladder statistics
| |-- check_mf_cache_equivalence.py # MF activation-cache bit-exactness check
| |-- run_optuna_search.py # hyper-parameter search driver
| |-- jpdc/ # analysis and audit scripts for the follow-up studies
| |-- slurm_scripts/ # SLURM launchers used on Athena
| `-- __init__.py # package marker for analysis imports
`-- src/
|-- algorithms/ # FF, CaFo and MF training and evaluation loops
|-- architectures/ # FF_MLP, MF_MLP and CaFo_CNN modules
|-- baselines/ # standard BP training baseline
|-- data_utils/ # torchvision datasets, splits, transforms
|-- plotting/ # tidy-table construction and plotting helpers
|-- training/ # experiment orchestration engine
|-- tuning/ # Optuna objectives per algorithm
`-- utils/ # config parsing, logging, NVML monitoring, profiling
|-- tests/ # unit and invariant tests
`-- CITATION.cff # PPAM 2026 citation metadata
data/, results/, checkpoints/ and wandb/ are created at run time and are not version
controlled.
Environment used for every run in the paper:
| Component | Version |
|---|---|
| Cluster | Athena, ACK Cyfronet AGH (Rocky Linux 9.5) |
| CPU | AMD EPYC 7742 |
| GPU | NVIDIA A100-SXM4-40GB |
| NVIDIA driver | 595.71.05 |
| CUDA toolkit | 12.4.0 |
| Python | 3.10.4 |
| PyTorch | 2.4.0+cu121 |
| Torchvision | 0.19.0 |
| pynvml | 12.0.0 |
Linux with an NVIDIA GPU (virtualenv)
git clone https://github.com/Przemyslaw11/BeyondBackpropagation.git
cd BeyondBackpropagation
python3.10 -m venv venv
source venv/bin/activate
python -m pip install --upgrade pip
pip install -r requirements.txt
python -c "import torch; print(torch.__version__, torch.cuda.is_available())"The same works with conda (conda create -n beyond-bp python=3.10). Energy and power measurement
needs NVML, so hardware numbers comparable with the paper require an NVIDIA GPU.
macOS on Apple Silicon (functional runs only)
python3 -m venv venv && source venv/bin/activate
pip install --upgrade pip
pip install torch torchvision --index-url https://download.pytorch.org/whl/cpu
pip install -r requirements.txtWith general.backend: local and general.device: auto, MPS is used when available and the CPU
otherwise. NVML is absent on macOS, so no energy or power is recorded.
SLURM (Athena)
module purge
module load Python/3.10.4 CUDA/12.4.0
python3 -m venv venv && source venv/bin/activate
pip install -r requirements.txtCommon installation issues
torch.cuda.is_available()isFalseon a login node: test inside a GPU allocation.- PyTorch installed a CPU wheel: reinstall the CUDA wheel that matches your cluster.
pynvmlreports missing drivers: run on an NVIDIA GPU node with NVML available.- Slow installs on the cluster: download wheels elsewhere and run
pip install --no-index --find-links=./wheels -r requirements.txt. - No network on compute nodes for Weights & Biases:
export WANDB_MODE=offlineand sync later.
Datasets are downloaded by torchvision on first use (data.download: true) into data.root
(default ./data).
| Dataset | Config name | Split |
|---|---|---|
| MNIST | MNIST |
fixed 50k train / 10k validation, official 10k test |
| Fashion-MNIST | FashionMNIST |
val_split: 0.1 of the training set, official test |
| CIFAR-10 | CIFAR10 |
val_split: 0.1 of the training set, official test |
| CIFAR-100 | CIFAR100 |
val_split: 0.1 of the training set, official test |
CIFAR training uses random horizontal flips and 4-pixel-padded random crops, applied identically
to every algorithm and its baseline; MNIST and Fashion-MNIST are not augmented. For the MLP
experiments, CIFAR inputs are flattened, following the Mono-Forward protocol. To use
pre-downloaded data, set data.root: /path/to/datasets and data.download: false.
Train and test one Mono-Forward model and its BP baseline:
python scripts/run_experiment.py --config configs/mf/cifar10_mlp_3x2000.yaml
python scripts/run_experiment.py --config configs/bp_baselines/cifar10_mlp_3x2000_bp.yamlOther entry points:
# local execution (MPS or CPU)
python scripts/run_experiment.py --config configs/mf/mnist_mlp_2x1000.yaml --backend local
# Forward-Forward and CaFo
python scripts/run_experiment.py --config configs/ff/fashion_mnist_mlp_4x2000.yaml
python scripts/run_experiment.py --config configs/cafo/cafodfa_cifar10_cnn_3block.yaml
# a directory of configs, locally
python scripts/run_local_array.py --config-dir configs/mf/Each run logs, among others, these final metrics:
| Metric | Meaning |
|---|---|
final/training_duration_sec |
wall-clock training time |
final/total_gpu_energy_wh |
trapezoidal integral of the 5 Hz NVML power trace |
final/peak_torch_alloc_mib |
per-process allocator peak (the memory figure used in the paper) |
final/peak_gpu_mem_used_mib |
device-wide NVML memory, including the CUDA context |
final/epochs_completed |
epochs actually trained under early stopping |
Configs are plain YAML merged with configs/base.yaml by
src/utils/config_parser.py.
| Field | Meaning |
|---|---|
experiment_name |
Run name used for logs, W&B, results and checkpoints |
general.backend |
"slurm" (default cluster profile: workers and pinned memory) or "local"; this is not a job-submission mechanism and is overridable with --backend |
algorithm.name |
BP, FF, CaFo or MF |
data.name, data.root, data.val_split |
Dataset, directory and validation fraction |
data_loader.batch_size |
Training batch size (128 for every MLP in the paper) |
model.name |
FF_MLP, MF_MLP or CaFo_CNN |
model.params.hidden_dims |
MLP hidden widths |
model.params.block_channels |
CaFo CNN block channels |
optimizer.lr, optimizer.weight_decay |
BP optimiser settings |
algorithm_params.lr, algorithm_params.epochs_per_layer |
MF layer-local optimiser and per-stage budget |
algorithm_params.ff_learning_rate, algorithm_params.downstream_learning_rate |
FF goodness and classifier learning rates |
algorithm_params.predictor_lr, algorithm_params.num_epochs_per_block |
CaFo predictor learning rate and budget |
monitoring.energy_enabled |
NVML energy and power sampling |
profiling.enabled |
Forward-pass GFLOP profiling |
New datasets go in src/data_utils/datasets.py and src/data_utils/preprocessing.py; new models
in src/architectures/ and src/training/engine.py.
Checkpoints are not distributed. A config writes them when checkpointing.checkpoint_dir is set,
for example checkpoints/mf_cifar10_mlp_3x2000/, which then holds one file per trained MF stage
(mf_matrix_M0_complete.pth, mf_layer_1_complete.pth, …). checkpoints/ is not version
controlled.
- Architectures. MF is evaluated on MLPs only; the headline savings are measured on the 3×2000 MLPs, and at 1.8 M weights the memory advantage has not yet appeared. No convolutional MF was run.
- Accuracy context. Flattening CIFAR for the MLP protocol caps accuracy far below the 85.6% our own 3-block CNN reaches under BP, so the comparisons are internally valid and say nothing about a competitive vision pipeline.
- Hardware. One accelerator and one driver. Board power at these low utilisations is dominated by static and clock-domain components of a large A100 die, so a smaller or better-saturated device would likely narrow the power gap; the memory saving and the pass multiplier should carry across.
- Convergence. MF's end-to-end cost is set by its 3.5× to 4.5× pass multiplier; per-stage schedules and stopping criteria are the natural next target.
If you use this code or data, please cite the PPAM 2026 paper:
@inproceedings{spyra2026energy,
title = {Energy-Efficient Deep Learning Without Backpropagation: A Rigorous
Hardware-Validated Benchmarking Study of Forward-Only Algorithms},
author = {Spyra, Przemys{\l}aw and Dzwinel, Witold},
booktitle = {Parallel Processing and Applied Mathematics (PPAM 2026)},
series = {Lecture Notes in Computer Science},
publisher = {Springer},
year = {2026},
note = {To appear}
}The earlier preprint:
@misc{spyra2025energyefficientdeeplearningbackpropagation,
title = {Energy-Efficient Deep Learning Without Backpropagation: A Rigorous Evaluation of Forward-Only Algorithms},
author = {Spyra, Przemys{\l}aw and Dzwinel, Witold},
year = {2025},
eprint = {2511.01061},
archivePrefix = {arXiv},
primaryClass = {cs.LG},
url = {https://arxiv.org/abs/2511.01061}
}Released under the MIT License.
This work was supported by the National Science Centre (NCN), Poland, under grant OPUS-29, DEC-2025/57/B/ST6/04377, and by the Ministry of Science and Higher Education of Poland. We thank the PLGrid Infrastructure and ACK Cyfronet AGH for computational resources on the Athena cluster under grant PLG/2025/018341, and AGH University of Krakow for institutional support.
The algorithms studied are due to Hinton (Forward-Forward), Zhao et al. (Cascaded Forward) and Gong, Li and Abdulla (Mono-Forward).