Research·July 2026

Tachyon: Parallelizing Discrete Spiking Neural Networks via Associative Operators & Bounded Discretization

Introduction

Spiking Neural Networks (SNNs) represent a promising frontier for high-efficiency, event-driven artificial intelligence. However, training deep spiking architectures on modern accelerator clusters has long been constrained by two fundamental computational walls: the $O (T)$ temporal unrolling bottleneck in discrete time steps and the $O (N_{o u t} N_{in})$ activation memory explosion in continuous-time event processing.

In this work, we present a unified mathematical framework and hardware-optimized runtime that resolves both temporal latency and spatial memory challenges without modifying underlying neuron equations or sacrificing exact firing dynamics. We introduce ATLAS (Associative Temporal Lattice for Asynchronous Spikes) and Railfire:

$O (lo g T)$ Parallel Discrete Backpropagation: By proving that discrete surrogate-gradient sweeps form associative affine sequence operators, we compress reverse time-stepping to logarithmic parallel depth, delivering a $37.4 \times$ backward speedup and an $80.7 \times$ end-to-end step speedup over standard autograd unrolling.

$110 \times$ Faster Low-Memory Rematerialization: Our hierarchical block-scan matches the minimal memory footprint of activation checkpointing ( $O (T / K)$ ) while executing $\approx 110 \times$ faster via fused, SRAM-resident wave rematerialization.

Deep Layer Scaling via ATLAS: Integrated into our continuous event engine, ATLAS maps continuous firing timestamps onto $M$ fixed, differentiable lattice slots. This reduces continuous activation memory from $O (N_{o u t} N_{in})$ down to $O (N_{o u t} M)$ , allowing continuous-time SNNs to scale beyond 20+ layers on modern hardware.

Certified Speculative Forward Pass: For exact hard-reset forward pass, our channel-independent draft-verify-commit mechanism achieves $3.60 - 14.51 \times$ speedups across sparse sequence regimes.

This framework fundamentally shifts the computational economics of spiking neural networks. By enabling the training of 20+ layer deep spiking architectures, we effectively open the door to running high-efficiency, 'brain-like' AI on standard data center hardware, overcoming the scalability barriers that have previously limited SNNs to shallow implementations.

1. Breaking the Temporal Unrolling Wall

Standard deep sequence architectures, such as Transformers or linear State Space Models (SSMs like Mamba), exploit sequence-wide parallelism during training. In contrast, discrete SNNs have remained trapped in step-by-step causal unrolling due to the discontinuous hard-reset mechanism:

m_{t} = u_{t} - s_{t} θ_{t}

Because a neuron's membrane potential at time $t$ strictly depends on whether a spike occurred at step $t - 1$ , standard Backpropagation Through Time (BPTT) processes sequences sequentially, leading to poor GPU Tensor Core utilization.

Standard discrete SNN (Sequential O(T) Unrolling)

Step 0m₀,s₀

Step 1m₁,s₁

Step 2m₂,s₂

Step Tm_T,s_T

Serial Recurrence

Parallel Associative SNN (Miruvor O(log T) Scan)

Input Tensor [t = 0, 1, 2, ..., T]

Parallel Affine Scan Tree

[Sub-Block 0-K]Logarithmic Parallel Depth

[Sub-Block K-2K]

Evaluated States [m₀, m₁, m₂, ..., m_T]

Standard discrete SNN sequential O(T) unrolling vs Miruvor O(log T) parallel associative scan

Furthermore, continuous-time event models (e.g., EventProp) require maintaining dynamic temporal graphs across presynaptic and postsynaptic pairs, scaling memory quadratically with layer width ( $O (N_{o u t} N_{in})$ ) and triggering Out-Of-Memory (OOM) faults on long sequences.

Prior parallel SNNs either change the neuron (PSN), approximate reset (PSU/fixed-point), or specialize continuous TTFS/events (Sakemi, Bullet Trains).

Miruvor keeps exact discrete graded-ALIF forwards and continuous event semantics, and parallelizes only where the operator is associative: reverse surrogate adjoints, hierarchical boundary rematerialization, certified speculative prefixes, and packed-event scans, accepting surrogate conventions, regime-dependent speculation, and fixed-topology event VJPs as explicit tradeoffs.

2. Architecture

Our architecture bridges State Space Model parallel operator theory with bounded temporal discretization.

Existing Approaches to Parallelizing SNN Training

Approach	Parallelism	Memory	Best for	Tradeoff
Serial BPTT (snnTorch / SpikingJelly)	Spatial only	$O (T)$	Short $T$	Simple, but collapses on long sequences
Activation checkpointing	Still serial in time	$O (T / K)$	Memory-bound training	Low memory, rematerialization is extremely slow
Kernel fusion	Channel-parallel	Lower traffic, still $O (T)$ traj	Medium $T$ , wide layers	Faster launches; no $O (lo g T)$ temporal depth
PSN (no-reset)	Full time-parallel	Good	Parallel accuracy work	Fast, but different neuron (reset removed)
PSU (estimated reset)	Time-parallel	Moderate	Binary, fixed $θ$	Parallel, but reset is approximate
Fixed-point / SPC	Conditional	$O (T)$	High sparsity	Can diverge; needs iteration / block knobs
Bullet Trains	Event-chunk parallel	Event-scaled	Continuous-time precision	Strong on events; hard reset, fixed $θ$ , not graded discrete ALIF
EventProp / adjoint ODE	Limited	Often heavy	Small exact-timing nets	Correct grads; historically hard to scale deep
DSTD / TTFS deep nets	Prefix over $M$ slots	$O (N_{out} M)$	Deep TTFS CNNs	Deep continuous training; TTFS / ≤1 spike regime
Miruvor (unified)	$O (lo g T)$ reverse scan · block remat · speculative prefixes · event scans	$O (T)$ or $O (T / K)$ · event-bounded	Discrete and continuous, long $T$	Keeps exact hard forwards; surrogate bwd convention explicit

Most methods buy speed by changing the neuron, approximating reset, or staying stuck in one regime (discrete or continuous). Miruvor keeps exact charge-fire-reset dynamics and attacks the real bottlenecks:

Training: associative reverse adjoint: $37.4 \times$ bwd, $80.7 \times$ end-to-end vs serial/autograd reference
Long $T$ : hierarchical block remat: checkpoint-level memory, $\sim 110 \times$ faster than naive checkpointing
Sparse forward pass: certified speculative prefixes: $3.6 - 14.5 \times$ in sparse/narrow regimes
Continuous events: packed event engine: $1.72 \times$ faster matched SHD step than a literal Bullet Trains parallel recreation^* (kernel microbenchmarks $41 - 187 \times$ )

^*Official Bullet Trains JAX was not publicly available at evaluation time; comparison uses a faithful recreation of the published algorithm on CUDA.

Bottom line: others parallelize by simplifying the model or specializing one setting. Miruvor parallelizes the operators that are actually associative, without giving up exact forwards across discrete and continuous stacks.

Miruvor Unified Spiking Engine

Discrete Sequence Layer

(Spiking Parallel Scan)

Forward: Certified Speculative Prefixes
Backward: O(log T) Reverse Associative Scan
Memory: O(T/K) Boundaries

Continuous Event Layer

(ATLAS Memory Bounded)

Forward: O(log K) Inter-Event Affine Scans
Backward: Dual-ODE IFT / Saltation
Memory: Bounded O(N_out M) via ATLAS
Stability: Railfire Temporal Regularization

Miruvor Unified Spiking Engine: discrete sequence layer and continuous event layer

2.1 Discrete Associative Reverse Adjoints

Under surrogate-gradient conventions, the backward gradient pass through time simplifies to an element-wise affine recurrence:

g_{t} = ℓ_{t} + a \cdot g_{t + 1}

Where $g_{t} = \partial L / \partial u_{t}$ represents the membrane adjoint and $a \in (0, 1)$ is the voltage decay rate.

Mathematical Operator Monoid:

Defining each step as an affine operator $T_{t} (λ) = aλ + ℓ_{t}$ , two successive transformations compose according to:

(A_{1}, c_{1}) \circ (A_{2}, c_{2}) = (A_{2} A_{1}, A_{2} c_{1} + c_{2})

Because affine composition is associative, the entire reverse temporal sweep evaluates as a Blelloch-style parallel prefix scan in $O (lo g T)$ parallel depth.

Time Step: T-3

Time Step: T-2

Time Step: T-1

Time Step: T

Local Maps(a, ℓ_T-3)

Local Maps(a, ℓ_T-2)

Local Maps(a, ℓ_T-1)

Local Maps(a, ℓ_T)

Pair Composite(a², aℓ_T-3 + ℓ_T-2)

Pair Composite(a², aℓ_T-1 + ℓ_T)

Global Composite(a⁴, Combined Adjoint) → Written to HBM

Reverse associative scan flow (SRAM-resident tiling)

In higher-order surrogate modes where adaptation and reset jumps retain non-zero Jacobians, state transformations expand into $3 \times 3$ matrix-affine operators, evaluated across the same parallel scan tree.

3. Scalable Memory via Hierarchical Block-Scanning

While full parallel scans maximize GPU throughput, storing all intermediate activations ( $u_{t}, θ_{t}, s_{t}$ ) over $T \geq 8192$ steps consumes significant High-Bandwidth Memory (HBM). Standard autograd activation checkpointing solves memory pressure but introduces heavy graph recomputation overhead.

Our Hierarchical Block-Scan resolves this trade-off by combining inter-block associative scans with SRAM-resident local rematerialization:

Forward Pass: The exact discrete forward pass executes, saving boundary state checkpoints $(m, b, s)$ every $K$ steps into HBM.

Inter-Block Summary: Each $K$ -step temporal block is compressed into an affine block summary:

λ_{start} = k = 0 \sum K - 1 a^{k + 1} ℓ_{k} + a^{K} λ_{end}

Reverse Wave Rematerialization: Boundary adjoints are evaluated globally using an inter-block reverse scan. High-speed GPU threadblocks load sparse boundaries into ultra-fast SRAM, regenerate local activation trajectories in place, and emit parameter gradients directly.

Selected

Hierarchical Block

Peak HBM: 88.8 MiB
Step time: 48.96 ms

Click a strategy to compare memory and latency at T = 8192.

Memory footprint comparison (sequence horizon T = 8192)

4. Continuous-Time Scaling via ATLAS

In continuous-time event models, tracking exact inter-spike intervals requires maintaining state matrices scaled to presynaptic and postsynaptic connectivity ( $O (N_{o u t} N_{in})$ ). To scale continuous SNNs to deep architectures, ATLAS (Associative Temporal Lattice for Asynchronous Spikes) provides the memory-bounded substrate.

Continuous Spikes

t*₁

t*₂

t*₃

Time (t)

ATLAS Lattice Projection

τ₁

τ₂

τ₃

τ₄

τ₅

τ₆

τ₇

τ₈

Memory Complexity: Bounded to O(N_out · M) instead of O(N_out · N_in)

ATLAS: continuous event mapping on an associative temporal lattice; memory bounded to O(N_out · M)

ATLAS projects asynchronous firing timestamps $t^{⋆} \in R$ onto $M$ fixed, differentiable lattice slots using soft assignment weights $α_{m} (t^{⋆})$ , chaining continuous timing sensitivities through a fixed associative temporal lattice. This decouples memory consumption from network connectivity, capping continuous activation memory at $O (N_{o u t} M)$ and enabling continuous architectures to scale up to 20+ layers.

5. Certified Speculative Forward Pass

Because hard resets destroy forward associativity in exact discrete execution, we introduce a speculative forward pass model:

Drafting: An associative scan calculates a free-response membrane trajectory $u^_{t}$ assuming no resets.

Verification: Candidate spikes $s^_{t}$ are evaluated alongside conditional reset corrections.

Certification: The first mismatch index $t^{⋆}$ is located independently per channel. The certified prefix $[1, t^{⋆}]$ is committed, and execution restarts from the certified state.

1. Draft Phase

[ û₁ → û₂ → û₃ → û₄ → û₅ ] (Free-Response Scan)

2. Verify Phase

[ ŝ₁ ŝ₂ ŝ₃* ŝ₄ ŝ₅ ] (* Mismatch Detected at Step 3)

3. Commit Phase

[ COMMIT COMMIT ] → [ RESTART at Step 3 ]

Speculative draft-verify-commit cycle

Because verification operates independently across feature channels, a single lagging neuron cannot stall the entire batch, yielding high commit efficiency in sparse processing regimes.

6. Comprehensive Empirical Evaluation

All benchmarks were evaluated on NVIDIA A100-SXM4-80GB GPUs.

6.1 Discrete Adjoint & End-to-End Training Throughput

Workload configuration: $[T, B, D, N] = [1024, 4, 64, 2]$ .

Backend Path	Execution Time	Speedup Factor	Numerical Parity Error
Serial Reverse Adjoint	27.08 ms	1.0×	Baseline Reference
Associative Reverse Scan	0.724 ms	37.4×	≤ 1.49 × 10⁻⁸
Reference Training Step	588.50 ms	1.0×	Baseline Reference
Scan Training Step (Ours)	7.29 ms	80.7×	≤ 1.49 × 10⁻⁸
Higher-Order Matrix Scan	50.40 ms	1.72×	≤ 2.98 × 10⁻⁸

6.2 Memory-Speed Pareto at Long Sequence Horizons ( $T = 8192$ )

Shape configuration: $[8192, 4, 64, 2]$ , block size $K = 64$ .

Execution Strategy	Step Time	Peak HBM Memory	Performance Notes
Full Scan	24.50 ms	248.0 MiB	Stores full trajectory tensors
Autograd Checkpointing	5399.00 ms	88.8 MiB	Slow autograd recomputation
Hierarchical Block-Scan (Ours)	48.96 ms	88.8 MiB	≈ 110× faster than checkpointing

6.3 Continuous Event Engine Microbenchmarks

Performance across varying node dimensions:

Workload Regime	Baseline Event Time	Our Continuous Engine Time	Speedup Factor	Spike Timing Error
Sparse Event Topology (N=32)	289.90 ms	1.55 ms	187×	< 7.0 × 10⁻⁷ s
Medium Event Density (N=128)	289.90 ms	2.71 ms	107×	< 2.4 × 10⁻⁵ s
Dense Network Scale (N=512)	289.90 ms	6.97 ms	41.6×	< 1.1 × 10⁻⁴ s
End-to-End Event Step	289.90 ms	169.10 ms	1.72×	Corrected Saltation VJP

6.4 Speculative Surface Performance

Across 32 tested sparse sequence configurations ( $T \in {512, 2048}$ ), speculative forward pass achieved $32/32$ performance wins, delivering $3.60 - 14.51 \times$ speedups over fused register-serial baselines with commit efficiencies $\geq 0.90$ .

7. Architectural Impact & Future Horizons

By combining $O (lo g T)$ parallel discrete associative scans, $O (T / K)$ hierarchical rematerialization, and $O (N_{o u t} M)$ ATLAS memory bounds, this framework bridges State Space Model parallel speed with neuromorphic event efficiency.

This unlocks the ability to train deep (20+ layer), long-horizon spiking architectures directly on modern parallel hardware, establishing a foundation for scalable neuromorphic foundational models.

Crucially, this compute efficiency serves as the missing link for the Physical AI stack, bridging the gap between high-performance hardware and neuromorphic architectures. By significantly lowering the compute barrier for neuromorphic approaches, this work aligns with the broader strategic thesis that efficient, event-driven compute is essential for intelligence in the physical world.