LLM inference visualizer
Loading…
How to read the colors
Transform colors classify each operation by what its operands are. Node colors show how long data lives: static weights, ephemeral activations, or state carried to the next token. Sub-blocks show logical dependencies; a fixed matrix product and its row sums are normally one fused kernel.
Saved for the next token
Where the work goes
Operation frequency and data movement for one decode increment.
How much can run in parallel?
Count the operation stages running at once. A whole tensor-product, exp or reduction sub-block is one operation. The four branches after normalized embeddings therefore start at parallelism 4; their internal steps then progress in dependency order.
Discrete operation timestep → · Hover/tap the plot or use the timestep slider for exact counts. Zoom to a block to inspect its branches.
Operation totals by type & scheduling assumptions
Each transform sub-block in Token pipeline counts once, using the same recipes and dependency edges as that diagram. Products, row collection, exponentials, divisions and other drawn stages each take one timestep. Matrix rows, tensor coordinates, heads and the ten routed expert replicas do not multiply the count. A matrix recipe retains its product and row-collection stages as drawn; reductions are not expanded into scalar trees.
Branches start when their input transforms finish, and joins wait for all producers. All blocks are expanded in their three-DeltaNet / one-attention order. The plot uses logical stage time; physical routing, memory latency and fanout hardware do not add timesteps. Data/state/weight bubbles are inputs or outputs, not operations. Explicit routing and save sub-blocks already drawn inside transforms are included.
The paired lookup supplies pre-normalized embeddings. Tensor dimensions and cached-history length change the work inside a stage, not its count. Softmax output ends at probabilities; greedy output ends at argmax. Total counts the sub-blocks executed across all layers; average parallelism is total divided by timesteps. The separate arithmetic ledger below retains scalar/MAC counts.
Operation families, most frequent first
One MAC is one multiply–accumulate pair (≈2 FLOPs). Scalar operations below exclude arithmetic already counted as MACs. Counts describe mathematical work, not instruction counts or equal-cost units.
What feeds what?
Node numbers are operations/token. Arrow numbers are newly produced values/token. Dashed arrows include retained outputs from earlier invocations. Select a node to follow its inputs and outputs.
Proposed layout types & why they differ
How this informs distance, and what these counts mean
Ranked weights count the payload at named mathematical stage boundaries, summed across blocks, heads and active experts. A length-256 dot product contributes 256 MACs to the arithmetic ledger but only one result to its next stage. Reduction inputs count as a vector; a reduced scalar sent onward counts once before broadcast. A branch contributes one payload to each distinct consumer. Two inputs from the same producer to the same stage can be locally reused. No internal MAC steps or reduction-tree edges are counted.
Start placement with Σ (handoff values × distance), then replace logical values with actual routed words, precision, fanout and reuse for each hardware mapping. Constrain the result by stage throughput, port capacity and feedback latency. Same-type arrows describe successive stages and may stay inside one assembly. Fusion can remove an external transfer. There is no single global “next instruction”: independent branches can run together. Execution stages keeps repeated uses separate within one chain; operation-type grouping merges them and can create apparent cycles. The default stage view orders current-token dependencies downward; dashed prior-token feedback may point upward. In the grouped views, node positions are computed from the selected chain’s handoff counts: weighted springs and repulsion establish the layout, then position exchanges reduce Σ (handoff count × center-to-center distance) while preserving label clearance. Opposite directions add for attraction; self-loops do not move nodes. Changing model or context recomputes the layout; inspecting a node or changing the token rate leaves it stable. Edge widths use a compressed power scale; the final distance refinement uses the uncompressed counts. This is a constrained graph-layout heuristic, not a proven optimum or physical equipment placement.
This graph covers the named decode arithmetic chains, including norms, gates, attention, recurrent update/readout and active FFN work. It excludes data-source nodes, memory reads/writes, cross-block scheduling, token lookup/address work, cache writes, top-k internals, sampling internals, tiling and restoration. Unconnected external inputs still exist. It is a first placement input, not a complete physical traffic model or an optimized floorplan. Select a node type to inspect incoming and outgoing handoffs; expand a row for its contributing stages.
Operation breakdown
Data movement & required bandwidth
Single sequence, batch 1; decimal bytes. An ideal streaming pass reads each active weight and each shared K/V element once. Actual HBM traffic depends on cache residency, tiling and reuse; on-chip coefficient reuse is not HBM traffic.