What Makes an Efficient VLA?

Navigating Action-Head Design,
Scaling, and Latency

Luoyang Sun^1,2,3,4* Guoyang Xia^5,4* Fengfa Li⁴ Lei Ren⁴†‡ Xinyu Cui^1,2 Haifeng Zhang¹‡ Fangxiang Feng⁵ Kaike Zhang⁴ Kun Zhan⁴ Yan Xie⁴ Jun Wang⁶ Cheng Deng⁷‡

¹Institute of Automation, Chinese Academy of Sciences ²University of Chinese Academy of Sciences ³AI Lab, The Yangtze River Delta ⁴Li Auto Inc. ⁵Beijing University of Posts and Telecommunications ⁶University College London ⁷University of Edinburgh

*Equal contribution †Project Leader ‡Corresponding author

arXiv — coming soon Paper (PDF) Code

The module-scale factorial, the eight action-head substrates, and every substrate on the accuracy-latency plane. — **A controlled design-space study of a modular VLA on one fixed backbone.** **(a)** The module-scale factorial: the vision encoder (V), language backbone (L) and action head (A) are held at an anchor and swept independently, crossed with the orthogonal action-head axes. **(b)** The eight action-head *substrates* — all share the same V+L backbone and differ only in the head. **(c)** Every measured substrate on the accuracy–latency plane; EffVLA holds the frontier.

Abstract

Vision-Language-Action (VLA) models combine a pretrained vision encoder, a language backbone, and an action head, but which of these design choices actually drives performance has not been measured under controlled, latency-paired conditions. We fix the backbone (SigLIP2-So400m and Qwen2.5) and the pipeline (DROID, LIBERO, LIBERO-Plus), and sweep the action head along four axes — decoder, loss, initialization, inference passes — and the per-module scale, pairing every configuration with measured on-device latency.

Three findings come out of the experiment. First, what decides the action head is not its decoder, loss, or inference budget but how it is initialized. Copying the last transformer layers of the language backbone into the head is the single largest lever we measure, raises accuracy at no latency cost, and is the only axis that pays at every module scale we test. It also reorders the rest of the design space: flow matching, extra inference passes, and a heavier decoder pay off only while the head is randomly initialized and stop paying once it is not, as if the expressive machinery were compensating for something the initialized head already has. We read that something as representation transfer, and offer it as the account that best organizes the measurements rather than as a demonstrated cause. Second, capacity only pays after alignment: the action head is the highest-return module to scale, but only once it is aligned. Third, those returns saturate at a model size close to the one today's π-series VLAs already use, so growing the model further only adds latency.

The three together specify EffVLA, a compact model that matches the strongest open-source VLAs on standard LIBERO, leads on most LIBERO-Plus perturbation axes at lower latency, and transfers to a real SO-ARM101 arm with the recipe unchanged.

Three findings

What the measurement says

Finding 1 · what to build

Align the head, then keep it simple

What the head is short of is alignment with the VLM, not expressive capacity. Initialization supplies it: +7.1 points at no latency cost. Once aligned, flow matching loses 4.4; a fourth pass and a heavier decoder fall within noise.

Finding 2 · where to scale

Capacity only pays after alignment

Aligned, the action head returns ~4 success points per added millisecond, against ~1 for the vision encoder and ~0.15 for the language backbone. Misaligned, the same scaling buys about a third as much (+3.1 vs +9.4).

Finding 3 · how big

Stop at a modest size

On LIBERO-Plus, scaling any module past the l configuration leaves accuracy flat while latency keeps climbing. The ~3.75B budget — close to the π-series scale — is the efficiency/accuracy knee.

The design space

Eight substrates, one backbone

A substrate is a fixed choice of the four action-head axes, independent of module size. All eight share the same SigLIP2 + Qwen2.5 backbone and read it through the same KV-share interface — they differ only in the action head.

Everything that usually moves between VLA papers is held still: same backbone, same DROID → LIBERO pipeline, same LIBERO-Plus protocol (four task suites × seven zero-shot perturbation axes, ~1,000 rollouts per cell), same hardware for latency (RTX 5090, bf16, batch 1, chunk size 8, torch.compile). Any difference below has to come from the head.

Substrate	Decoder	Loss	Init	Passes	Succ. (%)
FAST	AR trunk	NTP	—	many	57.7
OFT	MLP	L1	—	1	79.0
OFT-TF	TF block	L1	random	1	72.7
OFT-TF-r4	TF block	L1	random	4	74.8
EffVLA	TF block	L1	VLM-init	1	79.8
EffVLA-r4	TF block	L1	VLM-init	4	79.6
π₀	TF block	flow matching	random	4	77.4
π₀-vlminit	TF block	flow matching	VLM-init	4	75.2

LIBERO-Plus 4-suite mean success rate at V = L = A = l. Decoding actions as language tokens (FAST) trails every parallel head by at least 15 points and runs nearly 5× slower. A plain MLP head already lands within 1 point of the best cell. Stacking transformer blocks helps only when they are initialized from the backbone.

The result that reorganized the paper

Every expressive trick works — until you align the head

Take the same four axes and read each one twice: once on a head initialized at random, once on a head initialized from the backbone. The decoder and loss axes reverse sign. Alignment never does.

Axis	Variation	On a misaligned head (random init)	On an aligned head (VLM-init)
Decoder	MLP → transformer block	−6.3	+0.8
Loss	L1 → flow matching	+2.6	−4.4
Inference compute	1 pass → 4 passes	+2.1	−0.2
Alignment	random → VLM-init	+7.1 — positive at every module scale

LIBERO-Plus success-rate change in points, at V = L = A = l. The evaluation standard error is ≈1.4 points, so the compute axis sits inside it under both initializations — extra passes simply stop buying anything once the head is aligned.

Every axis that reverses is a way of adding expressive capacity to the head — a generative loss, a second pass, more decoder machinery — and each earns its keep only while the head is initialized at random. Once initialization supplies what a random head lacks, none of that machinery has anything left to contribute, and it becomes pure latency. What the head is short of is not expressiveness. It is alignment.

The mechanism

Alignment is not a metaphor — it is measurable

If alignment is really the variable, it should be visible in the weights and in the behaviour, not just in the success rates. It is, in both.

In weight space. Measuring linear CKA between the action head and the corresponding backbone layers, the aligned head holds 0.76 with the backbone throughout fine-tuning while a random head never rises above 0.24. The separation holds at every layer of the head, and the success-rate advantage holds across all six vision- and language-backbone sizes. Read per module, the K+V and output projections are preserved most (CKA 0.79 and 0.81) while the query path adapts most (0.54): the head keeps what the backbone reads and re-purposes how it attends.

CKA between action head and backbone through training and across layers, and the success-rate gain across sizes. — **The aligned head stays part of the backbone it was copied from.** **(a)** Through fine-tuning it holds a CKA of 0.76 with the backbone; a random head stays at 0.24. **(b)** The same separation at every layer of the head. **(c)** The success-rate gain holds across three vision-encoder and three language-backbone sizes.

In behaviour. Told to “put the black bowl in the bottom drawer of the cabinet,” the aligned head concentrates on the instruction and on the bowl it names, while the random head spreads attention diffusely across the scene and leaves the instruction almost unattended.

Attention overlays comparing random init and VLM-init on a single rollout. — **A single rollout.** Left: random init. Right: VLM-init. The vision panels overlay the top-K attended patches (agent and wrist views); the ribbon below each shades every instruction token by its attention. Text mass 0.030 vs 0.223.

And it is not one lucky rollout. Across all 128 rollouts the aligned head places roughly four times as much attention on the instruction (0.233 vs 0.055), and the two distributions do not overlap on a single rollout. Ranked by attention, the words it looks at are the object and location nouns that name the task.

Aggregate instruction attention across 128 rollouts and the top-ranked instruction words. — **Aggregated over all rollouts.** **(a)** Instruction attention per rollout, 64 per condition; the two conditions never overlap. **(b)** The top instruction words by VLM-init attention are the object and location nouns that name the task. The aligned head does not merely look at the instruction more — it looks at the part that specifies the goal.

An honest caveat. We manipulate alignment only through initialization, so this evidence explains the measurements rather than proving causation. The decisive control — initializing from the same backbone weights with the alignment destroyed (permuted layers, or a random orthogonal rotation) — would collapse both the CKA and the gain if alignment is what matters. We flag it as future work in the paper rather than claiming the causal link.

Then capacity

Capacity only pays after alignment — and then it stops

Once the head is aligned it becomes the single highest-return place to spend compute. Misaligned, the same parameters buy about a third as much.

Added capacity on an aligned head extends a computation the backbone already shares; on a misaligned head it must first spend itself relearning the interface. And it stops early: a 1B vision encoder scores 78.9, a 7B language backbone 77.8, a 650M head 78.3 — all at or below the 79.8 anchor, while latency only climbs.

Module scaled (s → l)	Δ success	Δ latency	Return (pts / ms)
Action head — aligned	+9.4	+2.4 ms	≈ 4
Action head — misaligned	+3.1	+2.4 ms	≈ 1.3
Vision encoder	+3.4	—	≈ 1
Language backbone	< 2	—	≈ 0.15

Success rate versus module parameter count for the language backbone, vision encoder and action head under three substrates. — **Only the aligned substrate turns action-head capacity into accuracy — and even that saturates.** The vision and language axes scale at similar rates across substrates; the action head separates them. Beyond the l configuration every curve flattens or dips, which is the empirical basis for fixing the model at ~3.75B.

The recipe

EffVLA — what the three rules select

The analysis specifies the model, not the other way round: a 4-layer transformer-block head copied from the last 4 layers of Qwen2.5-3B, trained with single-pass L1 regression, reading the backbone through a KV-share interface on a SigLIP2-So400m encoder.

Of ~3.75B parameters the head adds only ~0.35B, and it runs in 39.2 ms per action chunk. Nothing in the architecture is tuned beyond what the three rules prescribe.

Method	LIBERO					LIBERO-Plus
Method	Spatial	Object	Goal	Long	Avg	BG	Init	Cam	Lang	Noise	Layout	Light
OpenVLA	84.7	88.4	79.2	53.7	76.5	34.8	3.5	0.8	23.0	15.2	28.5	8.1
π₀	98.0	96.8	94.4	88.4	94.4	81.4	6.0	13.8	58.8	79.0	68.9	85.0
π₀-FAST	96.4	96.8	88.6	60.2	85.5	73.2	21.6	65.1	61.0	74.4	68.8	73.2
RIPT-VLA	98.6	98.6	99.0	93.8	97.5	91.6	31.2	55.2	77.6	73.5	74.2	88.4
OpenVLA-OFT	97.6	98.4	97.9	94.5	97.1	93.3	31.9	56.4	79.5	75.8	74.2	88.7
ABot-M0	98.8	99.8	99.0	96.6	98.6	91.6	67.9	60.4	86.4	86.4	82.6	96.2
EffVLA (ours)	98.6	99.2	99.0	96.0	98.2	96.7	69.8	68.3	87.5	66.1	84.8	97.4

Success rate (%), in the two-breakdown layout of the paper: standard LIBERO by its four task suites (left) and LIBERO-Plus by its seven zero-shot perturbation axes (right) — background, robot initial state, camera viewpoint, language instructions, sensor noise, object layout, lighting. Bold: column best. Underlined: second best. Standard LIBERO has compressed the top tier into a 96–99% band and EffVLA sits inside it, 0.4 points behind ABot-M0. LIBERO-Plus separates the same models: EffVLA leads on six of the seven perturbation axes, with its widest margins on exactly the spatial and language axes an aligned head is built to help — camera viewpoint (+7.9 over ABot-M0) and robot initial state (+37.9 over OpenVLA-OFT). It trails only on sensor noise, where the simpler MLP head was already strongest in our own ablation. Every row other than EffVLA is as reported by Yang et al.; EffVLA is the only row we evaluate ourselves, under the same protocol.

It transfers to a real arm

The recipe transfers unchanged to a low-cost SO-ARM101 open-source 6-DOF arm — same head, same loss, no axis of the study re-tuned for the new embodiment. Only the data pipeline changes: Stage A uses the SO-ARM101 community corpus in place of DROID, and Stage B fine-tunes on a small set of teleoperated demonstrations. Across five language-specified sorting tasks it succeeds on 40 of 50 trials.

Instruction	Objects	Success
Pick up the yellow cube and place it in the small compartment.	1	10/10
Pick up the blue marker and place it in the medium compartment.	1	9/10
Pick up the blackboard eraser and place it in the large compartment, then pick up the yellow cube and place it in the small compartment.	2	9/10
Pick up the blue marker and place it in the medium compartment, and pick up the yellow cube and place it in the small compartment.	2	7/10
Pick up the blackboard eraser and place it in the large compartment, pick up the blue marker and place it in the medium compartment, and pick up the yellow cube and place it in the small compartment.	3	5/10
Total	—	40/50

Instructions exactly as issued, ten trials each. An episode counts as a success only if every object named in the instruction ends in the compartment it names, so errors compound across a sequence: success falls from 10/10 on the single-object tasks to 5/10 on the three-object one. We report this as a feasibility and transfer result — we did not run the alternative action heads on hardware, so it does not show that the LIBERO-selected recipe stays preferable in the physical setting.

Does the effect hold outside this grid?

Two checks, neither of which extends the claim: they test whether the one result the paper rests on is specific to the configuration we happened to sweep.

It holds in every matched pair. Ten pairs of cells in the study differ only in how the action head was initialized, holding decoder, parameter count, latency, module scales and inference budget fixed. All ten favour VLM-init, by +5.8 points on average, across eight module-scale configurations, both inference budgets, and two benchmarks.

It reproduces on another embodiment. On SimplerEnv (WidowX), the same contrast is +7.3 points against +7.1 on LIBERO-Plus — and there the random-init head is architecture-matched: the same Qwen decoder layers are deep-copied and then re-drawn, so topology, parameter count and latency are identical and only the weight values differ. That places the gain in the weights, not in the head's architecture or capacity.

Substrate	Stack	Carrot	Spoon	Eggplant	Overall
OFT (MLP)	50.0	37.5	50.0	100.0	59.4
TF block, random init	8.3	58.3	79.2	70.8	54.2
π₀ (flow matching)	43.6	36.6	67.9	93.8	60.5
EffVLA (TF block, VLM-init)	37.5	50.0	75.0	83.3	61.5

SimplerEnv on a WidowX arm; 24 trials per task, 96 per model. The three strongest substrates again fall within a few points of each other and OFT and π₀ swap order between benchmarks, so we read them as tied here too. At 96 trials the standard error on a difference is about 7 points, so this is a replication of the effect size and the ordering, not an independent significance test.

BibTeX

@article{sun2026effvla,
  title   = {What Makes an Efficient VLA? Navigating Action-Head Design, Scaling, and Latency},
  author  = {Sun, Luoyang and Xia, Guoyang and Li, Fengfa and Ren, Lei and
             Cui, Xinyu and Zhang, Haifeng and Feng, Fangxiang and
             Zhang, Kaike and Zhan, Kun and Xie, Yan and Wang, Jun and
             Deng, Cheng},
  year    = {2026}
}