プレプリント / バージョン1

Vision-Language-Action Policies Do Not Catch Brain-Computer Interface Errors

A Replayed-SSVEP Study of Error Propagation and Confidence Gating

##article.authors##

  • he, pinzhen Independent Researcher

DOI:

https://doi.org/10.51094/jxiv.6975

キーワード:

brain-computer interface、 SSVEP、 vision-language-action、 robot manipulation、 error propagation、 confidence calibration

抄録

Non-invasive brain-computer interfaces (BCIs) are a natural front end for vision-language-action (VLA) robot policies: the user selects a task, the policy executes it. Decoding errors are frequent at practical speeds, so it matters what the policy does with a wrong command and whether either side can tell. We replayed real steady-state visual evoked potential (SSVEP) recordings (35 + 70 subjects) as task commands and measured two VLAs (pi0.5, OpenVLA-OFT) on LIBERO-Goal and five LIBERO-Plus perturbation families (2332 rollouts). Both policies executed a wrong but feasible instruction in 98-100% of clean episodes and never fell back to the intended task; their pre-execution uncertainty did not separate wrong from right instructions (AUROC 0.44-0.54). End-to-end success was therefore the decoding accuracy scaled by the policy success rate (r = 0.999). On the BCI side, a pooled temperature-scaled posterior left low-accuracy users over-confident (r = -0.88 between subject accuracy and over-confidence; replicated on a second dataset), and errors recurred on retries of the same target (P(error | previous error) 0.33 vs. 0.15 at 1 s). A gate that adds posterior entropy and an abstain option kept wrong-task execution near 5% at 8.7% less time per success than a confidence threshold, but VLA uncertainty contributed nothing measurable in clean scenes. Replacing real EEG by an accuracy-matched Bernoulli decoder overstated the time cost of safety gating (+9.6% vs. 0%). Two consequence-aware extensions did not beat tuned baselines in tests planned before running. Safety for BCI-commanded VLAs must come from the BCI side.

利益相反に関する開示

The author declares no conflicts of interest. This research received no external funding.

ダウンロード *前日までの集計結果を表示します

ダウンロード実績データは、公開の翌日以降に作成されます。

引用文献

Physical Intelligence, K. Black, N. Brown, J. Darpinian et al., "pi0.5: a vision-language-action model with open-world generalization," 2025, arXiv:2504.16054.

M. J. Kim, C. Finn, and P. Liang, "Fine-tuning vision-language-action models: Optimizing speed and success," 2025, arXiv:2502.19645.

S. Zhang, R. F. J. Dossa, L. Nunziante, M. Di Vincenzo et al., "An augmented reality brain-robot interface for generalist robot arm manipulation," 2026, arXiv:2606.16413.

L. Jin, R. Zhao, H. Zhao, V. C. K. Cheung et al., "Brain-conditioned action policies for neural motor decoding," 2026, arXiv:2609.34561.

Anonymous, "BrainVLA: Towards VLA-driven high-level embodied planning for motor brain-computer interfaces," 2026, iCLR 2027 submission, OpenReview HGjJcGGX3A.

S. Fei, S. Wang, J. Shi, Z. Dai et al., "LIBERO-Plus: In-depth robustness analysis of vision-language-action models," 2025, arXiv:2510.13626.

X. Zhou, Y. Xu, G. Tie, Y. Chen et al., "LIBERO-PRO: Towards robust and fair evaluation of vision-language-action models beyond memorization," 2025, arXiv:2510.03827.

H. Li, S. Tan, T. Ashizawa, B. Yen et al., "Paraphrase consensus is not self-consistency: Instruction reliance and robustness in VLA policies," 2026, arXiv:2610.19438.

R. Zhang, S. Lee, M. Hwang, A. Hiranaka et al., "NOIR: Neural signal operated intelligent robots for everyday activities," in Conference on Robot Learning (CoRL), 2023, arXiv:2311.01454.

T. Kim, Y. Wang, H. Cho, A. Hodges et al., "NOIR 2.0: Neural signal operated intelligent robots for everyday activities," 2025, arXiv:2511.20848.

T. Kim and O. Parker Jones, "Gated uncertainty-aware runtime dual invariants for neural signal-controlled robotics," 2025, arXiv:2511.20570.

T. Sultan, P. Sangani, K. Cool, P. Sikorski et al., "NeuroCommitSSM: Decision-centric shared autonomy for safe assistive manipulation via EEG-EMG-ET commit readiness," 2026, arXiv:2607.15395.

Z. Du, M. Bowman, and X. Zhang, "Capability-aware arbitration for semantic intent-based shared control," 2026, arXiv:2609.25369.

A. F. Salazar-Gomez, J. DelPreto, S. Gil, F. H. Guenther, and D. Rus, "Correcting robot mistakes in real time using EEG signals," in IEEE International Conference on Robotics and Automation (ICRA), 2017.

J. DelPreto, A. F. Salazar-Gomez, S. Gil, R. M. Hasani, F. H. Guenther, and D. Rus, "Plug-and-play supervisory control using muscle and brain signals for real-time gesture and error detection," in Robotics: Science and Systems (RSS), 2018.

S. Konstantinova, M. Altamirano Cabrera, A. Lykov, and D. Tsetserukou, "SocialVLA: A social perception gateway for human-reaction-based failure detection and recovery in VLA manipulation," 2026, arXiv:2610.02360.

J. Park, J. Jeong, and T. Gong, "Premover: Fast vision-language-action control via early execution during instruction delivery," 2026, arXiv:2605.12160.

M. Higger, F. Quivira, M. Akcakaya, M. Moghadamfalahi, H. Nezamfar, M. Cetin, and D. Erdogmus, "Recursive Bayesian coding for BCIs," IEEE Transactions on Neural Systems and Rehabilitation Engineering, vol. 25, no. 6, pp. 704-714, 2017.

S. Ahmadi, P. Desain, and J. Thielen, "A Bayesian dynamic stopping method for evoked response brain-computer interfacing," 2024, arXiv:2406.11081.

B. Dal Seno, M. Matteucci, and L. T. Mainardi, "The utility metric: a novel method to assess the overall performance of discrete brain-computer interfaces," IEEE Transactions on Neural Systems and Rehabilitation Engineering, vol. 18, no. 1, pp. 20-28, 2010.

E. Davis, "Bayesian complete-pooling in cross-subject classification for motor imagery electroencephalogram," 2026, arXiv:2607.22980.

G. Canal, Y. Diaz-Mercado, M. Egerstedt, and C. Rozell, "A low-complexity brain-computer interface for high-complexity robot swarm control," 2022, arXiv:2205.14265.

Y. Wang, X. Chen, X. Gao, and S. Gao, "A benchmark dataset for SSVEP-based brain-computer interfaces," IEEE Transactions on Neural Systems and Rehabilitation Engineering, vol. 25, no. 10, pp. 1746-1752, 2017.

B. Liu, X. Huang, Y. Wang, X. Chen, and X. Gao, "BETA: A large benchmark database toward SSVEP-BCI application," Frontiers in Neuroscience, vol. 14, p. 627, 2020.

X. Chen, Y. Wang, S. Gao, T.-P. Jung, and X. Gao, "Filter bank canonical correlation analysis for implementing a high-speed SSVEP-based brain-computer interface," Journal of Neural Engineering, vol. 12, no. 4, p. 046008, 2015.

C. Guo, G. Pleiss, Y. Sun, and K. Q. Weinberger, "On calibration of modern neural networks," in International Conference on Machine Learning (ICML), 2017.

B. Liu, Y. Zhu, C. Gao, Y. Feng, Q. Liu, Y. Zhu, and P. Stone, "LIBERO: Benchmarking knowledge transfer for lifelong robot learning," in Advances in Neural Information Processing Systems (NeurIPS) Datasets and Benchmarks Track, 2023, arXiv:2306.03310.

ダウンロード

公開済


投稿日時: 2026-10-26 05:46:50 UTC

公開日時: 2026-11-17 06:38:39 UTC
研究分野
情報科学