LatentSync: Brain-to-Text via Neural Latent Space Alignment

Brain–computer interfaces (BCIs) have advanced toward decoding neural activity into text, yet most systems remain constrained by closed vocabularies and supervised training requirements. We propose LatentSync Transformers, a novel framework that achieves zero-shot open-vocabulary brain-to-text translation by aligning neural latent spaces with pretrained transformer language models. Using multimodal EEG–fMRI data, LatentSync synchronizes latent neural embeddings with transformer hidden states through contrastive alignment, enabling robust decoding without retraining on task-specific corpora. Experimental results demonstrate superior generalization, semantic fidelity, and syntactic fluency compared to RNN, LSTM, and CNN baselines. This work establishes latent alignment as a scalable pathway for naturalistic brain-to-text communication, with implications for neuroprosthetics, cognitive monitoring, and assistive technologies.