Wancong Zhang

Wancong (Kevin) Zhang

I am a PhD student at NYU Courant, advised by Yann LeCun. My research focuses on world models, planning, and representation learning, with the goal of building agents that can learn from observation and interact with the world in an open-ended way.

I am currently especially interested in hierarchical world models and planning across multiple temporal scales, where structure in the model can enable better long-horizon reasoning, more efficient control, and stronger generalization from offline or reward-free data. More broadly, I am interested in self-supervised learning for decision making, computer vision, and embodied intelligence.

Email / CV / Google Scholar / Github

News

Oct 2026 — Released H-JEPA: End-to-End Learning of Hierarchical World Models for Visual Planning.
Sep 2026 — Paper accepted to CoRL 2026 - Hierarchical Planning with Latent World Models.
May 2026 — Started Research Internship at AMI Labs.
Apr 2026 — Released our new paper on arXiv Hierarchical Planning with Latent World Models.
Sep 2025 — Paper accepted to NeurIPS 2025 - Planning with Latent Dynamics Models.
Jul 2025 — Our paper on Planning with Latent Dynamics Models received the Best Paper Award at ICML 2025 Workshop on Physically Plausible World Models.
Jun 2025 — Started Research Scientist Internship at Meta AI (FAIR) under the mentorship of Nicolas Ballas.
Sep 2023 — Serving as a Teaching Assistant for Deep Learning (DS-GA 1008)
Sep 2023 — Began PhD at NYU Courant under the supervision of Yann LeCun.

Research (Updated 10/2026)

	H-JEPA: End-to-End Learning of Hierarchical World Models for Visual Planning Wancong Zhang, Basile Terver, Michael Rabbat, Yann LeCun†, Randall Balestriero† Paper / Website / Code arXiv preprint, 2026 Learns a hierarchy of JEPA world models end-to-end, with each level predicting farther ahead in its own latent space. Planning from coarse to fine improves long-horizon control while reducing planning compute, raising Visual AntMaze success from 18% to 73%.
	Hierarchical Planning with Latent World Models Wancong Zhang, Basile Terver, Artem Zholus, Soham Chitnis, Harsh Sutaria, Mido Assran, Amir Bar, Randall Balestriero, Adrien Bardes, Yann LeCun†, Nicolas Ballas† Website / Code Conference on Robot Learning (CoRL), 2026 Introduces hierarchical planning across latent world models at multiple temporal scales, improving long-horizon zero-shot control while reducing planning-time compute on robotics and simulated control tasks.
	Learning from Reward-Free Offline Data: A Case for Planning with Latent Dynamics Models Wancong Zhang, Vlad Sobal, Kynghyun Cho, Randall Balestriero, Tim Rudner, Yann LeCun Website Advances in Neural Information Processing Systems (NeurIPS), 2025 Best Paper Award, ICML 2025 Workshop on Building Physically Plausible World Models This work introduces zero-shot planning using the Joint Embedding Prediction Architecture (JEPA) trained from offline trajectories, demonstrating its strengths in generalization, trajectory stitching, and data efficiency compared to traditional offline RL.
	SkyJEPA: Learning Long-Horizon World Models for Zero-Shot Sim-to-Real Control of Quadrotors Pratyaksh Rao, Wancong Zhang, Randall Balestriero, Yann LeCun, Giuseppe Loianno arXiv Under review at IEEE Transactions on Robotics (T-RO), 2026 Introduces a JEPA-style latent dynamics model with a physics-inspired prober for real-time quadrotor control, enabling accurate long-horizon prediction and robust zero-shot sim-to-real transfer from automatically generated simulation data.
	Light-weight probing of unsupervised representations for Reinforcement Learning Wancong Zhang, Anthony GX-Chen, Vlad Sobal, Yann LeCun, Nicolas Carion Code Reinforcement Learning Conference (RLC), 2024 Presents an efficient probing benchmark to evaluate the fitness of unsupervised visual representations for reinforcement learning (RL). Applied it to systematically improve pre-existing SSL recipes for RL.
	Conformer-1: Robust ASR via Large-Scale Semisupervised Bootstrapping Wancong Zhang, Luka Chkhetiani, Francis McCann Ramirez, Yash Khare, Andrea Vanzo, Michael Liang, Sergio Ramirez Martin, Gabriel Oexle, Ruben Rousbib, Taufiquzzaman Peyash, Michael Nguyen, Dillon Pulliam, Domenic Donato 2024 Showcases an industrial-scale end-to-end Automatic Speech Recognition model trained on 570k hours of speech audio data using Noisy Student. It achieves competitive word error rates against larger and more computationally expensive models.
	MixUp Training Leads to Reduced Overfitting and Improved Calibration for the Transformer Architecture Wancong Zhang, Ieshan Vaidya 2021 Adapts the computer vision data augmentation technique MixUp to the natural language domain, reducing calibration error of transformers for sentence classification by up to 50%.
	A comprehensive EHR timeseries pre-training benchmark Matthew McDermott, Bret Nestor, Evan Kim, Wancong Zhang, Anna Goldenberg, Peter Szolovitz, Marzyeh Ghassemi Conference on Health, Inference, and Learning (CHIL), 2021 Establishes a pre-training benchmark protocol for electronic health record (EHR) data.

Teaching

Deep Learning (DS-GA 1008), Fall 2024

Designed and hosted the final course competition (GitHub)