News
PublicationsRobotic Manipulation
Scaling Manual-Grounded Appliance Manipulation with Data Synthesis and Unified PlanningACM International Conference on Multimedia (ACM MM) 2026
AppliancePlan learns to ground appliance parts, plan long-horizon tasks, and recover from disturbances using UseAppliance, a large-scale dataset synthesized from manuals through hierarchical appliance graphs. 
RealAppliance: Let High-fidelity Appliance Assets Controllable and Workable as Aligned Real ManualsConference on Computer Vision and Pattern Recognition (CVPR) 2026
(Highlight)
RealAppliance provides 100 realistic appliance assets with mechanical, electronic, and program behavior aligned with manuals, alongside a benchmark for grounding and manipulation planning. 
CheckManual: A New Challenge and Benchmark for Manual-based Appliance ManipulationConference on Computer Vision and Pattern Recognition (CVPR) 2025
(Highlight)
CheckManual introduces a benchmark for robots that consult appliance manuals to complete manipulation tasks, with simulated environments, evaluation metrics, and the ManualPlan planning baseline. 
ManipLLM: Embodied Multimodal Large Language Model for Object-Centric Robotic ManipulationConference on Computer Vision and Pattern Recognition (CVPR) 2024
ManipLLM combines object understanding and affordance reasoning to predict manipulation contact points and end-effector poses from RGB images, with closed-loop control and real-world adaptation. Embodied Navigation
One Scan is Enough: Demonstration-Free Adaptation for Language-Guided NavigationConference on Neural Information Processing Systems (NeurIPS) 2026
One Scan is Enough turns a single iPhone scan into a training environment with synthetic instructions, enabling navigation policies to adapt to new buildings without human demonstrations. 
VTInstructor: Visual Trajectory Prompting for Navigation Instruction Generation in Continuous EnvironmentsACM International Conference on Multimedia (ACM MM) 2026
VTInstructor converts RGB navigation videos into explicit visual trajectory prompts, generating grounded instructions in continuous environments without navigation graphs or pre-built maps. 
NavSpace: How Navigation Agents Follow Spatial Intelligence InstructionsInternational Conference on Robotics and Automation (ICRA) 2026
NavSpace evaluates navigation agents on six types of spatial instructions and introduces SNav, a model trained to follow these instructions in simulation and on real robots. 
CorrectNav: Self-Correction Flywheel Empowers Vision-Language-Action Navigation ModelAAAI Conference on Artificial Intelligence 2026
CorrectNav learns from its own navigation failures by repeatedly generating self-correction data, improving error recovery and instruction following through an iterative post-training process. 
InstructNav: Zero-shot System for Generic Instruction Navigation in Unexplored EnvironmentConference on Robot Learning (CoRL) 2024
InstructNav combines dynamic navigation reasoning with multi-sourced value maps to follow diverse instructions in unexplored environments, without navigation training or pre-built maps. 
Discuss Before Moving: Visual Language Navigation via Multi-expert DiscussionsInternational Conference on Robotics and Automation (ICRA) 2024
DiscussNav consults specialized large models for instruction understanding, scene perception, and progress estimation before each move, enabling zero-shot navigation through multi-expert reasoning. 
Bridging Zero-Shot Object Navigation and Foundation Models through Pixel-Guided Navigation SkillInternational Conference on Robotics and Automation (ICRA) 2024
PixNav connects foundation models to robot movement through an RGB navigation policy guided by target pixels, supporting zero-shot object navigation with language-model planning. Multimodal Reasoning
Embodied3DBench: Benchmarking Low-Level Embodied Spatial Intelligence of Vision Language ModelsACM International Conference on Multimedia (ACM MM) 2026
Embodied3DBench evaluates spatial understanding and interaction-oriented perception through over 21K questions across six task categories, with 1.3M synthetic training examples to improve these capabilities. 
Improving Situated Conversational Agents with Step-by-Step Multi-modal Logic ReasoningChampion of SIMMC 2.1 Competition,
DSTC 11 Workshop
(Best Paper)
A dual-system dialogue agent combines a transformer with explicit step-by-step multimodal reasoning to locate objects, query attributes, and resolve complex user requests in visual scenes. 
Whether you can locate or not? Interactive Referring Expression GenerationACM Multimedia (MM) 2023
IREG refines object descriptions through feedback from a comprehension model, using its localization results to generate clearer referring expressions over multiple rounds of interaction. 
Multimodal Recommendation Dialog with Subjective Preference: A New Challenge and BenchmarkFindings of the Association for Computational Linguistics (Findings of ACL) 2023
SURE introduces 12K shopping dialogues annotated with subjective preferences and expert recommendation strategies, alongside three benchmark tasks for evaluating multimodal recommendation agents. 
SPRING: Situated Conversation Agent Pretrained with Multimodal Questions from Incremental Layout GraphAAAI Conference on Artificial Intelligence 2023
(Oral)
SPRING learns spatial relations and visual attributes through multimodal question answering over incremental layout graphs, improving multi-hop reasoning for dialogue in complex visual scenes. Experience
2025–2026
 JD
2023–2025
 AgiBotServicesReviewer
Selected Awards and Honors
|
