Yiwen Song

Yiwen Song (宋怡文)

Research Scientist @ Google Cloud AI Research

I am a Research Scientist at Google Cloud AI Research. My research focuses on Large Language Models, particularly at the intersection of multimodality, generative AI, and agentic systems. I am passionate about advancing how models reason, plan, and interact with complex, real-world environments.

Prior to joining Google, I spent four years at Meta conducting research and building training frameworks in the computer vision domain. I was a core contributor to Llama 3 and served as a maintainer for the TorchVision library.

News

2026-10
Attending COLM 2026 in San Francisco (Oct 6–9). Join our demo & Q&A on Co-Director: Agentic Generative Video Storytelling at the Google booth (#107) on Tuesday, Oct 6, 1:00–1:30 PM PT.
2026-09
Two papers accepted to NeurIPS 2026: "ScholarPeer: A Context-Aware Multi-Agent Framework for Automated Peer Review" and "LLM-Based Multi-Agent Blackboard System for Information Discovery in Data Science".
2026-09
Two papers accepted to EMNLP 2026 (Findings): "CANVAS: Continuity-Aware Narratives via Visual Agentic Storyboarding" and "Judging with Confidence: Calibrating Autoraters to Preference Distributions".
2026-09
Google Research blog post on Co-Director: Automating coherent long-form video generation.
2026-07
Gave a demo for Co-Director at the Google booth & kiosk at ICML 2026.
2026-07
Two papers accepted to COLM 2026: "Co-Director: Agentic Generative Video Storytelling" and "PaperOrchestra: A Multi-Agent Framework for Automated AI Research Paper Writing".
2026-05
Two papers accepted to ICML 2026: "TFRBench: A Reasoning Benchmark for Evaluating Forecasting Systems" and "The ACUTE Protocol: Operationalizing Language Model Activations for Better Calibration, Utility, and Trust".
2026-04
Gave a talk about PaperOrchestra at the BAAI online seminar.
2026-04
Released two new preprints: "CANVAS: Continuity-Aware Narratives via Visual Agentic Storyboarding" and "Co-Director: Agentic Generative Video Storytelling".
2026-04
Released a new preprint: "PaperOrchestra: A Multi-Agent Framework for Automated AI Research Paper Writing".
2026-03
Released a new preprint: "VQQA: An Agentic Approach for Video Evaluation and Quality Improvement".
2026-03
Our paper "Watch and Learn: Learning to Use Computers from Online Videos" was accepted to CVPR 2026.
2026-02
Our paper "Synapse: Adaptive Arbitration of Complementary Expertise in Time Series Foundational Models" was accepted to TMLR.
2026-01
Released a new preprint: "ScholarPeer: A Context-Aware Multi-Agent Framework for Automated Peer Review".
2025-07
Released the technical report for Gemini 2.5.
2025-04
Released the technical report for ShieldGemma 2.

Publications

See all publications on Google Scholar.

Agentic Video Generation and Storytelling

Co-Director: Agentic Generative Video Storytelling
[Paper] [Project] [Code] [Blog]
Yale Song, Yiwen Song, Nick Losier, Nathan Hodson, Ye Jin, Rhyard Zhu, Yan Xu, Daniel Vlasic, Carina Claassen, Jasmine Leon, Khanh G. LeViet, Zack Chomyn, Joe Timmons, Brett Slatkin, Scott Penberthy, Tomas Pfister
COLM 2026
CANVAS: Continuity-Aware Narratives via Visual Agentic Storyboarding
[Paper]
Ishani Mondal, Yiwen Song, Mihir Parmar, Palash Goyal, Jordan Boyd-Graber, Tomas Pfister, Yale Song
EMNLP 2026 Findings
VQQA: An Agentic Approach for Video Evaluation and Quality Improvement
[Paper] [Project]
Yiwen Song, Tomas Pfister, Yale Song
arXiv 2026

AI Agents for Scientific Research

PaperOrchestra: A Multi-Agent Framework for Automated AI Research Paper Writing
[Paper] [Project] [Code]
Yiwen Song, Yale Song, Tomas Pfister, Jinsung Yoon
COLM 2026
ScholarPeer: A Context-Aware Multi-Agent Framework for Automated Peer Review
[Paper]
Palash Goyal, Mihir Parmar, Yiwen Song, Hamid Palangi, Tomas Pfister, Jinsung Yoon
NeurIPS 2026
LLM-Based Multi-Agent Blackboard System for Information Discovery in Data Science
[Paper]
Alireza Salemi, Mihir Parmar, Palash Goyal, Yiwen Song, Jinsung Yoon, Hamed Zamani, Tomas Pfister, Hamid Palangi
NeurIPS 2026
ScientistOne: Towards Human-Level Autonomous Research via Chain-of-Evidence
[Paper] [Project]
Rui Meng, Bhavana Dalvi Mishra, Jiefeng Chen, Chun-Liang Li, Palash Goyal, Mihir Parmar, Yiwen Song, Yale Song, Rajarishi Sinha, Parthasarathy Ranganathan, Burak Gokturk, Jinsung Yoon, Tomas Pfister
arXiv 2026
PaperBanana-Interact: Scientific Diagram Refinement with Multi-Turn Human Feedback
[Paper] [Project]
Xueqing Wu, Ashwin Balasubramanian, Bingxuan Li, Dawei Zhu, Kai-Wei Chang, Yale Song, Yiwen Song, Rui Meng, Tomas Pfister, Nanyun Peng
arXiv 2026

LLM Agents: Planning, Memory, and Test-Time Scaling

Watch and Learn: Learning to Use Computers from Online Videos
[Paper] [Project] [Media]
Chan Hee Song, Yiwen Song, Palash Goyal, Yu Su, Oriana Riva, Hamid Palangi, Tomas Pfister
CVPR 2026
Plan-Tuning: Post-Training Language Models to Learn Step-by-Step Planning for Complex Problem Solving
[Paper] [Code]
Mihir Parmar, Palash Goyal, Xin Liu, Yiwen Song, Mingyang Ling, Chitta Baral, Hamid Palangi, Tomas Pfister
EMNLP 2025
In Prospect and Retrospect: Reflective Memory Management for Long-term Personalized Dialogue Agents
[Paper]
Zhen Tan, Jun Yan, I-Hung Hsu, Rujun Han, Zifeng Wang, Long T. Le, Yiwen Song, Yanfei Chen, Hamid Palangi, George Lee, Anand Iyer, Tianlong Chen, Huan Liu, Chen-Yu Lee, Tomas Pfister
ACL 2025
On the Role of Feedback in Test-Time Scaling of Agentic AI Workflows
[Paper]
Souradip Chakraborty, Mohammadreza Pourreza, Ruoxi Sun, Yiwen Song, Nino Scherrer, Furong Huang, Amrit Singh Bedi, Ahmad Beirami, Jindong Gu, Hamid Palangi, Tomas Pfister
arXiv 2025
HEART: Emotionally-Driven Test-Time Scaling of Language Models
[Paper]
Gabriela Pinto, Palash Goyal, Mihir Parmar, Yiwen Song, Souradip Chakraborty, Zifeng Wang, Jinsung Yoon, Tomas Pfister, Hamid Palangi
arXiv 2025

Calibration, Evaluation, and Trust

The ACUTE Protocol: Operationalizing Language Model Activations for Better Calibration, Utility, and Trust
[Paper]
Nishant Subramani, Palash Goyal, Yiwen Song, Mani Malek, Yuan Xue, Tomas Pfister, Hamid Palangi
ICML 2026
Judging with Confidence: Calibrating Autoraters to Preference Distributions
[Paper]
Zhuohang Li, Xiaowei Li, Chengyu Huang, Guowang Li, Katayoon Goshvadi, Bo Dai, Dale Schuurmans, Paul Zhou, Hamid Palangi, Yiwen Song, Palash Goyal, Murat Kantarcioglu, Bradley A. Malin, Yuan Xue
EMNLP 2026 Findings

Time Series Foundation Models

TFRBench: A Reasoning Benchmark for Evaluating Forecasting Systems
[Paper] [Project]
Md Atik Ahamed, Mihir Parmar, Palash Goyal, Yiwen Song, Long T. Le, Qiang Cheng, Chun-Liang Li, Hamid Palangi, Jinsung Yoon, Tomas Pfister
ICML 2026
Synapse: Adaptive Arbitration of Complementary Expertise in Time Series Foundational Models
[Paper]
Sarkar Snigdha Sarathi Das, Palash Goyal, Mihir Parmar, Yiwen Song, Long T. Le, Lesly Miculicich, Jinsung Yoon, Rui Zhang, Hamid Palangi, Tomas Pfister
TMLR 2026

Computer Vision (Meta)

Emu: Enhancing Image Generation Models Using Photogenic Needles in a Haystack
[Paper]
Xiaoliang Dai, Ji Hou, Chih-Yao Ma, Sam Tsai, Jialiang Wang, Rui Wang, Peizhao Zhang, Simon Vandenhende, Xiaofang Wang, Abhimanyu Dubey, Matthew Yu, Abhishek Kadian, Filip Radenovic, Dhruv Mahajan, Kunpeng Li, Yue Zhao, Vladan Petrovic, Mitesh Kumar Singh, Simran Motwani, Yi Wen, Yiwen Song, Roshan Sumbaly, Vignesh Ramanathan, Zijian He, Peter Vajda, Devi Parikh
arXiv 2023
PACO: Parts and Attributes of Common Objects
[Paper] [Code]
Vignesh Ramanathan, Anmol Kalia, Vladan Petrovic, Yi Wen, Baixue Zheng, Baishan Guo, Rui Wang, Aaron Marquez, Rama Kovvuri, Abhishek Kadian, Amir Mousavi, Yiwen Song, Abhimanyu Dubey, Dhruv Mahajan
CVPR 2023

Technical Reports

Gemini 2.5: Pushing the Frontier with Advanced Reasoning, Multimodality, Long Context, and Next Generation Agentic Capabilities
[Paper] [Blog]
Gemini Team, Google
Technical Report, 2025
ShieldGemma 2: Robust and Tractable Image Content Moderation
[Paper] [Model]
Wenjun Zeng, Dana Kurniawan, Ryan Mullins, Yuchi Liu, Tamoghna Saha, Dirichi Ike-Njoku, Jindong Gu, Yiwen Song, Cai Xu, Jingjing Zhou, Aparna Joshi, Shravan Dheep, Mani Malek, Hamid Palangi, Joon Baek, Rick Pereira, Karthik Narasimhan
Technical Report, 2025
The Llama 3 Herd of Models
[Paper] [Webpage]
Llama Team, AI @ Meta Core Contributor
Technical Report, 2024