I am a PhD student (2024 - present) at University of Washington advised by Prof. Hannaneh Hajishirzi. I received my MS degree at University of Illinois Urbana-Champaign (UIUC), advised by Prof. Hao Peng. I received my B.S. degree from Tsinghua University in 2022. I work on Large Language Models.
Research Interest: Natural Language Processing, Large Language Model Pretraining, Reinforcement Learning, Machine Learning, Computer Systems, Distributed Computing.
Email / Github / Twitter / Google Scholar
Crystal: Illuminating LLM Abilities on Language and Code
COLM 2024
Web2Code: A Large-scale Webpage-to-Code Dataset and Evaluation Framework for Multimodal LLMs
NeurIPS 2024 Track Datasets and Benchmarks
SciCode: A Research Coding Benchmark Curated by Scientists
NeurIPS 2024 Track Datasets and Benchmarks
Pandora: Towards General World Model with Natural Language Actions and Video States
preprint
LLM360: Towards Fully Transparent Open-Source LLMs
COLM 2024
SlimPajama-DC: Understanding Data Combinations for LLM Training
preprint
Language Models Meet World Models: Embodied Experiences Enhance Language Models
NeurIPS 2023
On the Learning of Non-autoregressive Transformers
ICML 2022
Don't Take It Literally: An Edit-Invariant Sequence Loss for Text Generation
NAACL 2022