Di Chai

Assistant Professor

School of Computing and Artificial Intelligence
Shanghai University of Finance and Economics

Di Chai
Shanghai, China

My research focuses on machine learning systems, with a particular interest in LLM systems. I aim to build high-performance, trustworthy systems with real-world impact. I received my Ph.D. from HKUST, advised by Prof. Kai Chen and Prof. Qiang Yang.

I have also collaborated closely with Prof. Leye Wang and Prof. Junxue Zhang.

News

  • Canopy, on tree-aware rollout scheduling for agent reinforcement learning, accepted to NeurIPS 2026.
  • PrefixFlow, on training-time KV caching for prefix reuse in LLM RL training, accepted to ACM SIGOPS ATC 2026.
  • KVMem preprint released on arXiv: virtualizing million-token agent workspaces on a consumer GPU.
  • UCTB received the 2026 Best Paper Award from CCF Transactions on Pervasive Computing and Interaction.
  • Updated Centrifuge manuscript (ICLR 2026) available on arXiv.

Highlighted Projects

KVMem

arXiv 2026 · Open source

Virtualizes million-token agent workspaces on a consumer GPU, keeping history as reusable KV blocks and retrieving only the context needed for each step.

Paper ↗ Code ↗ Tutorial video ↗
Research insight

The paper separates the persistent KV workspace from the model's active context window. Query-conditioned retrieval selects relevant historical blocks and assembles a bounded execution view, reusing previously computed KV state instead of repeatedly prefilling recalled text.

Centrifuge

ICLR 2026 CCF-A · Open source

Algorithm–system co-design that turns token filtering into practical training speedups while preserving its model utility benefits.

Paper ↗ Code ↗
Research insight

Token filtering alone does not guarantee faster training. Centrifuge filters activations in attention backward and transforms sparse GEMM into dimension-reduced dense GEMM, enabling efficient execution with standard machine learning libraries.

Up to 34.7% less end-to-end training time. At 50% token filtering; maximum across evaluated settings on 1.1B–40B models.

Conference abstract ↗

To Prospective Students

Who I'm looking for

I am looking for master's and PhD students with strong computer science fundamentals and a genuine passion for building systems. You should enjoy understanding how systems work, turning ideas into working implementations, and investigating why something fails or performs poorly.

I value learning ability and curiosity more than extensive prior research experience. What matters is your willingness to tackle unfamiliar problems: learning the concepts and tools you need, asking thoughtful questions, and testing ideas through careful reasoning and experimentation. I appreciate students who take initiative and revise their assumptions when the evidence calls for it.

What you'll work on

Building high-performance LLM systems

Topics include, but are not limited to:

  • High-performance, compaction-free KV-cache management for long-horizon agent systems
  • High-performance inference systems for resource-constrained environments
  • High-performance LLM cascade systems

How to apply

Please email chaidi@mail.sufe.edu.cn with a brief introduction to your background and interests. You are welcome to share a project you have built or a technical problem you enjoyed solving—from coursework, personal exploration, open-source contributions, or research. I would like to hear how you approached it and what you learned.

Suggested subject line: [Prospective Student] Your Name - Master's or PhD - Entry Year

Email me

Selected Publications

2026

Canopy: Tree-Aware Rollout Scheduling for Agent Reinforcement Learning

Feiyuan Zhang, LI Pengbo, Ziniu Li, Yuhao Jiang, Di Chai, Han Tian, Junhao Wang, Taiqiang Wu, Guanhua Huang, Chaoliang Zeng, Yihao Liu, Kai Chen

NeurIPS 2026 (to appear) CCF-A

2025

2024

All publications (18)Show selected publications only

More publications

2026

2024

2023

2022

2021

2018

Click image or background to close · Esc