Ph.D. Students of Computer Science & Engineering at HKUST
Di Chai is a Ph.D. student of computer science and engineering at the Hong Kong University of Science and Technology. He is under the supervision of Prof. Kai Chen and Prof. Qiang Yang. His research interest is high-performance machine learning systems, such as efficient distributed matrix decompositions and training systems for large language models.
Email: dchai[at]cse[dot]ust[dot]hk
Curriculum Vitae and Research Statement
Token filtering aims to enhance the utility of large language models (LLMs) by removing inconsequential tokens during training. However, previous methods have not achieved significant improvements due to limited sparsity from filtering only in output layers and inefficient sparse GEMM processes. We address these issues by filtering tokens across all layers and optimizing GEMM operations, resulting in notable reductions in backpropagation time (more than 30%) and overall training time (more than 20%). Evaluations show that our system improves model utility compared to standard training while reducing training time significantly, and it can be easily integrated into existing frameworks. [Paper]
Di Chai, Junxue Zhang, Liu Yang, Yilun Jin, Leye Wang, Kai Chen, & Qiang Yang. “Efficient Decentralized Federated Singular Vector Decomposition”. USENIX ATC’24 Accepted. [Paper] [Code]
Di Chai, Leye Wang, Junxue Zhang, Liu Yang, Shuowei Cai, Kai Chen, & Qiang Yang. “Practical Lossless Federated Singular Vector Decomposition Over Billion-Scale Data”. KDD 2022, research track. [Paper] [Code]
Han Tian, Chaoliang Zeng, Zhenghang Ren, Di Chai, Junxue Zhang, Kai Chen, Qiang Yang, “Sphinx: Enabling privacy-preserving online learning over the cloud”. IEEE S&P 2022. [Paper]
Di Chai, Leye Wang, Qiang Yang. “Bike Flow Prediction with Multi-Graph Convolutional Networks”. SIGSPATIAL/GIS 2018: 397-400 (2018). [Paper] [Code] [Google Scholar 300+ citations]
Di Chai*, Leye Wang*, Liu Yang, Junxue Zhang, Kai Chen, & Qiang Yang. “A Survey for Federated Learning Evaluations: Goals and Measures”. IEEE TKDE accepted (2024). (*Equal Contribution) [Paper] [Code]
Di Chai, Leye Wang, Kai Chen, Qiang Yang, “Secure Federated Matrix Factorization”. IEEE Intelligent Systems, 36(5): 11-20 (2021). [Paper] [Code] [Google Scholar 300+ citations]
Di Chai, Leye Wang, Kai Chen, Qiang Yang, “Efficient Federated Matrix Factorization against Inference Attacks”. ACM TIST (2021). [Paper]
Leye Wang, Di Chai, Xuanzhe Liu, Liyue Chen, & Kai Chen. “Exploring the Generalizability of Spatio-Temporal Traffic Prediction: Meta-Modeling and an Analytic Framework”. IEEE TKDE, accepted (2021). [Paper] [Code]
Liu Yang, Junxue Zhang, Di Chai, Leye Wang, Kun Guo, Kai Chen, Qiang Yang, “Practical and secure federated recommendation with personalized mask”. International Workshop on Trustworthy Federated Learning (2022). [Paper]
Shuowei Cai, Di Chai, Liu Yang, Junxue Zhang, Yilun Jin, Leye Wang, Kun Guo, Kai Chen, “Secure forward aggregation for vertical federated neural networks”. International Workshop on Trustworthy Federated Learning (2022). [Paper]
Cengguang Zhang, Junxue Zhang, Di Chai, Kai Chen, “Aegis: A Trusted, Automatic and Accurate Verification Framework for Vertical Federated Learning”. IJCAI FL-Workshop (2021), Best Application Award. [Paper]