About Me
I’m a first year Ph.D. student in the VITA Group of Electrical and Computer Engineering Department, the Univeristiy of Texas at Austin, under the supervision of Prof. Zhangyang (Atlas) Wang. Prior to that, I obatined my B.E. degree from University of Science and Technology of China (USTC). [CV]
I’m currently working on machine learning , with research focus on:
- Efficient Model Training: Mixture of Experts (MoE), Model Merging & Recycling
- Trustworthy Machine Learning System: Adversarial Robustness, Data Poisoning, Federated Learning
NEWS
- May, 2024. My intern project at NVIDIA “Flextron: Many-in-One Flexible Large Language Model” is accepted by ICML2024!
- May, 2024. “Learning to Compress Long Contexts by Dropping-In Convolutions” is accepted by ICML2024!
- Sep, 2023. I’ve just begun my incredible internship journey in NVIDIA.
- Sep, 2023. “$\mathrm{H_2O}$: Heavy-Hitter Oracle for Efficient Generative Inference of Large Language Models” is accepted by NeurIPS2023!
- Jul, 2023. “Robust Mixture-of-Expert Training for Convolutional Neural Networks” is accepted by ICCV2023!
- Apr, 2023. “Robust Weight Signatures: Gaining Robustness as Easy as Patching Weights?” is accepted by ICML2023!
- Feb, 2023. “Robust Weight Signatures: Gaining Robustness as Easy as Patching Weights?” is available on ArXiv!
- Sep, 2022. “Randomized Channel Shuffling: Minimal-Overhead Backdoor Attack Detection without Clean Datasets” is accepted by NeurIPS2022!
- Sep, 2022. My personal webpage is on!
Publication List
(A superscript * denotes equal contribution)
Flextron: Many-in-One Flexible Large Language Model
Ruisi Cai, Saurav Muralidharan, Greg Heinrich, Hongxu Yin, Zhangyang Wang, Jan Kautz, Pavlo Molchanov
ICML2024: International Conference on Machine Learning
Learning to Compress Long Contexts by Dropping-In Convolutions
Ruisi Cai, Yuandong Tian, Zhangyang Wang, Beidi Chen
ICML2024: International Conference on Machine Learning
Robust Mixture-of-Expert Training for Convolutional Neural Networks
Yihua Zhang, Ruisi Cai, Tianlong Chen, Guanhua Zhang, Huan Zhang, Pin-Yu Chen, Shiyu Chang, Zhangyang Wang, Sijia Liu
ICCV2023: International Conference on Computer Vision [Paper] [Code]
$\mathrm{H_2O}$: Heavy-Hitter Oracle for Efficient Generative Inference of Large Language Models
Zhenyu Zhang, Ying Sheng, Tianyi Zhou, Tianlong Chen, Lianmin Zheng, Ruisi Cai, Zhao Song, Yuandong Tian, Christopher Ré, Clark Barrett, Zhangyang Wang, Beidi Chen
NeurIPS2023: Conference on Neural Information Processing Systems, [Paper] [Code]
Robust Weight Signatures: Gaining Robustness as Easy as Patching Weights?
Ruisi Cai, Zhenyu Zhang, Zhangyang Wang
ICML2023: International Conference on Machine Learning [Paper] [Code]
Many-Task Federated Learning: A New Problem Setting and a Simple Baseline
Ruisi Cai, Xiaohan Chen, Shiwei Liu, Jayanth Srinivasa, Myungjin Lee, Ramana Kompella, Zhangyang Wang
CVPRW: 2nd Workshop on Federated Learning for Computer Vision [Paper]
Randomized Channel Shuffling: Minimal-Overhead Backdoor Attack Detection without Clean Datasets
Ruisi Cai*, Zhenyu Zhang*, Tianlong Chen, Xiaohan Chen, Zhangyang Wang
NeurIPS2022: Conference on Neural Information Processing Systems [Paper] [Code]