I am an Associate Professor of Electrical Engineering and Computer Science and Associate Professor of Computer Science and Engineering, Department of Electrical Engineering and Computer Science, College of Engineering. My research centers on bringing machine learning models to resource constrained devices. Powerful computer vision or language models can be trained and run on powerful servers in the cloud, but for users to realize their full benefits, they should be able to run on edge resources closer to users, such as smartphones or mixed reality headsets, while being safe and secure. My methodology is mainly software algorithms deployed on commodity hardware in order to demonstrate practical impact.
Please describe one or two of your most interesting projects.
One project my group worked on is cost-efficient use of large language models (LLMs). Recently, there has been a proliferation of LLMs by multiple providers, each with different inference accuracy, monetary cost, and latency. If a user has a finite monetary budget and wants to use LLMs to solve a series of problems, how should they allocate their dollars across diverse LLMs? We propose a reinforcement learning approach, TREACLE, that learns which LLMs to choose, intelligently trading off accuracy for cost. This work appeared in NeurIPS 2024.
What are 1-3 interesting facts about yourself?
- I started off as a undergraduate studying biomedical engineering.
- I have a diploma in piano performance.
- I love traditional Chinese lion dance and used to perform all over New York City during university.