Towards Distributed MLOps: Theory and Practice

As machine learning (ML) technologies get widely applied to many domains, it has become essential to rapidly develop and deploy ML models. Towards this goal, MLOps has recently emerged as a set of tools and practices for operationalizing production-ready models in a reliable and efficient manner. However, several open problems exist, including how to automate the ML pipeline that includes data collection, model training, and deployment (inference) with support for distributed data and models stored at multiple edge sites. In this talk, I will cover some theoretical foundations and practical approaches towards enabling distributed MLOps, i.e., MLOps in large-scale edge computing systems. I will start with explaining the requirements and challenges. Then, I will describe how our recent theoretical developments in the areas of coreset, federated learning, and model uncertainty estimation can support distributed MLOps. As a concrete example, I will dive into the details of a federated learning algorithm with flexible control knobs, which adapts the learning process to accommodate time-varying and unpredictable resource availabilities, as often seen in systems in operation, while conforming to a given budget for model training. I will finish the talk by giving an outlook on some future directions.