Machine Learning at MoneyLion: From Days to Hours with Metaflow

MoneyLion, a U.S.-based fintech company with a tech hub in Malaysia, offers a suite of financial products, including loans and personalized banking services. Machine learning plays a critical role in automating their operations, especially in areas like loan underwriting and approval. Melvin Low, a key member of MoneyLion’s machine learning platform team, oversees the infrastructure that allows the data science team to build, deploy, and manage machine learning models.

The team at MoneyLion uses a variety of machine learning models for tasks such as classification, anomaly detection, and recommendation systems. As the demand for faster loan processing and personalization grew, the need for efficient MLOps became increasingly apparent. They needed a tool that could simplify the model development lifecycle, reduce deployment times, and streamline infrastructure management.

The Challenge: Complex Machine Learning Infrastructure

Before Metaflow, MoneyLion's machine learning operations were bogged down by the complexity of managing infrastructure and orchestrating workflows. The team was using Airflow, which, while powerful, was not developer-friendly. “Airflow is very orchestration and DAG-oriented, and it’s not developer-facing,” Low explained. This lack of developer accessibility slowed down model development and deployment.

Manual infrastructure management was a significant burden. Data scientists had to handle tasks such as setting up S3 buckets, managing Terraform scripts, and configuring data storage manually. This not only introduced delays but also required data scientists to spend valuable time on infrastructure, rather than on model building. Low noted, “Previously, they would need to manage their own S3 and go through the whole process of creating buckets.”

The overall process of deploying a machine learning model into production was slow and inefficient. Without automation, a simple model deployment could take up to three days, hindering MoneyLion’s ability to iterate quickly and scale their machine learning capabilities. The onboarding of new team members was also slow, as they had to learn the intricacies of infrastructure setup before becoming productive.

The Solution: Implementing Metaflow to Streamline MLOps

MoneyLion started a proof-of-concept (PoC) with Metaflow to address these challenges. The goal was to simplify model development and reduce deployment time by automating infrastructure management. Metaflow’s Python-centric design and its capabilities in data management and orchestration were immediately appealing. “They tried to deploy the whole Metaflow ecosystem using the step function version, and it was very well received,” Low explained.

One of the biggest advantages of Metaflow was its seamless data management, including automatic artifact storage and versioning. With Metaflow, data scientists no longer had to worry about manually configuring S3 or dealing with complex cloud infrastructure. “One thing about Metaflow that’s very nice is the client and artifact storage tooling where you don’t need to understand how it works under the hood,” Low said. This allowed the team to focus on building models instead of managing infrastructure.

To further optimize their workflow, MoneyLion developed an SDK wrapper around Metaflow, which made it even easier for data scientists to get started. The SDK pre-configured infrastructure settings, allowing users to simply install it and start developing pipelines. “We created custom commands and decorators, allowing us to cater to our specific needs,” Low added. This customization greatly reduced the time needed to onboard new team members and improved the overall efficiency of their workflow.

The Results: Faster Development, Reduced Overhead

Metaflow had a transformative impact on MoneyLion’s machine learning operations. One of the most significant improvements was the reduction in deployment time. What used to take up to three days—building and deploying a model—now only took about four hours. “Previously, maybe it would take three days, but now, technically, you can get it done in one day or less, half a day,” Low explained. This 88% reduction in deployment time allowed MoneyLion to iterate on models much faster, which in turn improved their ability to deliver quick loan approvals and enhanced customer experiences.

The automation of data management through Metaflow eliminated the need for manual configuration of S3 buckets and data versioning. This resulted in substantial time savings and allowed data scientists to focus solely on model development. “Metaflow already has a built-in connection to S3, so you skip that part, which is a major time saver,” Low noted.

Monitoring and troubleshooting were also streamlined. By integrating Metaflow with Slack, MoneyLion set up automatic notifications that alerted the team if a pipeline failed or encountered issues. “If a pipeline fails, we have automatic monitoring sending messages to Slack channels, saving time on setting that up,” Low explained. This proactive monitoring reduced the time spent on manual intervention and improved overall reliability.

Onboarding new data scientists became a much faster process, as well. Before Metaflow, setting up new hires required them to learn and configure various infrastructure components. With the custom SDK, however, the process was reduced to a simple pip install. “All the configuration is already in place. They just need to pip install, and they’re good to go,” Low said. This simplified onboarding cut the time it took for new team members to become productive in half.

MoneyLion’s adoption of Metaflow resulted in a significant transformation in how they manage their machine learning workflows. The time required to deploy models was reduced by 88%, going from three days to just four hours. This speed allowed the team to iterate on models faster and improve their loan underwriting processes, delivering a better experience to their customers.

Metaflow also greatly reduced the overhead associated with infrastructure management. By automating tasks like data versioning and S3 setup, the team no longer had to spend time on manual configurations. Monitoring pipelines became easier with automatic Slack notifications, improving the overall reliability of their machine learning operations.

The custom SDK built around Metaflow further enhanced MoneyLion’s workflows, making onboarding new data scientists much faster and reducing the complexity of setting up new pipelines. As Low emphasized, “Metaflow’s extensibility allowed us to tailor it to our needs, and that’s been a huge advantage.”

Looking ahead, MoneyLion plans to continue using Metaflow as a core part of their machine learning infrastructure. With its flexibility, ease of use, and ability to significantly reduce development time, Metaflow is poised to help MoneyLion scale their machine learning capabilities even further, as they continue to innovate in the fintech space.

By adopting Metaflow, MoneyLion achieved a dramatic reduction in deployment time, streamlined its machine learning operations, and improved overall productivity. This has positioned them for continued success as they expand their use of machine learning to deliver more advanced financial products and services.

‍