Today, we are announcing the general availability of Outerbounds Platform: A fully managed platform for data-intensive and ML workloads, which works seamlessly with open-source Metaflow, helping data scientists and engineers to design, develop, and deploy more ML applications, faster.
Why is building and operating AI and ML-powered applications so hard?
First, these systems are extremely complex beasts: It takes a large whiteboard to sketch the architecture of, say, a modern recommendation system. More so, the particular architecture you sketched can’t be directly applied to other use cases, thanks to the exhilarating diversity of applications that can benefit from ML and AI.
Second, these systems are subject to constant change: Models and business logic need to adapt swiftly to changes in their environment, reflected in data. Third, these systems need to be executed on a thick stack of machinery, tended by a small army of engineers.
Today, we are excited to announce the general availability of Outerbounds Platform – an enterprise-ready fully managed ML platform, which, together with open-source Metaflow, addresses the three challenges: Data scientists can design and develop applications iteratively using the ergonomic APIs of Metaflow. When it is time to test applications at scale and deploy them in production, data scientists can do this iteratively and independently on Outerbounds Platform, complying with your company's practices and processes, defined upfront by your engineers.
Since this is the first time we talk about the platform publicly, let’s start with the why and recount the story of how we got here.
Metaflow: A Human-Centric Way to Develop ML Apps
When we started developing open-source Metaflow at Netflix in 2017, we focused on the first two challenges: To tame the inherent complexity of these systems, we wanted to make it easy for data scientists to express their ideas in straightforward Python. We didn't believe in silver bullets, cute-but-simplistic gimmicks, or novel paradigms with a steep learning curve.
In our experience, the best ML applications are tailor-made by humans with domain expertise, using pragmatic tools that are purpose-built for developing data-intensive applications. The tool should keep track of all changes automatically – including changes in data, code, models, and people – so systems can grow over time through continuous, steady improvements.
In particular, Metaflow acknowledges that the journey from prototype to production is a gradual process, and the concept of “production” is not a binary one. Production can mean anything from an internal dashboard or a limited-scale A/B experiment to a Netflix-scale, low-latency, business-critical, customer-facing product. We wanted to make it possible to evolve the former to the latter without imposing too much overhead upfront, and without requiring drastic – and costly – re-platforming along the way.
Metaflow Success Stories
Thanks to increased productivity unlocked by Metaflow, Netflix was able to expand to whole new areas of ML-driven use cases, allow data scientists to conduct real-life experiments autonomously, and have a clear path for promoting the most promising ideas to production quickly. To give an idea of the scale, by 2021 Netflix’s Metaflow deployment included thousands of projects and millions of executions and the numbers are growing quickly.
Since open-sourcing of Metaflow in 2019, we have seen many other well-known organizations, such as CNN, 23andMe, Realtor.com, and MoneyLion, repeat this success story. With support from the friendly Metaflow community, their engineering teams have been able to leverage Metaflow’s open-source templates to set up and operate the infrastructure to support their data scientists.
Hearing your thoughts and feedback on our 2000+ member community has been immensely helpful in improving Metaflow further and understanding its gaps – thank you all! Keep your questions, thoughts, and feedback coming.
Besides the open-source community, the company got started thanks to support from Apoorva Pandhi (now at Zetta Ventures) and Steve Vassallo at Foundation Capital (also an early investor in Netflix):
"Netflix will spend $17 billion on content this year,” said Steve Vassallo, general partner at Foundation Capital. “And while real-life humans are still responsible for creating the bingeworthy scenes, how that gigantic budget is allocated rides on machine learning models that run on Metaflow, a powerful open-source framework created by the brilliant founding team of Outerbounds. We are absolutely thrilled to back them and add to Foundation’s deep portfolio of AI/ML infrastructure investments focused on empowering modern data teams with better, more scalable, more integrated, human-centered tools."
Introducing the Outerbounds Platform
As we supported hundreds of organizations on their Metaflow journey, we started to see patterns emerge. While the open-source Metaflow is common for all organizations, a great deal of variation exists on the infrastructure side.
Some variation is expected and desirable due to genuine differences in organizations and the businesses behind them, but much of it is accidental. Ultimately, the outer bounds of common engineering concerns - security, scalability, cost-efficiency, high-availability, and integrations to surrounding systems - look similar. At the high level, all happy infrastructures are alike.
Based on lessons we had learned from organizations we had worked with - small, large, and every size in between - we set out to bake all infrastructural concerns and best practices into one platform, Outerbounds Platform, which we are finally happy to make publicly available today.
We hope that Outerbounds Platform will help organizations produce ML-powered value faster and apply ML to new business domains, following the success stories of Netflix and other early adopters. The platform allows them to shortcut years of time and effort that they would otherwise incur.
Outerbounds Platform is a fully managed service that runs securely on your AWS account (crucially, data and code never leave your premises) providing an optimized backend for projects developed with Metaflow. The platform addresses the third challenge in ML projects: Instead of having a village of engineers providing infrastructure for ML, they can focus on solving business-specific challenges.
ML and data applications can’t live on an island or a walled garden, so a truly effective platform must adapt to its environment - your microservices, policies, and processes - with minimal friction. Hence, by design, Outerbounds Platform is not a black box. It adapts to the bespoke and changing requirements of your engineers and organization by providing a set of battle-hardened patterns and guardrails that can be composed to fit your needs. Better still, data scientists get to rely on the well-documented tried-and-true, Metaflow that will always remain open-source.
You can read more about Outerbounds Platform on our new product page. If you are interested in deploying it in your environment, which requires minimal engineering effort on your side, schedule a call with us.
In the words of another of our insightful investors, Greenoaks Capital:
"Metaflow is the way that the world’s best data scientists and engineers work,” said Sreyas Misra of Greenoaks. “We’re excited about Outerbounds because they abstract away complex engineering and infrastructure challenges, letting data scientists focus on what matters most: building new products and uncovering new insights. We think that few people know the challenges and opportunities of data science better than the team at Outerbounds, and we are thrilled to partner them as they accelerate their customers’ adoption of ML and AI.”
Onwards to an ML-driven future
We believe that we have seen only the early days of ML/AI adoption. Major technological shifts like ML, AI, and data-driven development will take decades to spread across industries – not only because of technical hurdles but also because of the pace of human change. Accounting for this is a part of our human-centric design: Metaflow and Outerbounds Platform can meet your organization where it is today, inviting your existing teams and systems to a long-term journey to an ML-driven future.
To support our long-term vision, we have partnered with top-tier investors, Foundation Capital, Amplify Partners, and Greenoaks Capital, who provide unique experience at the intersection of human-centric product design, developer tooling, ML/AI, enterprise infrastructure, and open-source. Thanks to them and a group of angel investors with deep domain expertise, we have raised over $24M in Seed and Series A funding.
In the words of Sarah Catanzaro at Amplify Partners:
Amplify is committed to backing companies that empower technical practitioners to build better, safer, models and applications. As such, we were thrilled to support Outerbounds as they dramatically improve the agility of ML teams and the reliability of the products they build.
For over a decade, Amplify has supported founders as they set new standards for model and application development and management. Based on this experience, we strongly believe that Outerbounds will redefine the process whereby data science projects are developed and deployed.
Amplify backs founders building technical tools and platforms that change how developers work. We’re so excited by how Outerbounds helps data scientists run more experiments and move from local development to production faster.
Get started today
Get a feel of Metaflow in the browser by signing up for a free Metaflow sandbox.
And, to learn more about Outerbounds Platform, read the product page and schedule a call with us.
Also, be sure to join our community of over 2000 data scientists and engineers!
PS. If our approach that combines a clear vision, deep tech, and a human-centric product experience resonates with you, we are hiring.