Inside Outerbounds: Architecture Explained

Learn what makes Outerbounds tick under the hood.

Outerbounds provides a simple, human-centric developer platform built on Metaflow, abstracting away infrastructure complexity so ML and AI developers can enjoy their work and focus on building and shipping innovative projects.

Born at Netflix, Metaflow was able to leverage the work of hundreds of top-notch platform engineers who had built systems for data, compute, orchestration, observability, infosec, and cloud cost optimization, all running on AWS. Working in close collaboration with hundreds of leading ML/AI organizations, Netflix included, we have been building Outerbounds to include these capabilities and best practices so that they are easily available to organizations across industries, across clouds.

A platform engineer’s platform

Outerbounds is developed by platform engineers as the kind of solution we ourselves would want to adopt. While our focus is unabashedly on a polished developer experience, the underlying platform is deliberately not overly prescriptive: it’s designed to adapt to the unique needs of each organization, enabling local engineering teams to configure it to match their specific policies and requirements, and to use it as a foundation for building their own domain-specific abstractions. AI and ML are eating the world, taking on a diverse range of use cases, so any one-size-fits-all approach is simply untenable.

If you are a platform engineer looking for a battle-hardened, easily extensible foundation for ML/AI projects, securely running in your environment, keep on reading for technical details. If you are already familiar with Metaflow, the following should give you an idea of how Outerbounds differs from a typical open-source setup.

The big picture - Bring Your Own Cloud(s)

Outerbounds deploys in one or more cloud accounts of yours, optionally spanning multiple cloud providers, regions, and on-premise resources. In this architecture diagram, the top boxes represent your data plane, spanning your cloud accounts and on-premise resources, connected to the Outerbounds control plane at the bottom:

The core platform is deployed as a dedicated Kubernetes cluster in your account, using the cloud-native Kubernetes services, GKE on Google, AKS on Azure, and EKS on AWS with scale-to-zero capabilities, managed by Outerbounds and optimized for compute-intensive workloads. Optionally, Outerbounds can be connected to customer-managed on-premise Kubernetes clusters.

All compute, data processing, and metadata resides securely in your (cloud) premises - including cloud workstations for developers and privately hosted GenAI/LLM endpoints. No data or metadata is stored outside the data plane. The control plane is only connected to the infrastructure layer, receiving telemetry about the platform health, which allows us to provide the complete platform as a fully managed service with a guaranteed SLA.

The deployment model is the same as with open-source Metaflow, which you can deploy in a single cloud account of yours without exposing data to any 3rd parties - a critical feature for most of our customers. However, Outerbounds takes the setup to the next level, as highlighted below.

Details matter

While it's straightforward to sketch out a high-level architecture that makes conceptual sense, the real challenge lies in implementation where the technical details get complex and gnarly to get right. This is what sets Outerbounds apart: years of production use at scale, managing petabytes of metadata, clusters spanning thousands of instances and billions of executions, all vetted by infosec and platform teams at top companies.

Outerbounds offers platform engineers full flexibility in their level of involvement. You can start using Outerbounds immediately with zero engineering effort - the deployment takes a few clicks and 15 minutes.

Alternatively, you can leverage Outerbounds as a foundation to create a customized development experience tailored to your organization’s unique needs, giving your developers superpowers tuned to your specific domain, following the footsteps of companies such as Netflix, Ramp, and Zillow.

Preflight checklist

When working with larger organizations, we often come across frequently asked questions that need to be clarified before diving into any technical work. Let’s explore some of the most common ones.

‍

Security, privacy, and compliance

Thanks to our strict interpretation of Bring-Your-Own-Cloud, all compute takes place in your (cloud) premises and all data, metadata, and logs are stored in your account. This allows us to breeze through security reviews. Furthermore, you can define granular access policies for your internal teams, as described below.

For larger enterprises, we provide support for custom security agents on nodes, preferred network configurations, and exportable audit trails to integrate with your event management and security systems.

Outerbounds is SOC2 and HIPAA compliant, with many existing customers in highly regulated industries such as financial services and healthcare. For details, see trust.outerbounds.com.

Cloud and region support

You can deploy Outerbounds to AWS, GCP, or Azure. Notably, you can unify resources from multiple clouds into a single platform, allowing you to leverage services and compute across clouds with a consistent API. We support nearly all regions with customers operating across regions globally to maintain data residency requirements.

Outerbounds can be procured through AWS, GCP, or Azure marketplaces, allowing you to leverage Enterprise Discount Programs or cloud credits for easy cost optimization.

Should you need access to cost-efficient GPU resources, you can leverage our integration with CoreWeave, NVIDIA’s GPU cloud, or on-premise GPU clusters.

Shared responsibility model, support, and SLA

Outerbounds is a fully managed platform with a guaranteed SLA, monitored 24/7, designed for business-critical workloads. We take care of the core infrastructure concerns, as depicted in our shared responsibility model:

You can build your workflows, models, and applications on top of this robust foundation. Naturally you are the expert in your own domain, although our partners are happy to help, if you need extra resources.

Along with the platform, you'll receive access to a dedicated, private Slack channel for real-time support from Outerbounds, connecting you directly with the core Metaflow team to address your advanced needs. This access allows you to benefit from our extensive experience across hundreds of ML/AI organizations—no chatbots involved.

Drawing the outer bounds

Over the past few years, we have received hundreds of questions related to authentication and access control on the Metaflow community Slack channel. One of the key benefits of Outerbounds is its comprehensive, streamlined approach to access control, eliminating the need for custom solutions beyond your existing SSO and security policies.

Authentication and authorization

On Outerbounds, all access is managed through an SSO provider such as Google, Azure AD, Okta, or other SAML-compatible service, which provides an end-to-end identity for all platform operations.

Developers benefit from seamless, consistent permissions across their development environments, cloud compute, and production deployments without extra configuration. Permissions are centrally managed through IAM, utilizing your existing tools with our Bring-Your-Own-Policy model, as outlined below.

Besides human users, you can mint revokable machine-to-machine tokens for integrations, e.g. to trigger events from surrounding microservices.

Perimeters, RBAC, and policies

There are many use cases for logically isolated environments within a deployment. Typically, you may want to separate development, staging, and production environments. Or, you may need to draw boundaries around business units or teams. This has been one of the highly requested features in Metaflow.

Thanks to Outerbounds’ comprehensive authorization framework, you can accomplish this with perimeters which define securely isolated environments, each with their own metadata, artifacts, users, and permissions:

You can attach an IAM role, managed by you with your own policies, to be used in each perimeter by default. Through these policies, you can control what data, secrets, and other services are accessible in each environment. Furthermore, you can enforce policies such as approved container images and resources limits per perimeter.

Perimeters support role-based access control (RBAC), allowing you to choose the users who should have access to each environment, and the scope of their access. Should a user need access to multiple perimeters, it is easy to navigate between them on a workstation.

Separate development/staging/production environments are also a key element of effective GitOps workflows which work seamlessly with our CI/CD integration. Moreover, you can assign compute resources to perimeters flexibly: you can either dedicate compute pools in an environment to maximize availability or share them across multiple environment to maximize utilization.

Ready, Set, Deploy

Deploying Outerbounds is fast and straightforward. We provide a documented, easy-to-review CloudFormation or Terraform template which you can execute with a few clicks in the UI:

The deployment template defines a tight security boundary for Outerbounds. On AWS, it uses an IAM Permission Boundary to grant Outerbounds permissions to deploy an EKS cluster. On Azure, this is accomplished with a dedicated resource group and on GCP with a dedicated project. In all instances, Outerbounds can only access resources tightly scoped for the platform.

Go and grab lunch - the platform is ready for use when you come back.

Curious to learn more?

If you want to learn more, just schedule a call and get your questions answered. We offer a 30 day free trial, so you can test all of the above in your environment - including our support - to make sure that Outerbounds works for your needs.