AI & ML from the Trenches: Adept.ai, Autodesk, and Epignosis – Massive Compute and GenAI
A summary of recent talks from enterprises building machine learning platforms with Metaflow.
Building the ML-driven future
Fortnightly posts covering major Metaflow and Outerbounds releases, community events, case studies, and all things machine learning infrastructure.
New in Metaflow: Train at scale with AI/ML frameworks
Today, we release support for distributed training and other high-performance computing workloads in Metaflow, powered by popular open-source frameworks like Ray, Torch, Deepspeed, Tensorflow and MPI, introduced in this article.
Compute Anything with Metaflow
Today, we release support for cost-efficient distributed training and other high-performance computing workloads in Metaflow. Read this post for the backstory.
LLMs, RAG, and Fine-Tuning: A Hands-On Guided Tour
A hands-on guided tour to LLMs, prompt engineering, retrieval-augmented generation, and building ML systems in the Metaflow Sandbox.
Outerbounds at KubeCon NA 2023
We’re excited to be in Chicago later this month for KubeCon North America.
AI & ML From the Trenches: Amazon Prime, DTN, and Realtor.com
A summary of recent talks from enterprises building machine learning platforms with Metaflow.
New in Metaflow: The Long-Awaited @pypi Decorator
You can now install dependencies from PyPI as well as Conda in your Metaflow steps.
AI & ML From the Trenches: DeliveryHero, Ramp, and Dell
A summary of recent talks from enterprises building machine learning platforms with Metaflow.
Better, Faster, Stronger LLM Fine-tuning
Reactive, configurable, cheaper LLM fine-tuning with Metaflow
Retrieval-Augmented Generation: How to Use Your Data to Guide LLMs
Learn how to use Retrieval Augmented Generation to control hallucinations and get more relevant responses from LLMs.
Seamless Data and ML Pipelines with Airflow and Metaflow
Learn how to develop ML, AI, and data science with Metaflow, deploy alongside existing data pipelines on Airflow.
Secure ML with Secure Software Dependencies
Engineers from Chainguard and Outerbunds discuss their common philosophy around security and machine learning workflows.
Announcing 4 New Features in the Outerbounds Platform: Cloud Workstations and More
Explore four major new features in the Outerbounds Platform
Learn Full-Stack Machine Learning: Batteries Included
Learn how you can get started with full-stack ML today in a hands-on, project-driven course using production-grade ML systems.
Fine-tuning a Large Language Model using Metaflow, featuring LLaMA and LoRA
A workflow template built with Metaflow for fine-tuning LLMs for custom use cases.
Large Language Models and the Future of Custom, Fine-tuned LLMs
An overview of instruction tuning for large language models using the LLaMA family of models.
The Open-Source Modern Data Stack featuring Apache Iceberg
We sat down with Jason Reid, a Co-founder at Tabular, to discuss the Apache Iceberg project and how it fits into the modern data stack.
New in Metaflow: Accessing Secrets Securely
You can now access secrets securely in Metaflow flows using the new secrets decorator.
Metaflow 2.9: Building Reactive Machine Learning Systems with Event Triggering
Using the new event triggering feature in Metaflow, you can compose systems using workflows as building blocks and trigger them based on external events.
Large Language Models, Business Value, and Risk
An interview with Federico Bianchi, a postdoctoral NLP researcher at Stanford exploring the boundaries of large language models.
Fast Data: Loading Tables From S3 At Lightning Speed
Load and process tabular data at lightning speed using Apache Arrow and Metaflow.
Case Study: How Realtor.com uses Metaflow
We sat down with Russell Brooks, Principal ML Engineer at Realtor.com, to discuss how his team uses Metaflow, and how it impacts Realtor.com.
Training a Large Language Model With Metaflow, Featuring Dolly
We use Metaflow to train Dolly, to show an example of fine-tuning LLMs and what it takes to use these models in practice.
Large Language Models and the Future of the ML Infrastructure Stack
A peek into how Outerbounds views the ongoing evolution of the machine learning stack, in the wake of recent LLM waves.
Announcing the Metaflow Card Viewer
Sharing results of your ML experiments with colleagues has never been easier with the Metaflow card viewer. Use it from your sandbox today!
Whisper with Metaflow on Kubernetes
How to run and optimize OpenAI's Whisper model on a state-of-the-art Kubernetes cluster powered by Metaflow.
Learn Machine Learning and AI Infrastructure in the Browser
Metaflow Sandboxes are now loaded with many new interactive ML tutorials: learn about cutting-edge AI and ML, right in your browser.
How MoneyLion Customized Metaflow
An interview with Yudhiesh Ravindranath, an MLOps Engineer at MoneyLion, detailing the extensions their team has built on top of Metaflow.
Better Airflow with Metaflow
You can now orchestrate workflows developed with Metaflow on Apache Airflow.
Announcing The Outerbounds Platform
Outerbounds Platform is now generally available! Get started with a managed platform for data and ML workloads that operates seamlessly with open-source Metaflow.
Event: How to Build a Full-Stack Recommender System
Join Jacopo Tagliabue and Hugo Bowne-Anderson in a live code along session to dive into how to build a production-grade recommender system.
Case Study: MLOps for FinTech using Metaflow
Yudhiesh Ravindranath, an MLOps Engineer at MoneyLion, discusses how his team builds and maintains an MLOps stack at a FinTech company.
Full Stack Machine Learning, ML Engineering, and SWE Skills for Data Science
Ethan Rosenthal joins Hugo to discuss how you can build your career and the layers of a full machine learning stack in your organization.
Metaflow Now Supports All the Major Clouds: Google Cloud, Azure, and AWS
Metaflow can now be used equally across Google Cloud, Azure, or AWS without any changes in user code or workflows.
The Moving Parts of the Full Machine Learning Stack and Building ML Platforms
Russell Brooks joins Hugo to discuss how you can build a valuable machine learning team and tech stack in your organization.
Case Study: MLOps for NLP-powered Media Intelligence using Metaflow
An interview with Jules Belveze, an MLOps Engineer at Hypefactors where they apply NLP and AI for media intelligence.
The 4 Main Tasks in the Production ML Lifecycle, MLOps, and Data-centric AI
A recap of a chat with Shreya Shankar about her paper, Operationalizing Machine Learning: Patterns and Pain Points from MLOps Practitioners.
Event: How to Build an Enterprise Machine Learning Platform from Scratch
A conversation with Russell Brooks and Hugo Bowne-Anderson on the journey to reproducible and automated ML-powered software.
MLOps for Foundation Models: Whisper and Metaflow
We show how to use large models like Whisper in a production-grade workflow by applying MLOps practices with Metaflow.
Now Available: Machine Learning with Metaflow on Azure
Metaflow on Azure provides one of the most user-friendly machine learning experiences on Azure, both for data scientists as well as engineers.
Ask Me Anything: Machine Learning Infrastructure for Humans
A collection of responses from an AMA event where Chip Huyen and Ville Tuulos addressed questions about machine learning infrastructure.
Event: Patterns and Pain Points from MLOps Practitioners
Shreya Shankar joins Hugo Bowne-Anderson to discuss her team’s recent paper Operationalizing Machine Learning: An Interview Study.
Applying the Practical Machine Learning Infrastructure Stack
How can you leverage modern tools to apply a state-of-the-art machine learning infrastructure stack in practice?
Announcing Metaflow Sandboxes: Free Data Science Infrastructure in the Browser
Use new Metaflow Sandboxes to evaluate Metaflow and the full infrastructure stack behind it in the browser without having to install anything locally
Parallelizing Stable Diffusion for Production Use Cases
How to run massively parallel Stable Diffusion for production use cases, producing new images automatically in a highly-available manner.
Event: Machine Learning + Infrastructure for Humans
A book launch event including a discussion with authors Chip Huyen and Ville Tuulos about machine learning education and practice.
Book Launch: Effective Data Science Infrastructure
By reading a new book, Effective Data Science Infrastructure, you will learn how to set up infrastructure for ML and data science applications, similar to the stack that powers Netflix and hundreds of other modern companies.
DataOps and MLOps for Reasonable Organizations
How to combine DataOps and MLOps to produce business value: good machine learning tools provide a better way to think about the division of work and culture.
Testable Technical Documentation with Notebooks, nbdev, and Docusaurus
Learn how you can streamline authoring, testing and maintenance of technical documentation with Jupyter Notebooks, nbdev and Docusaurus.
Machine Learning with the Modern Data Stack: A Case Study
Leveraging the Modern Data Stack for Machine Learning: how to combine DataOps and MLOps to produce business value.
5 Ways to Use the New Metaflow Tags
We are happy to announce support for mutable tags in Metaflow! Learn five key patterns for effective human-in-the-loop workflows which leverage the new feature.
Developing Scalable Feature Engineering DAGs
Discover the power of DAGs to level up your data science projects with examples from feature transforms to workflow orchestration.
Introducing Metaflow Resources for Data Science and Engineering
A knowledge base of free, practical data science and machine learning materials, including how to set up a modern data science infrastructure.
The Modern Stack of ML Infrastructure
A discussion about the machine learning deployment stack and why data makes software different.
Human-Centric Data Science on Kubernetes with Metaflow
Data scientists can now scale out their machine learning workflows to Kubernetes clusters using Metaflow and Argo, without needing to know the ins and outs of K8S.
Fireside chat #2: Teaching Practical Machine Learning
How to get started today with ML in production: tools, workflows, and mental models.
Fireside chat #1: How to Produce Sustainable Business Value with Machine Learning
A fireside chat about product management for ML products across various industries.
Excited to join Metaflow and Outerbounds
Hugo Bowne-Anderson discussed his journey to becoming Head of Developer Relations at Outerbounds.
Data Scientists Don't Need to Know Kubernetes with Metaflow
With Metaflow, data scientists can leverage K8s clusters for their work. Metaflow presents a user-friendly UX to data scientists, while working nicely with production infrastructure.
Notebooks in Production with Metaflow
Learn how to use notebooks in production ML pipelines with a new Metaflow feature. This is model framework-agnostic and so will work with all types of ML models.
Metaflow Highlights of 2021
Developed internally at Netflix and open-sourced in 2019, Metaflow is now used to power machine learning in production by hundreds of companies.
Integrating Pythonic Visual Reports into ML Pipelines
Introducing DAG cards for machine learning pipelines. These cards make it easy to attach custom visual reports in every workflow, without having to install any additional tooling.
MLOps vs DevOps: the Difference Data Makes
A discussion about the machine learning deployment stack and why data makes software different.
Machine Learning Pipelines: from Prototype to Production
Build a simple production-ready MLOps pipeline using Metaflow, an OSS framework allowing data scientists to build production-ready machine learning workflows using a simple Python API, and Seldon.
Smarter machines, built by happier humans
The future will be powered by dynamic, data intensive systems - built by happy humans using tooling that gives them superpowers
Get started for free