Orchestrating Data/ML Workflows at Scale With Netflix Maestro
The Netflix TechBlog
OCTOBER 18, 2022
Meson was based on a single leader architecture with high availability. In the data domain, it is common to have a super large number of jobs within a single workflow. For example, a workflow to backfill hourly data for the past five years can lead to 43800 jobs (24 * 365 * 5), each of which processes data for an hour.
Let's personalize your content