Supporting Diverse ML Systems at Netflix
The Netflix TechBlog
MARCH 7, 2024
Once we have discovered the Parquet files to be processed, MetaflowDataFrame takes over: it downloads data using Metaflow’s high-throughput S3 client directly to the process’ memory, which often outperforms reading of local files. The back-end auto-scales the number of instances used to back your service based on traffic.
Let's personalize your content