Supporting Diverse ML Systems at Netflix
The Netflix TechBlog
MARCH 7, 2024
Once we have discovered the Parquet files to be processed, MetaflowDataFrame takes over: it downloads data using Metaflow’s high-throughput S3 client directly to the process’ memory, which often outperforms reading of local files. In other cases, it is more convenient to share the results via a low-latency API.
Let's personalize your content