article thumbnail

Supporting Diverse ML Systems at Netflix

The Netflix TechBlog

For ETL and other heavy lifting of data, we mainly rely on Apache Spark. In addition to Spark, we want to support last-mile data processing in Python, addressing use cases such as feature transformations, batch inference, and training. Correspondingly, each application brings its own bespoke set of dependencies.

Systems 218
article thumbnail

Evolving from Rule-based Classifier: Machine Learning Powered Auto Remediation in Netflix Data…

The Netflix TechBlog

Service Integrations Figure 2 illustrates the integration of the services generating and applying the recommendations in the data platform. The major services are as follows: Nightingale is a service running the ML model trained using Metaflow and is responsible for generating a retry recommendation.

Tuning 202
Insiders

Sign Up for our Newsletter

This site is protected by reCAPTCHA and the Google Privacy Policy and Terms of Service apply.

article thumbnail

2021 Data/AI Salary Survey

O'Reilly

Most respondents participated in training of some form. Learning new skills and improving old ones were the most common reasons for training, though hireability and job security were also factors. Company-provided training opportunities were most strongly associated with pay increases. Demographics. Salaries by Gender.

Azure 145
article thumbnail

What is IT automation?

Dynatrace

AI that is based on machine learning needs to be trained. This requires significant data engineering efforts, as well as work to build machine-learning models. While automating IT processes without integrated AIOps can create challenges, the approach to artificial intelligence itself can also introduce potential issues.

article thumbnail

Post: Fauna, Sisu, Educative, PA File Sight, Etleap, Triplebyte, Stream

High Scalability

Etleap is analyst-friendly , enterprise-grade ETL-as-a-service , built for Redshift and Snowflake data warehouses and S3/Glue data lakes. Our intuitive software allows data engineers to maintain pipelines without writing code, and lets analysts gain access to data in minutes instead of months.

Education 105
article thumbnail

Post: Essilen Research, Fauna, Sisu, Educative, PA File Sight, Etleap, Triplebyte, Stream

High Scalability

Etleap is analyst-friendly , enterprise-grade ETL-as-a-service , built for Redshift and Snowflake data warehouses and S3/Glue data lakes. Our intuitive software allows data engineers to maintain pipelines without writing code, and lets analysts gain access to data in minutes instead of months.

Education 114
article thumbnail

Post: Essilen Research, Fauna, Sisu, Educative, PA File Sight, Etleap, Triplebyte, Stream

High Scalability

Etleap is analyst-friendly , enterprise-grade ETL-as-a-service , built for Redshift and Snowflake data warehouses and S3/Glue data lakes. Our intuitive software allows data engineers to maintain pipelines without writing code, and lets analysts gain access to data in minutes instead of months.

Education 113