article thumbnail

3. Psyberg: Automated end to end catch up

The Netflix TechBlog

Audit Run various quality checks on the staged data. Several audits, such as verifying source and target counts, are performed on this batch of data. Publish If the audits are successful, cherry-pick the staging snapshot to publish the data to production.

Tuning 244
article thumbnail

What is IT automation?

Dynatrace

Monitoring and logging are fundamental building blocks of observability. Adding AIOps to automation processes makes the volume of data that applications and multicloud environments generate much less overwhelming. Similarly, digital experience monitoring is another ongoing process that lends itself to IT automation.

Insiders

Sign Up for our Newsletter

This site is protected by reCAPTCHA and the Google Privacy Policy and Terms of Service apply.

Trending Sources

article thumbnail

Supporting Diverse ML Systems at Netflix

The Netflix TechBlog

In addition to Spark, we want to support last-mile data processing in Python, addressing use cases such as feature transformations, batch inference, and training. Occasionally, these use cases involve terabytes of data, so we have to pay attention to performance.

Systems 226
article thumbnail

Your technology architecture and engineering organization should coevolve as your startup grows

Abhishek Tiwari

Basic monitoring and logging offered by PaaS will be sufficient for now. Skills: Induct Full-stack engineers. A few senior (2-3) engineers but mostly mid-level and junior engineers (7-8) eager to learn, improve, and master their craft. Introduce centralised logging and comprehensive monitoring. Caveats ?:

article thumbnail

Netflix at AWS re:Invent 2019

The Netflix TechBlog

December 2 1pm-2pm CMP 326-R Capacity Management Made Easy with Amazon EC2 Auto Scaling Vadim Filanovsky , Senior Performance Engineer & Anoop Kapoor, AWS Abstract :Amazon EC2 Auto Scaling offers a hands-free capacity management experience to help customers maintain a healthy fleet, improve application availability, and reduce costs.

AWS 100
article thumbnail

Netflix at AWS re:Invent 2019

The Netflix TechBlog

December 2 1pm-2pm CMP 326-R Capacity Management Made Easy with Amazon EC2 Auto Scaling Vadim Filanovsky , Senior Performance Engineer & Anoop Kapoor, AWS Abstract :Amazon EC2 Auto Scaling offers a hands-free capacity management experience to help customers maintain a healthy fleet, improve application availability, and reduce costs.

AWS 100
article thumbnail

Post: Fauna, Sisu, Educative, PA File Sight, Etleap, Triplebyte, Stream

High Scalability

Etleap is analyst-friendly , enterprise-grade ETL-as-a-service , built for Redshift and Snowflake data warehouses and S3/Glue data lakes. Our intuitive software allows data engineers to maintain pipelines without writing code, and lets analysts gain access to data in minutes instead of months.

Education 105