Remove AWS Remove Big Data Remove Data Remove Systems
article thumbnail

Auto-Diagnosis and Remediation in Netflix Data Platform

The Netflix TechBlog

By Vikram Srivastava and Marcelo Mayworm Netflix has one of the most complex data platforms in the cloud on which our data scientists and engineers run batch and streaming workloads. As our subscribers grow worldwide and Netflix enters the world of gaming , the number of batch workflows and real-time data pipelines increases rapidly.

Big Data 238
article thumbnail

Delta: A Data Synchronization and Enrichment Platform

The Netflix TechBlog

Part I: Overview Andreas Andreakis , Falguni Jhaveri , Ioannis Papapanagiotou , Mark Cho , Poorna Reddy , Tongliang Liu Overview It is a commonly observed pattern for applications to utilize multiple datastores where each is used to serve a specific need such as storing the canonical form of data (MySQL etc.), caching (Memcached etc.),

Insiders

Sign Up for our Newsletter

This site is protected by reCAPTCHA and the Google Privacy Policy and Terms of Service apply.

Trending Sources

article thumbnail

Driving down the cost of Big-Data analytics - All Things Distributed

All Things Distributed

Werner Vogels weblog on building scalable and robust distributed systems. Driving down the cost of Big-Data analytics. The Amazon Elastic MapReduce (EMR) team announced today the ability to seamlessly use Amazon EC2 Spot Instances with their service, significantly driving down the cost of data analytics in the cloud.

Big Data 111
article thumbnail

Kubernetes in the wild report 2023

Dynatrace

The study analyzes factual Kubernetes production data from thousands of organizations worldwide that are using the Dynatrace Software Intelligence Platform to keep their Kubernetes clusters secure, healthy, and high performing. Kubernetes is emerging as the “operating system” of the cloud. Kubernetes moved to the cloud in 2022.

article thumbnail

How Netflix uses eBPF flow logs at scale for network insight

The Netflix TechBlog

At much less than 1% of CPU and memory on the instance, this highly performant sidecar provides flow data at scale for network insight. Challenges The cloud network infrastructure that Netflix utilizes today consists of AWS services such as VPC, DirectConnect, VPC Peering, Transit Gateways, NAT Gateways, etc and Netflix owned devices.

Network 325
article thumbnail

Allez, rendez-vous à Paris – An AWS Region is coming to France!

All Things Distributed

Today, I am very excited to announce our plans to open a new AWS Region in France! Over the past 10 years, we have seen tremendous growth at AWS. As a result, we have opened 35 Availability Zones (AZs), across 13 AWS Regions worldwide. As a result, we have opened 35 Availability Zones (AZs), across 13 AWS Regions worldwide.

AWS 166
article thumbnail

Expanding the Cloud – An AWS Region is coming to Hong Kong

All Things Distributed

Today, I am very excited to announce our plans to open a new AWS Region in Hong Kong! The new region will give Hong Kong-based businesses, government organizations, non-profits, and global companies with customers in Hong Kong, the ability to leverage AWS technologies from data centers in Hong Kong.

AWS 146