The Netflix TechBlog

MARCH 5, 2019

Netflix’s engineering culture is predicated on Freedom & Responsibility, the idea that everyone (and every team) at Netflix is entrusted with a core responsibility and they are free to operate with freedom to satisfy their mission. All these micro-services are currently operated in AWS cloud infrastructure.

Infrastructure 182

Infrastructure Cloud Scalability AWS 182

Dynatrace

MAY 3, 2024

Log data—the most verbose form of observability data, complementing other standardized signals like metrics and traces—is especially critical. As cloud complexity grows, it brings more volume, velocity, and variety of log data. When trying to address this challenge, your cloud architects will likely choose Amazon Data Firehose.

Cloud 240

Cloud Lambda AWS Analytics 240

The Netflix TechBlog

NOVEMBER 3, 2023

Democratizing Stream Processing @ Netflix By Guil Pires , Mark Cho , Mingliang Liu , Sujay Jain Data powers much of what we do at Netflix. On the Data Platform team, we build the infrastructure used across the company to process data at scale. The existing Data Mesh Processors have a lot of overlap with SQL.

Processing 265

Processing Engineering Infrastructure Latency 265

The Netflix TechBlog

OCTOBER 19, 2020

a Netflix member via Twitter This is an example of a question our on-call engineers need to answer to help resolve a member issue?—?which Now let’s look at how we designed the tracing infrastructure that powers Edgar. We needed to increase engineering productivity via distributed request tracing.

Infrastructure 233

Infrastructure Transportation Storage Open Source 233

The Netflix TechBlog

MARCH 25, 2019

Building and Scaling Data Lineage at Netflix to Improve Data Infrastructure Reliability, and Efficiency By: Di Lin , Girish Lingappa , Jitender Aswani Imagine yourself in the role of a data-inspired decision maker staring at a metric on a dashboard about to make a critical business decision but pausing to ask a question?—?“Can

Infrastructure 219

Infrastructure Big Data Transportation Architecture 219

Dynatrace

OCTOBER 28, 2021

Chaos engineering answers this need so organizations can deliver robust, resilient cloud-native applications that can stand up under any conditions. What is chaos engineering? Chaos engineers ask why. As chaos engineers grow confident in their testing, they change more variables and broaden the scope of the disaster.

Engineering 214

Engineering Entertainment Cloud Infrastructure 214

Dynatrace

APRIL 20, 2023

Modern organizations ingest petabytes of data daily, but legacy approaches to log analysis and management cannot accommodate this volume of data. At Dynatrace Perform 2023 , Maciej Pawlowski, senior director of product management for infrastructure monitoring at Dynatrace, and a senior software engineer at a U.K.

Analytics 177

Analytics Infrastructure Storage Efficiency 177

The Netflix TechBlog

FEBRUARY 1, 2022

By Karen Casella, Director of Engineering, Access & Identity Management Have you ever experienced one of the following scenarios while looking for your next role? Most backend engineering teams follow a process very similar to what is shown below. If so, we invite you to begin the interview process.

Engineering 199

Engineering Games Entertainment Innovation 199

Dynatrace

DECEMBER 8, 2022

As cloud-native, distributed architectures proliferate, the need for DevOps technologies and DevOps platform engineers has increased as well. DevOps engineer tools can help ease the pressure as environment complexity grows. ” What does a DevOps platform engineer do? .” What are DevOps engineer tools and platforms.

DevOps 180

DevOps Engineering Testing Open Source 180

The Netflix TechBlog

FEBRUARY 9, 2021

The Growth Engineering team is responsible for executing growth initiatives that help us anticipate and adapt to this change. For more background on Growth Engineering and the signup funnel, please have a look at our previous blog post that covers the basics. We need to be constantly adapting and innovating as a result of this change.

Engineering 142

Engineering Scalability Architecture Innovation 142

Dynatrace

JANUARY 23, 2024

AI data analysis can help development teams release software faster and at higher quality. So how can organizations ensure data quality, reliability, and freshness for AI-driven answers and insights? And how can they take advantage of AI without incurring skyrocketing costs to store, manage, and query data?

Performance 176

Performance DevOps Innovation Artificial Intelligence 176

DZone

OCTOBER 24, 2023

The ELK stack is an abbreviation for Elasticsearch, Logstash, and Kibana, which offers the following capabilities: Elasticsearch: a scalable search and analytics engine with a log analytics tool and application-formed database, perfect for data-driven applications.

Analytics 267

Analytics Storage Infrastructure Scalability 267

Dynatrace

MAY 1, 2023

IT operations analytics is the process of unifying, storing, and contextually analyzing operational data to understand the health of applications, infrastructure, and environments and streamline everyday operations. Here are the six steps of a typical ITOA process : Define the data infrastructure strategy.

Analytics 178

Analytics Artificial Intelligence Big Data Open Source 178

Dynatrace

MAY 17, 2022

By taking advantage of native Kubernetes standards, Dynatrace Cloud Native Full Stack injection empowers you to precisely provide the data that your teams need in exceptionally fast and automated ways. None of this complexity is exposed to application and infrastructure teams. Flexible automation for Kubernetes observability.

Availability 305

Availability Scalability Cloud Metrics 305

Dynatrace

NOVEMBER 3, 2023

that offers security, scalability, and simplicity of use. While Python code can address most data acquisition and ingest requirements, it comes at the cost of complexity in implementation and use-case modeling. supports SNMP, WMI, Prometheus, and SQL data sources, and now also JMX; no code is required to leverage these extensions.

Technology 236

Technology Technology Scalability Availability 236

Dynatrace

NOVEMBER 29, 2022

Infrastructure as code is a way to automate infrastructure provisioning and management. In this blog, I explore how Dynatrace has made cloud automation attainable—and repeatable—at scale by embracing the principles of infrastructure as code. Transparency and scalability. Infrastructure-as-code.

Infrastructure 164

Infrastructure Code Cloud DevOps 164

Dynatrace

DECEMBER 18, 2023

For busy site reliability engineers, ensuring system reliability, scalability, and overall health is an imperative that’s getting harder to achieve in ever-expanding, cloud-native, container-based environments. To get a more granular look into telemetry data, many analysts rely on custom metrics using Prometheus.

Metrics 169

Metrics Engineering Energy Tuning 169

The Netflix TechBlog

MARCH 7, 2024

Berg , Romain Cledat , Kayla Seeley , Shashank Srikanth , Chaoying Wang , Darin Yu Netflix uses data science and machine learning across all facets of the company, powering a wide range of business applications from our internal infrastructure and content demand modeling to media understanding.

Systems 226

Systems Media Cache Open Source 226

Dynatrace

OCTOBER 4, 2022

Some time ago, at a restaurant near Boston, three Dynatrace colleagues dined and discussed the growing data challenge for enterprises. At its core, this challenge involves a rapid increase in the amount—and complexity—of data collected within a company. Work with different and independent data types. Thus, Grail was born.

Artificial Intelligence 257

Artificial Intelligence Analytics Storage Architecture 257

The Netflix TechBlog

OCTOBER 27, 2020

By Tianlong Chen and Ioannis Papapanagiotou Netflix has more than 195 million subscribers that generate petabytes of data everyday. Data scientists and engineers collect this data from our subscribers and videos, and implement data analytics models to discover customer behaviour with the goal of maximizing user joy.

Latency 243

Latency Storage Big Data Tuning 243

Dynatrace

JUNE 10, 2024

Security analytics combines data collection, aggregation, and analysis to search for and identify potential threats. Using a combination of historical data and information collected in real time, security teams can detect threats earlier in the SDLC. Infrastructure type In most cases, legacy SIEM tools are on-premises.

Analytics 220

Analytics Network Hardware Open Source 220

Scalegrid

MAY 13, 2020

It can scale towards a multi-petabyte level data workload without a single issue, and it allows access to a cluster of powerful servers that will work together within a single SQL interface where you can view all of the data. This feature-packed database provides powerful and rapid analytics on data that scales up to petabyte volumes.

Big Data 321

Big Data Database Artificial Intelligence Open Source 321

DZone

NOVEMBER 27, 2023

An open-source distributed SQL query engine, Trino is widely used for data analytics on distributed data storage. Optimizing Trino to make it faster can help organizations achieve quicker insights and better user experiences, as well as cut costs and improve infrastructure efficiency and scalability.

Tuning 162

Tuning Speed Performance Open Source 162

Dynatrace

MAY 28, 2024

Logs provide answers, but monitoring is a challenge Manual tagging is error-prone Making sure your required logs are monitored is a task distributed between the data owner and the monitoring administrator. Finding the right logs is cumbersome Even if your logs are monitored, you need to make sense of the vast data volume.

Analytics 182

Analytics Infrastructure Database Monitoring 182

Scalegrid

MARCH 14, 2024

This approach allows companies to combine the security and control of private clouds with public clouds’ scalability and innovation potential. This combination allows for the fluid movement of data and applications across different environments, facilitating shared workloads seamlessly.

Strategy 130

Strategy Cloud Artificial Intelligence Infrastructure 130

Dynatrace

MARCH 18, 2024

Following FinOps practices, engineering, finance, and business teams take responsibility for their cloud usage, making data-driven spending decisions in a scalable and sustainable manner. Empowering teams to manage their FinOps practices, however, requires teams to have access to reliable multicloud monitoring and analysis data.

Best Practices 221

Best Practices Strategy Cloud AWS 221

The Netflix TechBlog

DECEMBER 3, 2019

Netflix applies data science to hundreds of use cases across the company, including optimizing content delivery and video encoding. Data scientists at Netflix relish our culture that empowers them to work autonomously and use their judgment to solve problems independently. How could we improve the quality of life for data scientists?

Open Source 232

Open Source AWS Infrastructure Energy 232

The Netflix TechBlog

DECEMBER 21, 2020

By Anupom Syam Background At Netflix, our current data warehouse contains hundreds of Petabytes of data stored in AWS S3 , and each day we ingest and create additional Petabytes. We built AutoOptimize to efficiently and transparently optimize the data and metadata storage layout while maximizing their cost and performance benefits.

Storage 203

Storage Latency Efficiency Data Engineering 203

The Netflix TechBlog

MARCH 7, 2019

To unlock these innovations we are making a strategic choice that our focus should be geared towards developing the surrounding infrastructure so that scientists’ work can be easily absorbed into the wider Netflix Experimentation Platform. Composition Data Science is a curiosity driven field, and should not be unnecessarily constrained [ 1 ].

Design 174

Design Engineering Innovation Java 174

High Scalability

FEBRUARY 19, 2021

This is a guest post by Paddy Byers , Co-founder and CTO at Ably , a realtime data delivery platform. This means a system that is not merely available but is also engineered with extensive redundant measures to continue to work as its users expect. You can view the original article on Ably's blog. This ensures reliability.

Engineering 215

Engineering Systems Scalability Availability 215

Dynatrace

AUGUST 30, 2023

GKE Autopilot empowers organizations to invest in creating elegant digital experiences for their customers in lieu of expensive infrastructure management. Dynatrace’s collaboration with Google addresses these needs by providing simple, scalable, and innovative data acquisition for comprehensive analysis and troubleshooting.

Google 208

Google Cloud Innovation Infrastructure 208

Dynatrace

DECEMBER 14, 2023

This innovative solution combines Snyk Container and Dynatrace observability data to provide comprehensive reporting—highlighting which running containers have undergone Snyk Container scans. Dynatrace provides powerful AI-based observability, putting all your infrastructure, applications, and events in context.

AWS 250

AWS Innovation Speed Efficiency 250

The Netflix TechBlog

JUNE 7, 2021

At much less than 1% of CPU and memory on the instance, this highly performant sidecar provides flow data at scale for network insight. Challenges The cloud network infrastructure that Netflix utilizes today consists of AWS services such as VPC, DirectConnect, VPC Peering, Transit Gateways, NAT Gateways, etc and Netflix owned devices.

Network 325

Network Transportation AWS Cloud 325

The Netflix TechBlog

JUNE 6, 2022

Our customers are product and engineering teams at Netflix that build these software services and platforms. Our goal is to manage security risks to Netflix via clear, opinionated security guidance, and by providing risk context to Netflix engineering teams to make pragmatic risk decisions at scale.

Software Engineering 169

Software Engineering Scalability Education Engineering 169

Dynatrace

APRIL 7, 2023

Streaming raises the default 6 MB hard limit to a 20 MB soft limit, adding greater scalability and flexibility to their applications. Despite being serverless, the function still requires infrastructure on which to run. What is a Lambda serverless function?

Lambda 204

Lambda AWS Serverless Latency 204

The Netflix TechBlog

FEBRUARY 16, 2021

Membership Engineering at Netflix is responsible for the plan and pricing configurations for every market worldwide. To solve the challenges mentioned above and meet our rapidly evolving business needs, we re-architected the legacy SKU catalog from the ground up and partnered with the Growth Engineering team to build a scalable SKU platform.

Mobile 209

Mobile Engineering Infrastructure Scalability 209

Dynatrace

MARCH 24, 2023

Containers enable developers to package microservices or applications with the libraries, configuration files, and dependencies needed to run on any infrastructure, regardless of the target system environment. Originally created by Google, Kubernetes was donated to the CNCF as an open source project.

Infrastructure 176

Infrastructure Open Source Operating System Cloud 176