The Netflix TechBlog

NOVEMBER 14, 2023

By Abhinaya Shetty , Bharath Mummadisetty At Netflix, our Membership and Finance Data Engineering team harnesses diverse data related to plans, pricing, membership life cycle, and revenue to fuel analytics, power various dashboards, and make data-informed decisions. We expect complete and accurate data at the end of each run.

Data Engineering 264

Data Engineering Engineering Processing Games 264

Dynatrace

JANUARY 23, 2024

AI data analysis can help development teams release software faster and at higher quality. AI-enabled chatbots can help service teams triage customer issues more efficiently. So how can organizations ensure data quality, reliability, and freshness for AI-driven answers and insights?

Performance 176

Performance DevOps Innovation Artificial Intelligence 176

The Netflix TechBlog

MARCH 4, 2024

Operational automation–including but not limited to, auto diagnosis, auto remediation, auto configuration, auto tuning, auto scaling, auto debugging, and auto testing–is key to the success of modern data platforms. In this way, no human intervention is required in the remediation process. Multi-objective optimizations.

Tuning 210

Tuning Efficiency Big Data Engineering 210

The Netflix TechBlog

JULY 26, 2021

This happens at an unprecedented scale and introduces many interesting challenges; one of the challenges is how to provide visibility of Studio data across multiple phases and systems to facilitate operational excellence and empower decision making. With the latest Data Mesh Platform, data movement in Netflix Studio reaches a new stage.

Big Data 253

Big Data Government Analytics Processing 253

Dynatrace

JANUARY 26, 2021

This allows teams to sidestep much of the cost and time associated with managing hardware, platforms, and operating systems on-premises, while also gaining the flexibility to scale rapidly and efficiently. REST APIs, authentication, databases, email, and video processing all have a home on serverless platforms. The Serverless Process.

Serverless 239

Serverless Efficiency Lambda Azure 239

Dynatrace

OCTOBER 4, 2022

Some time ago, at a restaurant near Boston, three Dynatrace colleagues dined and discussed the growing data challenge for enterprises. At its core, this challenge involves a rapid increase in the amount—and complexity—of data collected within a company. Work with different and independent data types. Thus, Grail was born.

Artificial Intelligence 257

Artificial Intelligence Analytics Storage Architecture 257

Scalegrid

MAY 13, 2020

Greenplum Database is a massively parallel processing (MPP) SQL database that is built and based on PostgreSQL. It can scale towards a multi-petabyte level data workload without a single issue, and it allows access to a cluster of powerful servers that will work together within a single SQL interface where you can view all of the data.

Big Data 321

Big Data Database Artificial Intelligence Open Source 321

Dynatrace

SEPTEMBER 14, 2022

As an industry leader, Dynatrace promotes primarily using software and AI to deal with this complexity at scale instead of just putting data on dashboards. Does that mean that reactive and exploratory data analysis, often done manually and with the help of dashboards, are dead? Why today’s data analytics solutions still fail us.

Metrics 217

Metrics Analytics Monitoring Efficiency 217

Dynatrace

OCTOBER 6, 2023

Grail: Enterprise-ready data lakehouse Grail, the Dynatrace causational data lakehouse, was explicitly designed for observability and security data, with artificial intelligence integrated into its foundation. Tables are a physical data model, essentially the type of observability data that you can store.

Artificial Intelligence 252

Artificial Intelligence Analytics Metrics Storage 252

The Netflix TechBlog

DECEMBER 21, 2020

By Anupom Syam Background At Netflix, our current data warehouse contains hundreds of Petabytes of data stored in AWS S3 , and each day we ingest and create additional Petabytes. On the other hand, these optimizations themselves need to be sufficiently inexpensive to justify their own processing cost over the gains they bring.

Storage 203

Storage Latency Efficiency Data Engineering 203

Dynatrace

FEBRUARY 15, 2023

Grail – the foundation of exploratory analytics Grail can already store and process log and business events. Now we’re adding Smartscape to DQL and two new data sources to Grail: Metrics on Grail and Traces on Grail. Ensuring observability across these environments requires access to data at a massive scale.

Analytics 218

Analytics Social Media Metrics Efficiency 218

Dynatrace

MAY 7, 2024

Fluent Bit is a telemetry agent designed to receive data (logs, traces, and metrics), process or modify it, and export it to a destination. Fluent Bit can serve as a proxy before you send data to Dynatrace or similar. However, you can also use Fluent Bit as a processor because you can perform various actions on the data.

Best Practices 231

Best Practices IoT Metrics Storage 231

The Netflix TechBlog

OCTOBER 18, 2022

by Jun He , Akash Dwivedi , Natallia Dzenisenka , Snehal Chennuru , Praneeth Yenugutala , Pawan Dixit At Netflix, Data and Machine Learning (ML) pipelines are widely used and have become central for the business, representing diverse use cases that go beyond recommendations, predictions and data transformations.

Java 202

Java Scalability Traffic Architecture 202

The Netflix TechBlog

MARCH 25, 2019

Building and Scaling Data Lineage at Netflix to Improve Data Infrastructure Reliability, and Efficiency By: Di Lin , Girish Lingappa , Jitender Aswani Imagine yourself in the role of a data-inspired decision maker staring at a metric on a dashboard about to make a critical business decision but pausing to ask a question?—?“Can

Infrastructure 219

Infrastructure Big Data Transportation Architecture 219

The Netflix TechBlog

OCTOBER 28, 2021

Data Engineers of Netflix?—?Interview Interview with Pallavi Phadnis This post is part of our “ Data Engineers of Netflix ” series, where our very own data engineers talk about their journeys to Data Engineering @ Netflix. Pallavi Phadnis is a Senior Software Engineer on the Product Data Science and Engineering team.

Data Engineering 173

Data Engineering Engineering Big Data Software Engineering 173

Dynatrace

SEPTEMBER 7, 2022

Dynatrace and the Dynatrace Intelligent Observability Platform have added support for the newly introduced Amazon VPC Flow Logs to Amazon Kinesis Data Firehose. This support enables customers to define specific endpoint delivery of real-time streaming data to platforms such as Dynatrace. What is VPC Flow Logs? Why Dynatrace?

Traffic 204

Traffic AWS Network Cloud 204

Dynatrace

APRIL 8, 2024

This blog explores how vertically integrated risk management solutions that use AI and automation enable unparalleled visibility, control, and efficiency for risk management in banking. Optimize the IT infrastructure supporting risk management processes and controls for maximum performance and resilience.

Analytics 189

Analytics Infrastructure Efficiency Technology 189

Dynatrace

NOVEMBER 20, 2023

The first goal is to demonstrate how generative AI can bring key business value and efficiency for organizations. While technologies have enabled new productivity and efficiencies, customer expectations have grown exponentially, cyberthreat risks continue to mount, and the pace of business has sped up.

AWS 200

AWS Artificial Intelligence Analytics Cloud 200

Dynatrace

MARCH 29, 2024

In today’s rapidly evolving landscape, incorporating AI innovation into business strategies is vital, enabling organizations to optimize operations, enhance decision-making processes, and stay competitive. AI innovation elevates efficiency and performance of Google Cloud AI adoption is increasingly critical for any organization.

Google 250

Google Innovation Cloud Analytics 250

The Netflix TechBlog

NOVEMBER 14, 2023

By Abhinaya Shetty , Bharath Mummadisetty This blog post will cover how Psyberg helps automate the end-to-end catchup of different pipelines, including dimension tables. In the previous installments of this series, we introduced Psyberg and delved into its core operational modes: Stateless and Stateful Data Processing.

Tuning 244

Tuning Processing C++ Efficiency 244

The Netflix TechBlog

JUNE 25, 2021

By Gim Mahasintunan on behalf of Data Platform Engineering. Supporting a rapidly growing base of engineers of varied backgrounds using different data stores can be challenging in any organization. In this blog post, we are thrilled to share that we are open-sourcing one such tool: the Netflix Data Explorer.

Metrics 152

Metrics Best Practices Strategy Design 152

Dynatrace

FEBRUARY 28, 2024

In the previous blog post of this series , we discussed the crucial role of Dynatrace as an orchestrator that steps in to stop the testing phase in case of any errors. In this blog post of the series, we will explore the use of Site Reliability Guardian (SRG) in more detail.

DevOps 193

DevOps Metrics Engineering Analytics 193

Scalegrid

JULY 17, 2020

It is an open standard format which organizes data into key/value pairs and arrays detailed in RFC 7159. JSON is the most common format used by web services to exchange data, store documents, unstructured data, etc. You can also check out our Working with JSON Data in PostgreSQL vs. JSONB Patterns & Antipatterns.

Storage 321

Storage Database Efficiency Processing 321

The Netflix TechBlog

DECEMBER 17, 2019

Andreas Andreakis , Ioannis Papapanagiotou Overview Change-Data-Capture (CDC) allows capturing committed changes from a database in real-time and propagating those changes to downstream consumers [1][2]. This motivated the development of DBLog , which offers log and dump processing under a generic framework.

Database 212

Database Traffic Transportation Open Source 212

Dynatrace

MAY 2, 2024

With the ability to generate new content—such as images, text, audio, and other data—based on patterns and examples taken from existing data, organizations are rushing to capitalize on the AI model. As organizations train generative AI systems with critical data, they must be aware of the security and compliance risks.

Artificial Intelligence 218

Artificial Intelligence Cloud Best Practices DevOps 218

Dynatrace

MAY 23, 2024

Dynatrace ® AppEngine features a no- and low-code toolset and leverages Davis AI to empower teams to easily create and share custom, intelligent, and secure apps that leverage insights from data generated by their clouds. The following figure shows the benefits of Azure Native Dynatrace Service.

Azure 213

Azure Cloud Metrics Innovation 213

The Netflix TechBlog

DECEMBER 17, 2019

Andreas Andreakis , Ioannis Papapanagiotou Overview Change-Data-Capture (CDC) allows capturing committed changes from a database in real-time and propagating those changes to downstream consumers [1][2]. This motivated the development of DBLog , which offers log and dump processing under a generic framework.

Database 197

Database Traffic Transportation Open Source 197

Dynatrace

NOVEMBER 28, 2023

In the previous installment of this blog series , we explored how to set up Dynatrace as a build-stage orchestrator to effectively address the challenges faced by Site Reliability Engineers (SREs). Davis AI efficiently identified the deployment change as the potential root cause for the malfunctioning of nginx.

Traffic 252

Traffic Best Practices Strategy Engineering 252

Official Blog - World Web Technology

DECEMBER 4, 2023

How does it contribute to the software development process ? This article will explain why most companies and their software developers use SDLC to manage their development process. SDLC is an essential framework for the total software development process as it aids in managing software development projects effectively.

Processing 64

Processing Software Software Development 64

Dynatrace

FEBRUARY 1, 2024

The adoption process takes time and consideration. The complexity and numerous moving parts of Kubernetes multicloud clusters mean that when monitoring the health of these clusters—which is critical for ensuring reliable and efficient operation of the application—platform engineers often find themselves without an easy and efficient solution.

Engineering 252

Engineering Efficiency Azure Monitoring 252

The Netflix TechBlog

JUNE 13, 2023

Our previous blog post presented replay traffic testing — a crucial instrument in our toolkit that allows us to implement these transformations with precision and reliability. A process that doesn’t just minimize risk, but also facilitates a continuous evaluation of the rollout’s impact.

Traffic 279

Traffic Metrics Systems Strategy 279

Dynatrace

APRIL 19, 2024

In this blog post, we explore real-life scenarios shared by our customers, partners, and internal users at Dynatrace, offering inspiration to kickstart your own journey with custom solutions powered by AppEngine. Using AppEngine as the foundation, they fed disk utilization data into Davis ® AI to forecast the required resizing operation.

AWS 209

AWS Analytics Cloud Monitoring 209

Dynatrace

AUGUST 8, 2019

This short blog post is a follow up to the announcement we made on May 6, 2019: OneAgent for Windows—Enhancements to *.msi-based The installation (and automatic update) of OneAgent is an important part of the delivery process. For the installation process itself, there are additional space requirements for Windows.

Efficiency 116

Efficiency Network Storage Architecture 116

Percona

MAY 1, 2023

This blog was originally published in August 2018 and was updated in May 2023. In this blog post, we will delve into the world of PostgreSQL performance tuning and establish some basic guidelines for setting database parameters to enhance the performance of your PostgreSQL database, based on the workload. Nobody has time for that!

Tuning 52

Tuning Efficiency Database Performance 52

Dynatrace

OCTOBER 10, 2023

In this blog post, we’ll delve deeper into these categories to gain a comprehensive understanding of their significance and the challenges they present. Process Improvements (50%) The allocation for process improvements is devoted to automation and continuous improvement SREs help to ensure that systems are scalable, reliable, and efficient.

Engineering 215

Engineering DevOps Innovation Strategy 215

Percona

AUGUST 9, 2023

MySQL 8 brought a significant architectural transformation by replacing the traditional MyISAM-based system tables with the Transaction Data Dictionary (TDD), a more efficient and reliable approach. the mysql system database was represented by a subdirectory named ‘mysql’ in the data directory. In MySQL 5.7,

Database 77

Database Storage Scalability Games 77

Dynatrace

SEPTEMBER 13, 2023

This is a set of best practices and guidelines that help you design and operate reliable, secure, efficient, cost-effective, and sustainable systems in the cloud. The framework comprises six pillars: Operational Excellence, Security, Reliability, Performance Efficiency, Cost Optimization, and Sustainability.

AWS 236

AWS Efficiency Azure Cloud 236