The Netflix TechBlog

NOVEMBER 22, 2019

by Shefali Vyas Dalal AWS re:Invent is a couple weeks away and our engineers & leaders are thrilled to be in attendance yet again this year! Netflix shares how Amazon EC2 Auto Scaling allows its infrastructure to automatically adapt to changing traffic patterns in order to keep its audience entertained and its costs on target.

AWS 100

AWS Entertainment Open Source Benchmarking 100

The Netflix TechBlog

NOVEMBER 22, 2019

by Shefali Vyas Dalal AWS re:Invent is a couple weeks away and our engineers & leaders are thrilled to be in attendance yet again this year! Netflix shares how Amazon EC2 Auto Scaling allows its infrastructure to automatically adapt to changing traffic patterns in order to keep its audience entertained and its costs on target.

AWS 100

AWS Entertainment Open Source Benchmarking 100

The Netflix TechBlog

MARCH 7, 2024

Berg , Romain Cledat , Kayla Seeley , Shashank Srikanth , Chaoying Wang , Darin Yu Netflix uses data science and machine learning across all facets of the company, powering a wide range of business applications from our internal infrastructure and content demand modeling to media understanding.

Systems 226

Systems Media Cache Open Source 226

The Netflix TechBlog

OCTOBER 21, 2019

By making consumption on-demand, you’re able to freely publish all of your data to Mantis. Mantis is Cost-Effective in Answering Questions Publishing 100% of your operational data so that you’re able to answer new questions in the future is traditionally cost prohibitive at scale. We should be able to access these events in realtime.

Open Source 101

Open Source Metrics Engineering Processing 101

The Netflix TechBlog

OCTOBER 21, 2019

By making consumption on-demand, you’re able to freely publish all of your data to Mantis. Mantis is Cost-Effective in Answering Questions Publishing 100% of your operational data so that you’re able to answer new questions in the future is traditionally cost prohibitive at scale. We should be able to access these events in realtime.

Open Source 100

Open Source Metrics Engineering Processing 100

The Netflix TechBlog

DECEMBER 17, 2019

Andreas Andreakis , Ioannis Papapanagiotou Overview Change-Data-Capture (CDC) allows capturing committed changes from a database in real-time and propagating those changes to downstream consumers [1][2]. There are several open source CDC projects, often using the same underlying libraries, database APIs, and protocols.

Database 212

Database Traffic Transportation Open Source 212

The Netflix TechBlog

DECEMBER 17, 2019

Andreas Andreakis , Ioannis Papapanagiotou Overview Change-Data-Capture (CDC) allows capturing committed changes from a database in real-time and propagating those changes to downstream consumers [1][2]. There are several open source CDC projects, often using the same underlying libraries, database APIs, and protocols.

Database 197

Database Traffic Transportation Open Source 197

Dynatrace

MAY 17, 2023

OpenTelemetry , the open source observability tool, has become the go-to standard for instrumenting custom applications to collect observability telemetry data. However, this method limited us to instrumenting the code manually and collecting specific sets of data we defined upfront. What is OneAgent?

Metrics 176

Metrics Monitoring Database Network 176

Dynatrace

JANUARY 28, 2021

Organizations that want a high-performance language with a great ecosystem for their applications often use Golang , an open-source programming language. Such additional telemetry data includes user-behavior analytics, code-level visibility, and metadata (including open-source data).

Traffic 170

Traffic Open Source Servers Cloud 170

Dynatrace

SEPTEMBER 17, 2019

Kubernetes (k8s) provides basic monitoring through the Kubernetes API and you can find instructions like Top 9 Open Source Tools for Monitoring Kubernetes as a “do it yourself guide”. Some vendors have just bolted on individual capabilities which often leads to six data silos. PostgreSQL & Elastic for data storage.

AWS 128

AWS Monitoring Ecommerce Lambda 128

The Netflix TechBlog

MARCH 5, 2019

the order of the rows on your Netflix home page, issuing content licenses when you click play, finding the Open Connect cache closest to you with the content you requested, and many more). All these micro-services are currently operated in AWS cloud infrastructure. In the Reliability space, our data teams focus on two main approaches.

Infrastructure 182

Infrastructure Cloud Scalability AWS 182

Percona

DECEMBER 21, 2023

Percona Monitoring and Management (PMM) is a state-of-the-art piece of software that exists in part thanks to great open source projects like VictoriaMetrics, PostgreSQL, and ClickHouse. This is made to avoid SSL certificates since everything goes through the same VPC (everything runs on ec2 instances on AWS).

Availability 96

Availability Monitoring Open Source Traffic 96

Dynatrace

NOVEMBER 22, 2022

For retail organizations, peak traffic can be a mixed blessing. While high-volume traffic often boosts sales, it can also compromise uptimes. Complicating the situation further, increasingly connected services are pushing more data processing to the edge. Gather observability data from all digital touchpoints.

Infrastructure 188

Infrastructure Availability Systems Retail 188

The Netflix TechBlog

APRIL 29, 2019

We use and contribute to many open-source Python packages, some of which are mentioned below. takes place in Amazon Web Services (AWS), whereas everything that happens afterwards (i.e., video streaming) takes place in the Open Connect network. If any of this interests you, check out the jobs site or find us at PyCon.

Open Source 279

Open Source Network Infrastructure Big Data 279

Adrian Cockcroft

FEBRUARY 9, 2023

The virtualization and networking platform could be datacenter based, with something like VMware, or cloud based using one of the cloud providers such as AWS EC2. Above that there’s a deployment platform such as Kubernetes or AWS Lambda. Netflix made many contributions to open source projects by it’s cloud platform team.

Engineering 137

Engineering Serverless Lambda AWS 137

The Netflix TechBlog

OCTOBER 19, 2020

Troubleshooting a session in Edgar When we started building Edgar four years ago, there were very few open-source distributed tracing systems that satisfied our needs. Our tactical approach was to use Netflix-specific libraries for collecting traces from Java-based streaming services until open source tracer libraries matured.

Infrastructure 233

Infrastructure Transportation Storage Open Source 233

Scalegrid

DECEMBER 16, 2019

MongoDB is the #3 open source database and the #1 NoSQL database in the world. So, what makes DigitalOcean a compelling alternative to other popular MongoDB hosting providers like AWS and Azure ? DigitalOcean Advantages for MongoDB. What’s most impressive is that you’re not compromising performance for cost.

Azure 187

Azure AWS Latency Database 187

Dynatrace

AUGUST 9, 2023

However, our customers often want to augment the data Dynatrace provides with data from third-party tools. Dynatrace Grail  is a data lakehouse that provides context-rich analytics capabilities for observability, security, and business data. To keep it real, we have a load generator that creates benign traffic.

Analytics 201

Analytics AWS Strategy Infrastructure 201

Percona

JULY 1, 2023

Now that Database-as-a-service (DBaaS) is in high demand, there are multiple questions regarding AWS services that cannot always be answered easily: When should I use Aurora and when should I use RDS MySQL ? Amazon Aurora is a proprietary, cloud-native, fully managed relational database service developed by Amazon Web Services (AWS).

AWS 52

AWS Database Serverless Storage 52

The Netflix TechBlog

AUGUST 13, 2020

This is a critical application so you drag yourself out of bed, open your laptop, and start poring through dashboards for more info. You’re not yet convinced there’s a real problem but you’re also aware that the clock is ticking as you dig through a mountain of data looking for clues. Regional traffic evacuations.

Monitoring 210

Monitoring Tuning Traffic Metrics 210

Percona

FEBRUARY 24, 2023

PMM can also show you historical data that can help you identify how long a service has been in a given state. PostgreSQL in the cloud CPU Considering AWS as your cloud platform of choice, the configuration made for your infrastructure will influence the performance of your application and monthly costs.

Cloud 115

Cloud Serverless AWS Storage 115

The Netflix TechBlog

MARCH 10, 2023

And more specifically, how we index and query over 7TB of data in a read-heavy and continuously growing environment and keep our Elasticsearch cluster healthy. You define the data types for each field, or use dynamic mapping for unknown fields. It was time to take a step back and reevaluate our ES data indexing and sharding strategy.

Strategy 258

Strategy Cache Analytics Storage 258

The Netflix TechBlog

NOVEMBER 22, 2019

by Shefali Vyas Dalal AWS re:Invent is a couple weeks away and our engineers & leaders are thrilled to be in attendance yet again this year! Netflix shares how Amazon EC2 Auto Scaling allows its infrastructure to automatically adapt to changing traffic patterns in order to keep its audience entertained and its costs on target.

AWS 37

AWS Entertainment Open Source Benchmarking 37

KeyCDN

FEBRUARY 22, 2023

With cloud-based infrastructure, organizations can easily scale their web applications to handle increased traffic or demand without the need for expensive hardware upgrades. Another benefit is cost savings associated with server and data center setup and maintenance.

Artificial Intelligence 111

Artificial Intelligence Development Serverless Website 111

Abhishek Tiwari

JULY 22, 2023

These policies encompass a wide range of concerns, including security best practices, data protection measures, industry regulations, privacy laws, accessibility requirements, and internal organizational policies. " Open Policy Agent (OPA) : OPA is an open-source policy as code framework that provides a general-purpose policy engine.

AWS 52

AWS Best Practices Infrastructure Traffic 52

Percona

APRIL 17, 2023

We’ll also go over some best practices for MongoDB security as well as MongoDB data modeling. The main objective of this post is to share my experience over the past years tuning MongoDB and centralize the diverse sources that I crossed in this journey in a unique place. For example: $ /opt/mongodb/4.0.6/bin/mongos

Best Practices 52

Best Practices Design Tuning Database 52

Percona

DECEMBER 5, 2023

It was a fascinating read, shedding light on leveraging AWS snapshots’ capabilities to avoid initial data copy within the RDS environment. Certainly, you can take advantage of this when you work with a large data set, and the initial copy could lead to long timeframes or network saturation. pg_basebackup -h 127.0.0.1 -p

Database 74

Database Traffic C++ Servers 74

All Things Distributed

APRIL 11, 2018

Last year at re:Invent, we launched AWS Fargate , and today, I want to explore how Fargate fundamentally changes the landscape of container technology. You should be able to write your code and have it run, without having to worry about configuring complex management tools, open source or not. Getting Started.

Cloud 148

Cloud AWS Innovation Traffic 148

Dynatrace

APRIL 30, 2019

A single indicator is defined as a query against a data source such as a monitoring, testing, security or code quality tool. Queries the results through the defined data sources (Prometheus & Dynatrace). Pitometer has been architected to be extensible for both data sources as well as graders.

Architecture 125

Architecture Open Source Azure Metrics 125

Percona

OCTOBER 19, 2023

This setup empowers your database to efficiently manage increased data volumes, enhance performance, and maintain availability. A security group in AWS for PostgreSQL with inbound rules allowing traffic on ports 5432 , 5433 , and 5434 for PostgreSQL. This tutorial will use Ubuntu 20.04 and Percona Distribution for PostgreSQL 15.

Scalability 81

Scalability Database Open Source Traffic 81

Smashing Magazine

AUGUST 3, 2023

It increases our visibility and enables us to draw a steady stream of organic (or “free”) traffic to our site. While paid marketing strategies like Google Ads play a part in our approach as well, enhancing our organic traffic remains a major priority. The higher our organic traffic, the more profitable we become as a company.

Performance 74

Performance Cache Traffic Metrics 74

All Things Distributed

MAY 19, 2017

Soon after the launch of Amazon Relation Dabase Service (RDS) AWS customers were giving us feedback that they would love to migrate to RDS but what they would love even more was if we could also unshackle them from the high-cost, punitive licensing schemes that came with the proprietary databases.

Database 117

Database Design Cloud Storage 117

The Netflix TechBlog

JULY 30, 2019

and beyond By Anoop Panicker and Kishore Banala Conductor is a workflow orchestration engine developed and open-sourced by Netflix. Our cluster size ranges from 12–18 instances of AWS EC2 m4.4xlarge instances, typically running at ~30% capacity. *?

Lambda 189

Lambda Media Open Source Metrics 189

High Scalability

APRIL 16, 2019

This scenario is more commonly known as split-brain scenario, and if not handled properly, can lead to more than one node claiming to be a master MySQL which in turn leads to data inconsistencies and corruption. Application traffic will be redirected to this new master MySQL node and the slave S2 will start replicating from the new master.

Availability 149

Availability Network Azure AWS 149

O'Reilly

AUGUST 8, 2023

At its peak, ChatGPT was in very exclusive company: it’s not quite on the level of Python, Kubernetes, and Java, but it’s in the mix with AWS and React, and significantly ahead of Docker. Langchain is frequently used to look up relevant articles that weren’t in ChatGPT’s training data and package them as part of a lengthy prompt.

Artificial Intelligence 71

Artificial Intelligence Programming Games Open Source 71

Percona

MAY 2, 2023

In today’s data-driven world, losing critical data can be catastrophic for any organization. As a MongoDB user, it’s crucial to ensure that your data is safe and secure in the event of a disaster or system failure. For organizations that rely on data to operate, database backups are critical for business continuity.

Best Practices 52

Best Practices Database Storage Servers 52

All Things Distributed

SEPTEMBER 6, 2013

This spring I travelled through Europe for the AWS Global Summit series. Europe is a continent with much diversity and for each country there are great AWS customer examples to tell. To reach those goals, Shell in 2010 began using AWS. These sensors create massive amounts of geological data.

Cloud 129

Cloud Energy AWS Healthcare 129