The Netflix TechBlog

OCTOBER 19, 2020

Now let’s look at how we designed the tracing infrastructure that powers Edgar. If we had an ID for each streaming session then distributed tracing could easily reconstruct session failure by providing service topology, retry and error tags, and latency measurements for all service calls.

Infrastructure 233

Infrastructure Transportation Storage Open Source 233

Scalegrid

JANUARY 8, 2024

Infrastructure Excellence ScaleGrid’s infrastructure is designed to facilitate hosting in your cloud account and provides cost-saving options with AWS or Azure Reserved Instances or GCP. This results in faster response times and reduced network traffic, enhancing the overall efficiency and effectiveness of cloud services.

Artificial Intelligence 147

Artificial Intelligence Cloud Scalability Analytics 147

The Netflix TechBlog

NOVEMBER 22, 2019

by Shefali Vyas Dalal AWS re:Invent is a couple weeks away and our engineers & leaders are thrilled to be in attendance yet again this year! Netflix shares how Amazon EC2 Auto Scaling allows its infrastructure to automatically adapt to changing traffic patterns in order to keep its audience entertained and its costs on target.

AWS 37

AWS Entertainment Open Source Benchmarking 37

All Things Distributed

NOVEMBER 26, 2013

As I discussed in my re:Invent keynote earlier this month, I am now happy to announce the immediate availability of Amazon RDS Cross Region Read Replicas , which is another important enhancement for our customers using or planning to use multiple AWS Regions to deploy their applications. Cross Region Read Replicas are available for MySQL 5.6

Cloud 145

Cloud AWS Traffic Latency 145

InfoQ

JUNE 16, 2023

The event-driven architecture (EDA) can accommodate bursts in traffic in the multi-tenant platform with reasonable ingestion latency and acceptable operational costs. Datadog created a dedicated data ingestion architecture offering exactly-once semantics for their third-generation event store, Husky. By Rafal Gancarz

Architecture 55

Architecture Scalability Latency Traffic 55

All Things Distributed

AUGUST 16, 2011

Expanding the Cloud - The AWS GovCloud (US) Region. Today AWS announced the launch of the AWS GovCloud (US) Region. The concept of regions gives AWS customers control over the placement of their resources and services. The Cloud First strategy is most visible with new Federal IT programs, which are all designed to be â??Cloud

AWS 111

AWS Government Big Data Cloud 111

Percona

DECEMBER 11, 2023

A quick configuration change may do the trick in improving the performance of your AWS RDS for MySQL instance. A Dedicated Log Volume (DLV) is a specialized storage volume designed to house database transaction logs separately from the volume containing the database tables. and later v10 versions MySQL: 8.0.28

AWS 92

AWS Benchmarking Performance Traffic 92

The Netflix TechBlog

NOVEMBER 9, 2022

to a larger AWS instance size, from m5.4xl (16 vCPUs) to m5.12xl (48 vCPUs). A quick canary test was free of errors and showed lower latency, which is expected given that our standard canary setup routes an equal amount of traffic to both the baseline running on 4xl and the canary on 12xl. let’s call it GS2?—?to

Hardware 363

Hardware Cache Performance Latency 363

VoltDB

JUNE 1, 2017

An opening scene involving a traffic jam of Viking boats and a musical number (“Love Can’t Afjord to wait”). These systems were designed to have a lifetime of half a decade or more, and rapidly changing hardware meant that the initial deployment had to be sized for 5-7 years out. Vikings fight zombies. With transformer Viking ships.

Hardware 56

Hardware Latency Virtualization Cloud 56

VoltDB

JUNE 1, 2017

An opening scene involving a traffic jam of Viking boats and a musical number (“Love Can’t Afjord to wait”). These systems were designed to have a lifetime of half a decade or more, and rapidly changing hardware meant that the initial deployment had to be sized for 5-7 years out. Vikings fight zombies. With transformer Viking ships.

Hardware 40

Hardware Latency Virtualization Cloud 40

All Things Distributed

MARCH 26, 2018

At AWS, we don't mark many anniversaries. A concept that has changed infrastructure architecture is now at the core of both AWS and customer reliability and operations. Powering the virtual instances and other resources that make up the AWS Cloud are real physical data centers with AWS servers in them.

AWS 132

AWS Latency Lambda Architecture 132

Scalegrid

DECEMBER 16, 2019

While you may assume a great majority of the cloud database deployments are run on AWS, Azure, or Google Cloud Platform, small to medium-sized businesses in particular are gravitating towards the developer-friendly cloud provider, DigitalOcean , for their hosting for MongoDB® needs. DigitalOcean Advantages for MongoDB. DigitalOcean Droplets.

Azure 187

Azure AWS Latency Database 187

The Netflix TechBlog

FEBRUARY 18, 2022

We thus assigned a priority to each use case and sharded event traffic by routing to priority-specific queues and the corresponding event processing clusters. This separation allows us to tune system configuration and scaling policies independently for different event priorities and traffic patterns.

Systems 334

Systems Traffic Architecture Mobile 334

The Netflix TechBlog

NOVEMBER 3, 2022

In that scenario, the system would need to deal with the data propagation latency directly, for example, by use of timeouts or client-originated update tracking mechanisms. With traffic growth, a single leader node handling all request volume started becoming overloaded. The cache is kept in sync with the current leader process.

Cache 224

Cache Latency Traffic Systems 224

The Netflix TechBlog

AUGUST 16, 2023

By Arthur Gonigberg , Argha C Plaintext Past When Zuul was designed and developed , there was an inherent assumption that connections were effectively free, given we weren’t using mutual TLS (mTLS). That’s a significant amount and certainly more than is necessary relative to the traffic on most clusters.

Traffic 170

Traffic Servers Google Metrics 170

The Netflix TechBlog

AUGUST 29, 2023

A brief history of IPC at Netflix Netflix was early to the cloud, particularly for large-scale companies: we began the migration in 2008, and by 2010, Netflix streaming was fully run on AWS. Today we have a wealth of tools, both OSS and commercial, all designed for cloud-native environments.

Traffic 220

Traffic Latency Cloud C++ 220

Percona

JULY 1, 2023

Now that Database-as-a-service (DBaaS) is in high demand, there are multiple questions regarding AWS services that cannot always be answered easily: When should I use Aurora and when should I use RDS MySQL ? Amazon Aurora is a proprietary, cloud-native, fully managed relational database service developed by Amazon Web Services (AWS).

AWS 52

AWS Database Serverless Storage 52

Smashing Magazine

AUGUST 3, 2023

These pages serve as a pivotal tool in our digital marketing strategy, not only providing valuable information about our services but also designed to be easily discoverable through search engines. It increases our visibility and enables us to draw a steady stream of organic (or “free”) traffic to our site. SEO is key to our success.

Performance 74

Performance Cache Traffic Metrics 74

Percona

APRIL 17, 2023

The CFQ works well for many general use cases but lacks latency guarantees. The deadline excels at latency-sensitive use cases ( like databases ), and noop is closer to no schedule at all. On the other hand, MongoDB schema design takes a document-oriented approach. Two other schedulers are deadline and noop.

Best Practices 52

Best Practices Design Tuning Database 52

High Scalability

JULY 20, 2018

DonHopkins : NeWS differs from the current technology stack in that it was all coherently designed at once by James Gosling and David Rosenthal, by taking several steps back and thinking deeply about all the different problems it was trying to solve together. Some are lucky enough to also have a production environment."

Internet 121

Internet Internet Scalability Automotive 121

The Netflix TechBlog

FEBRUARY 9, 2021

Before designing a solution it’s important to understand the main product requirements for such a feature: The content needs to be new, relevant, and regional (not all countries have the same catalogue). To reduce latency, assets should be generated in an offline fashion and not in real time.

Engineering 165

Engineering Storage Latency Entertainment 165

All Things Distributed

MARCH 11, 2016

The epoch of AWS is the launch of Amazon S3 on March 14, 2006, now almost 10 years ago. Given that AWS is a pioneer in building and operating these services world-wide, these lessons have been of crucial importance to our business. AWS helps its customers do this too. your resource usage. Build security in from the ground up.

AWS 144

AWS Hardware Retail Virtualization 144

The Netflix TechBlog

DECEMBER 17, 2019

Nonetheless, we found a number of limitations that could not satisfy our requirements e.g. stalling the processing of log events until a dump is complete, missing ability to trigger dumps on demand, or implementations that block write traffic by using table locks. Designed with High Availability in mind. Writing events to any output.

Database 197

Database Traffic Transportation Open Source 197

The Netflix TechBlog

DECEMBER 17, 2019

Nonetheless, we found a number of limitations that could not satisfy our requirements e.g. stalling the processing of log events until a dump is complete, missing ability to trigger dumps on demand, or implementations that block write traffic by using table locks. Designed with High Availability in mind. Writing events to any output.

Database 212

Database Traffic Transportation Open Source 212

All Things Distributed

OCTOBER 2, 2017

With these requirements in mind, and a willingness to question the status quo, a small group of distributed systems experts came together and designed a horizontally scalable distributed database that would scale out for both reads and writes to meet the long-term needs of our business. This was the genesis of the Amazon Dynamo database.

Internet 128

Internet Internet AWS Performance 128

All Things Distributed

JUNE 21, 2018

Further, with the growth and scale of Amazon.com, boundless horizontal scale needed to be a key design point--scaling up simply wasn't an option. In fact, this is been proven by our customers as Amazon Aurora remains the fastest growing service in AWS history. The opposite is true. Take Expedia, for example.

Database 167

Database AWS Games Latency 167

Scalegrid

JANUARY 12, 2024

High availability storage options within the context of cloud computing involve highly adaptable storage solutions specifically designed for storing vast amounts of data while providing easy access to it. Storage is a critical aspect to consider when working with cloud workloads.

Cloud 130

Cloud Virtualization Storage Efficiency 130

All Things Distributed

JANUARY 18, 2012

a Fast and Scalable NoSQL Database Service Designed for Internet Scale Applications. Today is a very exciting day as we release Amazon DynamoDB , a fast, highly reliable and cost-effective NoSQL database service designed for internet scale applications. Amazon DynamoDB offers low, predictable latencies at any scale. Comments ().

Scalability 167

Scalability Database Ecommerce Latency 167

Percona

MAY 2, 2023

Percona Backup for MongoDB is an uncomplicated command-line tool by design that lends itself well to backing up larger data sets. Especially if going into or out of storage types that may throttle bandwidth/network traffic. It also supports storing the backups on filesystem and AWS S3 buckets. 8 connecting to: mongodb://127.0.0.1:27017/?compressors=disabled&gssapiServiceName=mongodb

Best Practices 52

Best Practices Database Storage Servers 52

All Things Distributed

NOVEMBER 21, 2017

Redis's microsecond latency has made it a de facto choice for caching. Four years ago, as part of our AWS fast data journey, we introduced Amazon ElastiCache for Redis , a fully managed, in-memory data store that operates at microsecond latency. TB of in-memory capacity in a single cluster.

Games 112

Games Retail Latency Education 112

Adrian Cockcroft

NOVEMBER 11, 2019

Another problem is that a design control, intended to mitigate a failure mode, may not work as intended. Collecting some critical metrics at one second intervals, with a total observability latency of ten seconds or less matches the human attention span much better. This is why most AWS regions have three availability zones.

Latency 52

Latency Engineering Systems Hardware 52

Adrian Cockcroft

NOVEMBER 11, 2019

Another problem is that a design control, intended to mitigate a failure mode, may not work as intended. Collecting some critical metrics at one second intervals, with a total observability latency of ten seconds or less matches the human attention span much better. This is why most AWS regions have three availability zones.

Latency 53

Latency Engineering Systems Hardware 53

Smashing Magazine

JANUARY 11, 2021

Performance isn’t just a technical concern: it affects everything from accessibility to usability to search engine optimization, and when baking it into the workflow, design decisions have to be informed by their performance implications. Looking back now, things seem to have changed quite significantly. Large preview ). Large preview ).

Performance 145

Performance Cache Media Metrics 145

Smashing Magazine

JANUARY 6, 2020

Performance isn’t just a technical concern: it affects everything from accessibility to usability to search engine optimization, and when baking it into the workflow, design decisions have to be informed by their performance implications. Looking back now, things seem to have changed quite significantly. thanks, Guy Podjarny! ).

Performance 145

Performance Cache Network Metrics 145

Smashing Magazine

JANUARY 7, 2019

Performance isn’t just a technical concern: it matters, and when baking it into the workflow, design decisions have to be informed by their performance implications. Without a strong alignment between dev/design and business/marketing teams, performance isn’t going to sustain long-term. Large preview ).

Performance 104

Performance Cache Metrics Network 104