Brendan Gregg

AUGUST 29, 2021

Problem Statement The microservice managed and processed large files, including encrypting them and then storing them on S3. 1072-aws (xxx) 12/18/2018 _x86_64_ (16 CPU) avg-cpu: %user %nice %system %iowait %steal %idle 5.03 1072-aws (xxx) 12/18/2018 _x86_64_ (16 CPU) avg-cpu: %user %nice %system %iowait %steal %idle 5.03

Cache 105

Cache C++ AWS Latency 105

All Things Distributed

MARCH 5, 2012

Driving Compute Cost Down for AWS Customers. AWS today announced a substantial price drop from March 1, 2012 for many of the Amazon EC2, Amazon RDS, and Amazon ElastiCache instances types around the world. Amazon ElastiCache customers will see their prices drop by up to 10%, depending on their cache node types. Comments ().

AWS 110

AWS Innovation Scalability Cache 110

Percona

DECEMBER 23, 2022

The use case is the TPC-C benchmark but executed not on a high-end server but on a lower-spec virtual machine that is I/O limited like for example, with AWS EBS volumes. The storage device has performance characteristics pretty similar to an AWS gp2 EBS volume of about 330 GB in size. The following figure illustrates the impacts.

Storage 55

Storage Cache C++ Virtualization 55

Dynatrace

OCTOBER 7, 2021

Amazon Web Services (AWS), offers a wide range of serverless solutions. To get a better understanding of AWS serverless, we’ll first explore the basics of serverless architectures, review AWS serverless offerings, and explore common use cases. AWS serverless offerings. Common use cases for AWS serverless services.

Serverless 273

Serverless AWS Lambda Storage 273

The Netflix TechBlog

OCTOBER 15, 2019

caching (Memcached etc.), Another thread or process is constantly polling events from the log table and writes them to one or multiple datastores, optionally removing events from the log table after acknowledged by all datastores. High availability, via standby instances across AWS Availability Zones.

Transportation 192

Transportation Architecture Processing Storage 192

The Netflix TechBlog

MARCH 5, 2019

the order of the rows on your Netflix home page, issuing content licenses when you click play, finding the Open Connect cache closest to you with the content you requested, and many more). All these micro-services are currently operated in AWS cloud infrastructure.

Infrastructure 182

Infrastructure Cloud Scalability AWS 182

The Netflix TechBlog

NOVEMBER 3, 2022

We introduce a caching mechanism in the API gateway layer, allowing us to offload processing from singleton leader elected controllers without giving up strict data consistency and guarantees clients observe. cell): Titus Job Coordinator is a leader elected process managing the active state of the system.

Cache 224

Cache Latency Traffic Systems 224

The Morning Paper

MARCH 8, 2020

If all data was read from S3 every time, performance would suffer, so of course Snowflake has a caching layer – a distributed ephemeral storage service shared by all the nodes in a warehouse. The caching use case may be the most familiar, but in fact it’s not the primary purpose of the ephemeral storage service.

Storage 112

Storage Engineering Cache Serverless 112

Brendan Gregg

FEBRUARY 28, 2023

Titus, the Netflix container management platform, is now open source,” [link] Apr 2018 - [Cutress 19] Dr. DDR6: Here's What to Expect in RAM Modules,” [link] Nov 2020 - [Salter 20] Jim Salter, “Western Digital releases new 18TB, 20TB EAMR drives,” [link] Jul 2020 - [Spier 20] Martin Spier, Brendan Gregg, et al.,

Performance 110

Performance Latency Cache Virtualization 110

The Morning Paper

AUGUST 29, 2019

If you’re moving an OLAP workload to the cloud (AWS in the context of this paper), what DBMS setup should you go with? We focused on OLAP-oriented parallel data warehouse products available for AWS and restricted our attention to commercially available systems. Query performance is measured from both warm and cold caches.

Architecture 76

Architecture Cloud Storage Serverless 76

Brendan Gregg

AUGUST 29, 2021

Problem Statement The microservice managed and processed large files, including encrypting them and then storing them on S3. 1072-aws (xxx) 12/18/2018 _x86_64_ (16 CPU) avg-cpu: %user %nice %system %iowait %steal %idle 5.03 1072-aws (xxx) 12/18/2018 _x86_64_ (16 CPU) avg-cpu: %user %nice %system %iowait %steal %idle 5.03

Cache 40

Cache C++ AWS Systems 40

HammerDB

APRIL 19, 2023

For example, if you are buying the latest Amazon memory-optimized EC2 instance (R7iz), the AWS page ( [link] ) tells us the following: Up to 3.9 In the context of MySQL performance evaluation, it simulates a typical online transaction processing (OLTP) workload on your MySQL database.

Performance 54

Performance Benchmarking Cache Storage 54

All Things Distributed

NOVEMBER 26, 2013

As I discussed in my re:Invent keynote earlier this month, I am now happy to announce the immediate availability of Amazon RDS Cross Region Read Replicas , which is another important enhancement for our customers using or planning to use multiple AWS Regions to deploy their applications. Cross Region Read Replicas are available for MySQL 5.6

Cloud 145

Cloud AWS Traffic Latency 145

All Things Distributed

DECEMBER 18, 2012

After the AWS re: Invent conference I spent two weeks in Europe for the last customer visits of the year. Although there are still a few very exciting AWS news updates to happen this year. Jul 4 - Leases: An efficient fault-tolerant mechanism for distributed file cache consistency , Gray, Cary, and David Cheriton, Vol.

Operating System 144

Operating System Cache AWS Scalability 144

Brendan Gregg

FEBRUARY 28, 2023

Titus, the Netflix container management platform, is now open source,” [link] , Apr 2018 [Cutress 19] Dr. DDR6: Here's What to Expect in RAM Modules,” [link] , Nov 2020 [Salter 20] Jim Salter, “Western Digital releases new 18TB, 20TB EAMR drives,” [link] , Jul 2020 [Spier 20] Martin Spier, Brendan Gregg, et al.,

Performance 52

Performance Latency Cache Virtualization 52

The Netflix TechBlog

NOVEMBER 15, 2022

Rachel Kelley (AWS), Ranjit Raju (AWS) Rendering is core to the the VFX process VFX studios around the world create amazing imagery for Netflix productions. Rendering on AWS provides the flexibility to control how quickly a project is completed.

Cloud 282

Cloud Entertainment AWS Infrastructure 282

The Netflix TechBlog

OCTOBER 18, 2019

The key insight was that by assuming a latent Gaussian Process (GP) prior on the key business metric actions like viral engagement, job applications, etc., In particular, he talked about the misattribution potential in a complex microservice architecture where often intermediary results are cached.

Infrastructure 126

Infrastructure Metrics Architecture Efficiency 126

All Things Distributed

AUGUST 17, 2011

But while this blog happily runs out of S3, the process of creating and updating the content still required a server to run my Moveable Type installation and hold the database. My templates and blog posts are now located in DropBox and thus locally cached at each machine I use. Driving Storage Costs Down for AWS Customers.

Servers 126

Servers Social Media AWS Website 126

The Netflix TechBlog

OCTOBER 18, 2019

The key insight was that by assuming a latent Gaussian Process (GP) prior on the key business metric actions like viral engagement, job applications, etc., In particular, he talked about the misattribution potential in a complex microservice architecture where often intermediary results are cached.

Infrastructure 100

Infrastructure Metrics Architecture Efficiency 100

The Netflix TechBlog

SEPTEMBER 24, 2021

By Xiaomei Liu , Rosanna Lee , Cyril Concolato Introduction Behind the scenes of the beloved Netflix streaming service and content, there are many technology innovations in media processing. Packaging has always been an important step in media processing. Uploading and downloading data always come with a penalty, namely latency.

Cloud 237

Cloud Media Storage Cache 237

Smashing Magazine

AUGUST 3, 2023

However, achieving a good LCP score is often a multi-faceted process that involves optimizing several stages of loading and rendering. HTML Processing Once a web page’s HTML file has been downloaded, the browser begins to process the contents line by line, translating code into the visual website that users interact with.

Performance 75

Performance Cache Traffic Metrics 75

All Things Distributed

SEPTEMBER 3, 2013

AWS offers its customers a choice of different database services, each optimized for different workloads. Amazon ElastiCache is a fully managed, in-memory caching service for customers to optimize the latency, performance and cost of their read workloads.

Cache 129

Cache Cloud Performance Retail 129

The Netflix TechBlog

MARCH 10, 2023

Elasticsearch recommends each shard to be under 65GB (AWS recommends them to be under 50GB), so we could create time based indices where each index holds somewhere between 16–20GB of data, giving some buffer for data growth. For every asset indexing request, we look at the cache to determine the corresponding time bucket index for the asset.

Strategy 258

Strategy Cache Analytics Storage 258

The Morning Paper

FEBRUARY 6, 2020

Last week we looked at a function shipping solution to the problem; Cloudburst uses the more common data shipping to bring data to caches next to function runtimes (though you could also make a case that the scheduling algorithm placing function execution in locations where the data is cached a flavour of function-shipping too).

Lambda 98

Lambda Serverless Cache Latency 98

IO River

NOVEMBER 2, 2023

It’s not just limited to cloud resources like AWS and Azure; Terraform is versatile, extending its capabilities to key performance areas like Content Delivery Network (CDN) management, ensuring efficient content delivery and optimal user experience.‍Started ‍Terraform is a revolution in the way we handle infrastructure.

Open Source 52

Open Source DevOps AWS Azure 52

Dynatrace

OCTOBER 4, 2022

However, organizations must structure and store data inputs in a specific format to enable extract, transform, and load processes, and efficiently query this data. Data lakehouses take advantage of low-cost object stores like AWS S3 or Microsoft Azure Blob Storage to store and manage data cost-effectively. Query language.

Artificial Intelligence 202

Artificial Intelligence Storage Analytics Government 202

All Things Distributed

JUNE 21, 2017

Today, I'm excited to announce the general availability of Amazon DynamoDB Accelerator (DAX) , a fully managed, highly available, in-memory cache that can speed up DynamoDB response times from milliseconds to microseconds, even at millions of requests per second. DynamoDB was the first service at AWS to use SSD storage.

Speed 121

Speed Cache Latency AWS 121

IO River

NOVEMBER 2, 2023

In fact, Amazon Web Services (AWS) paved the way with its service, CloudFormation, which introduced the concept of Infrastructure as Code (IaC). Taking inspiration from AWS's CloudFormation, HashiCorp introduced Terraform with a broader vision. Terraform is a revolution in the way we handle infrastructure.

Open Source 40

Open Source DevOps AWS Azure 40

Smashing Magazine

JULY 6, 2021

Moving to a headless CMS is not an easy decision and the selection process should not be taken lightly. The process of defining this sound, logical taxonomy structure for the content items you create and publish is known as content modeling. From a developer perspective, not only static assets need to be cached on a CDN.

Cache 143

Cache Website Mobile Google 143

The Morning Paper

JUNE 23, 2019

Factor VI in the 12-factor app manifesto , “Execute the app as one or more stateless processes,” to be dropped and replaced with “Execute the app as one or more stateful processes.” session state that you want to survive an application process crash), and to keep the application server/services layer stateless.

Cache 79

Cache Latency Google Lambda 79

High Scalability

JULY 20, 2018

That means multiple data indirections mean multiple cache misses. jaybo_nomad : The Allen Institute for Brain Science is in the process of imaging 1 cubic mm of mouse visual cortex using TEM at a resolution of 4nm per pixel. They are very expensive. This is where your performance goes. Some say MRAM will never work in automotive.

Internet 121

Internet Internet Scalability Automotive 121

The Netflix TechBlog

OCTOBER 16, 2019

Often the data is held in memory by consumers and used as a “total cache”, where it is accessed at runtime by client code and atomically swapped out under the hood. for example Open Connect Appliance cache configuration, supported device type IDs, supported payment method metadata, and A/B test configuration.

Cache 187

Cache Architecture Metrics Java 187

All Things Distributed

OCTOBER 12, 2016

Three years ago, as part of our AWS Fast Data journey we introduced Amazon ElastiCache for Redis , a fully managed in-memory data store that operates at sub-millisecond latency. While caching continues to be a dominant use of ElastiCache for Redis, we see customers increasingly use it as an in-memory NoSQL database.

Scalability 130

Scalability Analytics Cache AWS 130

The Netflix TechBlog

DECEMBER 17, 2019

Nonetheless, we found a number of limitations that could not satisfy our requirements e.g. stalling the processing of log events until a dump is complete, missing ability to trigger dumps on demand, or implementations that block write traffic by using table locks. Some of DBLog’s features are: Processes captured log events in-order.

Database 212

Database Traffic Transportation Open Source 212

The Netflix TechBlog

DECEMBER 17, 2019

Nonetheless, we found a number of limitations that could not satisfy our requirements e.g. stalling the processing of log events until a dump is complete, missing ability to trigger dumps on demand, or implementations that block write traffic by using table locks. Some of DBLog’s features are: Processes captured log events in-order.

Database 197

Database Traffic Transportation Open Source 197

IO River

NOVEMBER 2, 2023

‍Think of a situation where you're asked to build a service in AWS that distributes static content to your users. Your first option is to use its native CDN - Amazon CloudFront, as it seamlessly integrates with all of your other AWS Services that you're using to build your service.‍For What is vendor lock-in?‍Think

Lambda 52

Lambda AWS Cache Website 52

Percona

MARCH 27, 2023

This release adds the GCache encryption and Write-Set cache encryption feature in tech preview , along with improvements and bug fixes introduced by Oracle for MySQL 8.0.31. This release is focused on the Percona XtraDB Cluster-based deployment variation. Download Percona Distribution for MySQL (PXC-based variant) 8.0.31 was released.

Open Source 77

Open Source Servers Scalability Cache 77