The Morning Paper

FEBRUARY 6, 2020

Last week we looked at a function shipping solution to the problem; Cloudburst uses the more common data shipping to bring data to caches next to function runtimes (though you could also make a case that the scheduling algorithm placing function execution in locations where the data is cached a flavour of function-shipping too).

Lambda 98

Lambda Serverless Cache Latency 98

The Morning Paper

AUGUST 29, 2019

If you’re moving an OLAP workload to the cloud (AWS in the context of this paper), what DBMS setup should you go with? We focused on OLAP-oriented parallel data warehouse products available for AWS and restricted our attention to commercially available systems. Query performance is measured from both warm and cold caches.

Architecture 76

Architecture Cloud Storage Serverless 76

Brendan Gregg

APRIL 14, 2022

The Netflix stack is more diverse than I was expecting, and is explained in detail in the [Netflix tech blog]: The production cloud is AWS EC2, Ubuntu Linux, Intel x86, mostly Java with some Node.js (and other languages), microservices, Cassandra (storage), EVCache (caching), Spinnaker (deployment), Titus (containers), Apache Spark (analytics), Atlas (..)

Java 145

Java Cloud Cache AWS 145

All Things Distributed

MARCH 5, 2012

Driving Compute Cost Down for AWS Customers. AWS today announced a substantial price drop from March 1, 2012 for many of the Amazon EC2, Amazon RDS, and Amazon ElastiCache instances types around the world. Amazon ElastiCache customers will see their prices drop by up to 10%, depending on their cache node types. Comments ().

AWS 110

AWS Innovation Scalability Cache 110

HammerDB

APRIL 19, 2023

As database performance is heavily influenced by the performance of storage, network, memory, and processors, we must understand the upper limit of these key components. For example, if you are buying the latest Amazon memory-optimized EC2 instance (R7iz), the AWS page ( [link] ) tells us the following: Up to 3.9 Be sure to check back.

Performance 54

Performance Benchmarking Cache Storage 54

Brendan Gregg

FEBRUARY 28, 2023

Titus, the Netflix container management platform, is now open source,” [link] Apr 2018 - [Cutress 19] Dr. DDR6: Here's What to Expect in RAM Modules,” [link] Nov 2020 - [Salter 20] Jim Salter, “Western Digital releases new 18TB, 20TB EAMR drives,” [link] Jul 2020 - [Spier 20] Martin Spier, Brendan Gregg, et al.,

Performance 110

Performance Latency Cache Virtualization 110

All Things Distributed

AUGUST 22, 2011

Today AWS has launched Amazon ElastiCache , a new service that makes it easy to add distributed in-memory caching to any application. Amazon ElastiCache handles the complexity of creating, scaling and managing an in-memory cache to free up brainpower for more differentiating activities. Comments ().

Cloud 117

Cloud Cache AWS Storage 117

Smashing Magazine

JUNE 25, 2020

Today, we’ll address storing and serving files for both single-server and scalable deployments while considering factors like compression, caching, and availability. We’ll also discuss the costs and benefits of CDNs and dedicated file storage solutions. First, you’ll need to install the libraries boto3 and django-storages.

Media 69

Media Cache Storage AWS 69

The Netflix TechBlog

OCTOBER 15, 2019

caching (Memcached etc.), Thus, ensuring the atomicity of writes across different storage technologies remains a challenging problem for applications [3]. High availability, via standby instances across AWS Availability Zones. We currently support MySQL and Postgres, including when deployed in AWS RDS and its Aurora flavor.

Transportation 192

Transportation Architecture Processing Storage 192

IO River

NOVEMBER 2, 2023

It’s not just limited to cloud resources like AWS and Azure; Terraform is versatile, extending its capabilities to key performance areas like Content Delivery Network (CDN) management, ensuring efficient content delivery and optimal user experience.‍Started ‍Terraform is a revolution in the way we handle infrastructure.

Open Source 52

Open Source DevOps AWS Azure 52

Percona

MAY 2, 2023

Cons of logical backups As it reads all data, it can be slow and will require disk reads too for databases that are larger than the RAM available for the WT cache—the WT cache pressure increases, which slows down the performance. Especially if going into or out of storage types that may throttle bandwidth/network traffic.

Best Practices 52

Best Practices Database Storage Servers 52

The Netflix TechBlog

NOVEMBER 3, 2022

We introduce a caching mechanism in the API gateway layer, allowing us to offload processing from singleton leader elected controllers without giving up strict data consistency and guarantees clients observe. When a new leader is elected it loads all data from external storage. The cache is kept in sync with the current leader process.

Cache 224

Cache Latency Traffic Systems 224

IO River

NOVEMBER 2, 2023

In fact, Amazon Web Services (AWS) paved the way with its service, CloudFormation, which introduced the concept of Infrastructure as Code (IaC). Taking inspiration from AWS's CloudFormation, HashiCorp introduced Terraform with a broader vision. Terraform is a revolution in the way we handle infrastructure. Hold that thought.

Open Source 40

Open Source DevOps AWS Azure 40

The Morning Paper

JUNE 23, 2019

Coupled with stateless application servers to execute business logic and a database-like system to provide persistent storage, they form a core component of popular data center service archictectures. Oh, you mean a cache? Yes, a bit like those 2nd-level caches we were talking about earlier, e.g. Ehcache from 2003 onwards.

Cache 79

Cache Latency Google Lambda 79

All Things Distributed

DECEMBER 18, 2012

After the AWS re: Invent conference I spent two weeks in Europe for the last customer visits of the year. Although there are still a few very exciting AWS news updates to happen this year. Jul 4 - Leases: An efficient fault-tolerant mechanism for distributed file cache consistency , Gray, Cary, and David Cheriton, Vol.

Operating System 144

Operating System Cache AWS Scalability 144

All Things Distributed

AUGUST 17, 2011

As some of you may remember I was pretty excited when Amazon Simple Storage Service (S3) released its website feature such that I could serve this weblog completely from S3. My templates and blog posts are now located in DropBox and thus locally cached at each machine I use. Driving Storage Costs Down for AWS Customers.

Servers 126

Servers Social Media AWS Website 126

Brendan Gregg

FEBRUARY 28, 2023

Titus, the Netflix container management platform, is now open source,” [link] , Apr 2018 [Cutress 19] Dr. DDR6: Here's What to Expect in RAM Modules,” [link] , Nov 2020 [Salter 20] Jim Salter, “Western Digital releases new 18TB, 20TB EAMR drives,” [link] , Jul 2020 [Spier 20] Martin Spier, Brendan Gregg, et al.,

Performance 52

Performance Latency Cache Virtualization 52

Brendan Gregg

APRIL 14, 2022

The Netflix stack is more diverse than I was expecting, and is explained in detail in the Netflix tech blog : The production cloud is AWS EC2, Ubuntu Linux, Intel x86, mostly Java with some Node.js (and other languages), microservices, Cassandra (storage), EVCache (caching), Spinnaker (deployment), Titus (containers), Apache Spark (analytics), Atlas (..)

Java 52

Java Cache Cloud AWS 52

Percona

APRIL 17, 2023

Make sure the drives are mounted with noatime and also if the drives are behind a RAID controller with appropriate battery-backed cache. For example: Read/Write tickets WiredTiger uses tickets to control the number of read / write operations simultaneously processed by the storage engine. By default, it uses 50% of the memory + 1 GB.

Best Practices 52

Best Practices Design Tuning Database 52

All Things Distributed

DECEMBER 5, 2010

Route 53 has the business properties that you have come to expect from an AWS service: fully self-service and programmable, with transparent pay-as-you-go pricing and no minimum usage commitments. There are two main types of DNS servers: authoritative servers and caching resolvers. Some fundamentals on Naming. No lock-in. Contact Info.

Cloud 117

Cloud Internet Internet AWS 117

The Morning Paper

APRIL 9, 2019

Does your web application make use of local storage? If so, then like many developers you may well be making the assumption that when you read from local storage, it will only contain the data that you put there. There are two basic requirements for a storage-based XSS attack. As Steffens et al. no sanitisation) at a sink.

Storage 44

Storage Cache Network Transportation 44

The Morning Paper

JUNE 16, 2019

If we do that naively though, we’re going to end up with a lot of universes to store and maintain and the storage requirements alone will be prohibitive. The amount of computation required on a base update can be reduced by sharing computation and cached data between universes.

Database 90

Database Cache Benchmarking Efficiency 90

Percona

JANUARY 17, 2019

AWS Aurora (based on MySQL 5.6) now has a version which will support parallelism for SELECT queries (utilizing the read capacity of storage nodes underneath the Aurora cluster). I will compare AWS Aurora with MySQL (Percona Server) 5.6 I will compare AWS Aurora with MySQL (Percona Server) 5.6 This query is 100% cached.

Cache 48

Cache C++ AWS Airlines 48

Smashing Magazine

JANUARY 11, 2021

Build Optimizations JavaScript modules, module/nomodule pattern, tree-shaking, code-splitting, scope-hoisting, Webpack, differential serving, web worker, WebAssembly, JavaScript bundles, React, SPA, partial hydration, import on interaction, 3rd-parties, cache. This improves page load time and caching during navigations.

Performance 145

Performance Cache Media Metrics 145

The Netflix TechBlog

NOVEMBER 15, 2022

Rachel Kelley (AWS), Ranjit Raju (AWS) Rendering is core to the the VFX process VFX studios around the world create amazing imagery for Netflix productions. Rendering on AWS provides the flexibility to control how quickly a project is completed. By: Peter Cioni (Netflix), Alex Schworer (Netflix), Mac Moore (Conductor Tech.),

Cloud 282

Cloud Entertainment AWS Infrastructure 282

Enprowess

JANUARY 12, 2022

So it is convenient for all to use irrespective of internet speed and it works offline using cached data. Other benefits: It has other benefits like a Quicker launch to the market, Easier distribution, saving device power and storage, seamless maintenance, and updating. Blockchain technology is an encrypted database storage system.

Blockchain 59

Blockchain IoT Development Serverless 59

Smashing Magazine

OCTOBER 22, 2019

Instead, you can integrate your existing image server (on your web host) or external storage service (like S3) with ImageKit. However, once you integrate your images or image storage with ImageKit, the tool will take control of your image sizing. There was no need to move their images to ImageKit or another third-party storage service.

Mobile 62

Mobile Media Website Storage 62

Smashing Magazine

JANUARY 6, 2020

The idea is quite straightforward: Push the minimal code needed to get interactive for the initial route to render quickly, then use service worker for caching and pre-caching resources and then lazy-load routes that you need, asynchronously. An application shell is the minimal HTML, CSS, and JavaScript powering a user interface.

Performance 145

Performance Cache Network Metrics 145

Smashing Magazine

JANUARY 7, 2019

The idea is quite straightforward: Push the minimal code needed to get interactive for the initial route to render quickly, then use service worker for caching and pre-caching resources and then lazy-load routes that you need, asynchronously. An application shell is the minimal HTML, CSS, and JavaScript powering a user interface.

Performance 103

Performance Cache Metrics Network 103

The Netflix TechBlog

SEPTEMBER 24, 2021

From chunk encoding to assembly and packaging, the result of each previous processing step must be uploaded to cloud storage and then downloaded by the next processing step. Since not all projects are terabytes projects, allocating the largest cloud storage to all packager instances is not an efficient use of cloud resources.

Cloud 237

Cloud Media Storage Cache 237

The Netflix TechBlog

MARCH 8, 2021

Below is a broad technical overview of how to go from an AWS instance to a Netflix Workstation. They could need a GPU when doing graphics-intensive work or extra large storage to handle file management. Instead, we created a service to take the most popular configurations and cache them.

Entertainment 306

Entertainment Storage Hardware Open Source 306

The Netflix TechBlog

MARCH 10, 2023

are stored in secure storage layers. Amsterdam is built on top of three storage layers. Elasticsearch recommends each shard to be under 65GB (AWS recommends them to be under 50GB), so we could create time based indices where each index holds somewhere between 16–20GB of data, giving some buffer for data growth.

Strategy 258

Strategy Cache Analytics Storage 258

High Scalability

FEBRUARY 19, 2019

Redis Cluster is the native sharding implementation available within Redis that allows you to automatically distribute your data across multiple nodes without having to rely on external tools and utilities. At ScaleGrid, we recently added support for Redis Clusters on our platform through our fully managed Redis hosting plans.

Availability 204

Availability Open Source Cache Storage 204

All Things Distributed

JUNE 21, 2017

Today, I'm excited to announce the general availability of Amazon DynamoDB Accelerator (DAX) , a fully managed, highly available, in-memory cache that can speed up DynamoDB response times from milliseconds to microseconds, even at millions of requests per second. DynamoDB was the first service at AWS to use SSD storage.

Speed 121

Speed Cache Latency AWS 121

High Scalability

JULY 20, 2018

That means multiple data indirections mean multiple cache misses. Mark LaPedus : MRAM, a next-generation memory type, is being touted as a replacement for embedded flash and cache applications. Cliff Click : The JVM is very good at eliminating the cost of code abstraction, but not the cost of data abstraction. They are very expensive.

Internet 121

Internet Internet Scalability Automotive 121

All Things Distributed

APRIL 27, 2011

We use high-performance transactions systems, complex rendering and object caching, workflow and queuing systems, business intelligence and data analytics, machine learning and pattern recognition, neural networks and probabilistic decision making, and a wide variety of other techniques. Driving Storage Costs Down for AWS Customers.

Technology 151

Technology Technology AWS Storage 151

All Things Distributed

OCTOBER 2, 2017

As we began growing the AWS business, we realized that external customers might find our Dynamo database just as useful as we found it within Amazon.com. So, we set out to build a fully hosted AWS database service based upon the original Dynamo design.

Internet 128

Internet Internet AWS Performance 128