Brendan Gregg

FEBRUARY 28, 2023

My personal opinion is that I don't see a widespread need for more capacity given horizontal scaling and servers that can already exceed 1 Tbyte of DRAM; bandwidth is also helpful, but I'd be concerned about the increased latency for adding a hop to more memory. Ford, et al., “TCP

Performance 110

Performance Latency Cache Virtualization 110

All Things Distributed

DECEMBER 14, 2011

Introducing the AWS South America (Sao Paulo) Region. Today, Amazon Web Services is expanding its worldwide coverage with the launch of a new AWS Region in Sao Paulo, Brazil. Several prominent South American customers have been using AWS since the early days. RAMA , a Brazilian financial services firm and AWS customer : â??The

AWS 114

AWS Latency Storage Big Data 114

All Things Distributed

MARCH 2, 2011

Expanding the Cloud - Introducing the AWS Asia Pacific (Tokyo) Region. Today Amazon Web Services is expanding its world-wide coverage with the launch of a new AWS Region located in Tokyo, Japan. The advanced Asia Pacific network infrastructure also makes the AWS Tokyo Region a viable low-latency option for customers from South Korea.

AWS 112

AWS Cloud Games Latency 112

Scalegrid

JANUARY 8, 2024

Infrastructure Excellence ScaleGrid’s infrastructure is designed to facilitate hosting in your cloud account and provides cost-saving options with AWS or Azure Reserved Instances or GCP. Meeting Business Demands with Hybrid Cloud A range of business demands can be met by the design of hybrid cloud solutions.

Artificial Intelligence 147

Artificial Intelligence Cloud Scalability Analytics 147

The Netflix TechBlog

NOVEMBER 22, 2019

by Shefali Vyas Dalal AWS re:Invent is a couple weeks away and our engineers & leaders are thrilled to be in attendance yet again this year! To sustain this data growth at Netflix, it has deployed open-source software Ceph using AWS services to achieve the required SLOs of some of the post-production workflows.

AWS 37

AWS Entertainment Open Source Benchmarking 37

All Things Distributed

AUGUST 16, 2011

Expanding the Cloud - The AWS GovCloud (US) Region. Today AWS announced the launch of the AWS GovCloud (US) Region. The concept of regions gives AWS customers control over the placement of their resources and services. The Cloud First strategy is most visible with new Federal IT programs, which are all designed to be â??Cloud

AWS 111

AWS Government Big Data Cloud 111

Brendan Gregg

JULY 4, 2021

It's an exciting time for developments in computer performance, not just for the BPF technology (which I often [write about]) but also for processors with 3D stacking and cloud vendor CPUs (e.g., Ford, et al., “TCP on Upcoming Sapphire Rapids CPUs,” [link] Oct 2020 - [Liu 20] Linda Liu, “Samsung QVO vs EVO vs PRO: What’s the Difference?

Performance 52

Performance Latency Hardware Storage 52

The Netflix TechBlog

DECEMBER 21, 2020

By Anupom Syam Background At Netflix, our current data warehouse contains hundreds of Petabytes of data stored in AWS S3 , and each day we ingest and create additional Petabytes. We built AutoOptimize to efficiently and transparently optimize the data and metadata storage layout while maximizing their cost and performance benefits.

Storage 203

Storage Latency Efficiency Data Engineering 203

The Morning Paper

AUGUST 29, 2019

If you’re moving an OLAP workload to the cloud (AWS in the context of this paper), what DBMS setup should you go with? We focused on OLAP-oriented parallel data warehouse products available for AWS and restricted our attention to commercially available systems. The design space. VLDB’19. Key findings. Serverless o?erings

Architecture 76

Architecture Cloud Storage Serverless 76

Brendan Gregg

FEBRUARY 28, 2023

My personal opinion is that I don't see a widespread need for more capacity given horizontal scaling and servers that can already exceed 1 Tbyte of DRAM; bandwidth is also helpful, but I'd be concerned about the increased latency for adding a hop to more memory. Ford, et al., “TCP

Performance 52

Performance Latency Cache Virtualization 52

All Things Distributed

DECEMBER 3, 2009

Expanding the Cloud - New AWS Region: US-West (Northern California). We have expanded the AWS footprint in the US and starting today a new AWS Region is available for use: US-West (Northern California). AWS is committed to making its services available at low cost. Driving Storage Costs Down for AWS Customers.

AWS 60

AWS Cloud Latency Storage 60

All Things Distributed

APRIL 28, 2010

Expanding the Cloud - Opening the AWS Asia Pacific (Singapore) Region. Today Amazon Web Services has taken another important step in serving customers worldwide: the AWS Asia Pacific (Singapore) Region is now launched. The new Singapore Region offers customers in APAC lower-latency access to AWS services. Comments ().

AWS 61

AWS Cloud Latency Storage 61

Percona

DECEMBER 11, 2023

A quick configuration change may do the trick in improving the performance of your AWS RDS for MySQL instance. A Dedicated Log Volume (DLV) is a specialized storage volume designed to house database transaction logs separately from the volume containing the database tables. Who can benefit from DLV? and later v13 versions, 14.7

AWS 96

AWS Benchmarking Performance Traffic 96

All Things Distributed

JULY 13, 2010

Cluster Computer Instances for Amazon EC2 are a new instance type specifically designed for High Performance Computing applications. Other industries using Amazon EC2 for HPC-style workloads include pharmaceuticals, oil exploration, industrial and automotive design, media and entertainment, and more. Countdown to What is Next in AWS.

Cloud 118

Cloud AWS Automotive Latency 118

InfoQ

JUNE 16, 2023

The event-driven architecture (EDA) can accommodate bursts in traffic in the multi-tenant platform with reasonable ingestion latency and acceptable operational costs. Datadog created a dedicated data ingestion architecture offering exactly-once semantics for their third-generation event store, Husky. By Rafal Gancarz

Architecture 55

Architecture Scalability Latency Traffic 55

All Things Distributed

DECEMBER 5, 2010

Route 53 has the business properties that you have come to expect from an AWS service: fully self-service and programmable, with transparent pay-as-you-go pricing and no minimum usage commitments. We have designed Route 53 to propagate updates very quickly and give the customer the tools to find out when all changes have been propagated.

Cloud 117

Cloud Internet Internet AWS 117

All Things Distributed

JULY 11, 2011

As a part of that process, we also realized that there were a number of latency sensitive or location specific use cases like Hadoop, HPC, and testing that would be ideal for Spot. To get started using Spot or for more details visit the Amazon EC2 Spot Instance web page, the AWS developer blog , and the EC2 Release Notes. Contact Info.

AWS 85

AWS Storage Cloud Big Data 85

All Things Distributed

NOVEMBER 19, 2010

Understanding Throughput-Oriented Architectures - background article in CACM on massively parallel and throughput vs latency oriented architectures. Driving Storage Costs Down for AWS Customers. Expanding the Cloud - The AWS Storage Gateway. Countdown to What is Next in AWS. All postings. Recent Entries.

AWS 67

AWS Cloud Benchmarking Storage 67

Dynatrace

DECEMBER 22, 2019

AWS offers a broad set of global, cloud-based services including computing, storage, networking, Internet of Things (IoT), and many others. At Dynatrace, we’re constantly improving our AWS monitoring capabilities. Monitor and understand additional AWS services. Get up to 300 new AWS metrics out of the box.

AWS 136

AWS Metrics IoT Storage 136

Dynatrace

DECEMBER 22, 2019

AWS offers a broad set of global, cloud-based services including computing, storage, networking, Internet of Things (IoT), and many others. At Dynatrace, we’re constantly improving our AWS monitoring capabilities. Monitor and understand additional AWS services. Get up to 300 new AWS metrics out of the box.

AWS 133

AWS Metrics IoT Storage 133

The Netflix TechBlog

SEPTEMBER 24, 2021

From chunk encoding to assembly and packaging, the result of each previous processing step must be uploaded to cloud storage and then downloaded by the next processing step. Uploading and downloading data always come with a penalty, namely latency.

Cloud 237

Cloud Media Storage Cache 237

Percona

JULY 1, 2023

Now that Database-as-a-service (DBaaS) is in high demand, there are multiple questions regarding AWS services that cannot always be answered easily: When should I use Aurora and when should I use RDS MySQL ? Amazon Aurora is a proprietary, cloud-native, fully managed relational database service developed by Amazon Web Services (AWS).

AWS 52

AWS Database Serverless Storage 52

Scalegrid

MARCH 14, 2024

Public Cloud Infrastructure Third-party providers run public cloud services, delivering a broad array of offerings like computing power, storage solutions, and network capabilities that enhance the functionality of a hybrid cloud architecture. We will examine each of these elements in more detail.

Strategy 130

Strategy Cloud Artificial Intelligence Infrastructure 130

Scalegrid

DECEMBER 16, 2019

While you may assume a great majority of the cloud database deployments are run on AWS, Azure, or Google Cloud Platform, small to medium-sized businesses in particular are gravitating towards the developer-friendly cloud provider, DigitalOcean , for their hosting for MongoDB® needs. DigitalOcean Advantages for MongoDB. DigitalOcean Droplets.

Azure 187

Azure AWS Latency Database 187

Percona

APRIL 17, 2023

The CFQ works well for many general use cases but lacks latency guarantees. The deadline excels at latency-sensitive use cases ( like databases ), and noop is closer to no schedule at all. By default, most Linux installs use the CFQ ( Completely-Fair Queue ) scheduler. Two other schedulers are deadline and noop.

Best Practices 52

Best Practices Design Tuning Database 52

The Netflix TechBlog

OCTOBER 27, 2020

The processed data is typically stored as data warehouse tables in AWS S3. The data warehouse is not designed to serve point requests from microservices with low latency. Therefore, we must efficiently move data from the data warehouse to a global, low-latency and highly-reliable key-value store.

Latency 243

Latency Storage Big Data Tuning 243

The Netflix TechBlog

APRIL 26, 2022

We will share how its design has evolved over the years and the lessons learned while building it. To understand Axion’s design, we need to know the various components that interact with it. The motivation has not changed since then; the design has. Design evolution Axion fact store has four components?—?fact

Storage 187

Storage Design Scalability Latency 187

Dynatrace

OCTOBER 4, 2022

Data warehouses offer a single storage repository for structured data and provide a source of truth for organizations. Unlike data warehouses, however, data is not transformed before landing in storage. A data lakehouse provides a cost-effective storage layer for both structured and unstructured data. Data management.

Artificial Intelligence 209

Artificial Intelligence Analytics Storage Government 209

High Scalability

SEPTEMBER 8, 2018

No, I don’t think that is because AWS is earning a 355x margin on DynamoDB! They've posted about Anna's new superpowers in Going Fast and Cheap: How We Made Anna Autoscale : Using Anna v0 as an in-memory storage engine, we set out to address the cloud storage problems described above.

Storage 199

Storage Performance AWS Efficiency 199

Scalegrid

JANUARY 12, 2024

Storage is a critical aspect to consider when working with cloud workloads. High availability storage options within the context of cloud computing involve highly adaptable storage solutions specifically designed for storing vast amounts of data while providing easy access to it. What is an example of a workload?

Cloud 130

Cloud Virtualization Storage Efficiency 130

The Netflix TechBlog

NOVEMBER 3, 2022

When a new leader is elected it loads all data from external storage. In that scenario, the system would need to deal with the data propagation latency directly, for example, by use of timeouts or client-originated update tracking mechanisms. Active data includes jobs and tasks that are currently running.

Cache 224

Cache Latency Traffic Systems 224

The Morning Paper

FEBRUARY 6, 2020

’ Stateless is fine until you need state, at which point the coarse-grained solutions offered by current platforms limit the kinds of application designs that work well. On the Cloudburst design teams’ wish list: A running function’s ‘hot’ data should be kept physically nearby for low-latency access.

Lambda 98

Lambda Serverless Cache Latency 98

The Netflix TechBlog

FEBRUARY 9, 2021

Before designing a solution it’s important to understand the main product requirements for such a feature: The content needs to be new, relevant, and regional (not all countries have the same catalogue). To reduce latency, assets should be generated in an offline fashion and not in real time. This requires an asset storage solution.

Engineering 165

Engineering Storage Latency Entertainment 165

All Things Distributed

MARCH 11, 2016

The epoch of AWS is the launch of Amazon S3 on March 14, 2006, now almost 10 years ago. Given that AWS is a pioneer in building and operating these services world-wide, these lessons have been of crucial importance to our business. AWS helps its customers do this too. your resource usage.

AWS 144

AWS Hardware Retail Virtualization 144

All Things Distributed

APRIL 9, 2015

Details on the AWS Blog. AWS has been offering a range of storage solutions: objects, block storage, databases, archiving, etc. When we designed Amazon EFS we decided to build along the AWS principles: Elastic, scalable, highly available, consistent performance, secure, and cost-effective. Amazon Lambda.

Lambda 122

Lambda Cloud IoT AWS 122

High Scalability

JULY 20, 2018

DonHopkins : NeWS differs from the current technology stack in that it was all coherently designed at once by James Gosling and David Rosenthal, by taking several steps back and thinking deeply about all the different problems it was trying to solve together. Some are lucky enough to also have a production environment."

Internet 121

Internet Internet Scalability Automotive 121

Percona

MAY 2, 2023

The speed of backup also depends on allocated IOPS and type of storage since lots of read/writes would be happening during this process. Percona Backup for MongoDB is an uncomplicated command-line tool by design that lends itself well to backing up larger data sets. It also supports storing the backups on filesystem and AWS S3 buckets.

Best Practices 52

Best Practices Database Storage Servers 52