All Things Distributed

NOVEMBER 26, 2013

As I discussed in my re:Invent keynote earlier this month, I am now happy to announce the immediate availability of Amazon RDS Cross Region Read Replicas , which is another important enhancement for our customers using or planning to use multiple AWS Regions to deploy their applications. Cross Region Read Replicas are available for MySQL 5.6

Cloud 147

Cloud AWS Traffic Latency 147

All Things Distributed

OCTOBER 25, 2013

Many of you know Thorsten von Eicken as the founder of Rightscale , the company that has helped numerous organizations find their way onto AWS. In what seems almost a previous life by now Thorsten was one of the top young professors in Distributed Systems and I had the great pleasure of working with him at Cornell in the early 90''s.

Network 99

Network Latency Operating System Speed 99

Dynatrace

SEPTEMBER 30, 2021

Cloud services platforms like AWS, Azure, and GCP are reshaping how organizations deliver value to their customers, making cloud migration an increasingly attractive option for running applications. This can dramatically decrease network latency and its effect on the end-user experience. Inconsistent performance.

Cloud 167

Cloud Traffic Best Practices Strategy 167

Brendan Gregg

FEBRUARY 28, 2023

This talk originated from my updates to Systems Performance 2nd Edition , and this was the first time I've given this talk in person! CXL in a way allows a custom memory controller to be added to a system, to increase memory capacity, bandwidth, and overall performance. Ford, et al., “TCP

Performance 52

Performance Latency Cache Virtualization 52

All Things Distributed

SEPTEMBER 25, 2012

Werner Vogels weblog on building scalable and robust distributed systems. As with all AWS services, if you are in doubt about whether your application can benefit from Provisioned IOPS, it is very easy to try it out and decide for yourself, as there are no upfront costs. s architecture. Expanding the Cloud â?? Comments ().

Cloud 118

Cloud AWS Storage Database 118

The Netflix TechBlog

FEBRUARY 18, 2022

To this end, we developed a Rapid Event Notification System (RENO) to support use cases that require server initiated communication with devices in a scalable and extensible manner. In this blog post, we will give an overview of the Rapid Event Notification System at Netflix and share some of the learnings we gained along the way.

Systems 334

Systems Traffic Architecture Mobile 334

All Things Distributed

JULY 13, 2010

Werner Vogels weblog on building scalable and robust distributed systems. Not just for HPC but for mission critical enterprise systems such as OLTP. Additionally, many high-end HPC applications take advantage of knowing their in-house hardware platforms to achieve major speedup by exploiting the specific processor architecture.

Cloud 119

Cloud AWS Automotive Latency 119

All Things Distributed

DECEMBER 5, 2010

Werner Vogels weblog on building scalable and robust distributed systems. I am very excited that today we have launched Amazon Route 53, a high-performance and highly-available Domain Name System (DNS) service. Naming is one of the fundamental concepts in Distributed Systems. By Werner Vogels on 05 December 2010 02:00 PM.

Cloud 117

Cloud Internet Internet AWS 117

All Things Distributed

NOVEMBER 8, 2012

Werner Vogels weblog on building scalable and robust distributed systems. For example, AWS customers use SQS for asynchronous communication pipelines, buffer queues for databases, asynchronous work queues, and moving latency out of highly responsive requests paths. All Things Distributed. Comments (). Within Amazonâ??s

Efficiency 107

Efficiency Cloud Games Scalability 107

ScaleOut Software

APRIL 6, 2021

Message are delivered to the grid using messaging hubs, such as Azure IoT Hub, AWS IoT Core, Kafka, a built-in REST service, or directly using APIs. This simplifies the installation process and ensures portability across operating systems. The management console installs as a set of Docker containers on the management server.

IoT 52

IoT Azure Analytics Healthcare 52

VoltDB

JUNE 1, 2017

Telcos and Fortune 500 corporations used to spend years defining custom systems, down to the last cable. These systems were designed to have a lifetime of half a decade or more, and rapidly changing hardware meant that the initial deployment had to be sized for 5-7 years out. The “Public Private Cloud” folks.

Hardware 56

Hardware Latency Virtualization Cloud 56

Scalegrid

MARCH 14, 2024

Understanding Hybrid Cloud Strategy A hybrid cloud merges the capabilities of public and private clouds into a singular, coherent system. The architecture usually integrates several private, public, and on-premises infrastructures. We will examine each of these elements in more detail.

Strategy 130

Strategy Cloud Artificial Intelligence Infrastructure 130

O'Reilly

DECEMBER 2, 2019

I don’t need more bandwidth for video conferences or movies, but I would like to be able to download operating system updates and other large items in seconds rather than minutes. There are impressive estimates for latency for 5G, but reality has a tendency to be harsh on such predictions. That’s the real promise of 5G. Mike Loukides.

Wireless 53

Wireless Serverless IoT Artificial Intelligence 53

All Things Distributed

NOVEMBER 19, 2010

Werner Vogels weblog on building scalable and robust distributed systems. Understanding Throughput-Oriented Architectures - background article in CACM on massively parallel and throughput vs latency oriented architectures. Driving Storage Costs Down for AWS Customers. Expanding the Cloud - The AWS Storage Gateway.

AWS 68

AWS Cloud Benchmarking Storage 68

Dynatrace

JANUARY 26, 2021

Traditional computing models rely on virtual or physical machines, where each instance includes a complete operating system, CPU cycles, and memory. Within this paradigm, it is possible to run entire architectures without touching a traditional virtual server, either locally or in the cloud. What is serverless computing?

Serverless 244

Serverless Efficiency Lambda Azure 244

VoltDB

JUNE 1, 2017

Telcos and Fortune 500 corporations used to spend years defining custom systems, down to the last cable. These systems were designed to have a lifetime of half a decade or more, and rapidly changing hardware meant that the initial deployment had to be sized for 5-7 years out. The “Public Private Cloud” folks.

Hardware 40

Hardware Latency Virtualization Cloud 40

Adrian Cockcroft

MAY 6, 2023

Then they tried to scale it to cope with high traffic and discovered that some of the state transitions in their step functions were too frequent, and they had some overly chatty calls between AWS lambda functions and S3. His first edition in 2015 was foundational, and he updated it in 2021 with a second edition.

Serverless 145

Serverless Lambda Best Practices Traffic 145

Adrian Cockcroft

JANUARY 20, 2023

on Myths and Legends of High Performance Computing  — it’s a somewhat light-hearted look at some of the same issues by the leader of the team that built the Fugaku system I mention below. Next generation architectures will use CXL3.0 HPCG is led by Japan’s RIKEN Fugaku system at 16 petaflops, which is 3% of it’s peak capacity.

Architecture 52

Architecture Latency Benchmarking AWS 52

All Things Distributed

MARCH 26, 2018

At AWS, we don't mark many anniversaries. A concept that has changed infrastructure architecture is now at the core of both AWS and customer reliability and operations. Powering the virtual instances and other resources that make up the AWS Cloud are real physical data centers with AWS servers in them.

AWS 134

AWS Latency Lambda Architecture 134

Dynatrace

JANUARY 14, 2022

The following figure shows the high-level architecture where any load testing solution (e.g. The optimization goal was to improve the application efficiency, that is to improve the ratio between service throughput and cloud costs while not increasing the application latency (e.g. below 500ms) and error rates (e.g. lower than 2%.).

Latency 206

Latency Tuning Efficiency AWS 206

The Netflix TechBlog

NOVEMBER 3, 2022

As the number of Titus users increased over the years, the load and pressure on the system increased substantially. The original assumptions and architectural choices were no longer viable. Overview The figure below depicts a simplified high-level architecture of a single Titus cluster (a.k.a

Cache 224

Cache Latency Traffic Systems 224

The Netflix TechBlog

SEPTEMBER 24, 2021

Table 1: Movie and File Size Examples Initial Architecture A simplified view of our initial cloud video processing pipeline is illustrated in the following diagram. Lastly, the packager kicks in, adding a system layer to the asset, making it ready to be consumed by the clients.

Cloud 237

Cloud Media Storage Cache 237

High Scalability

FEBRUARY 19, 2021

This means a system that is not merely available but is also engineered with extensive redundant measures to continue to work as its users expect. Fault tolerance The ability of a system to continue to be dependable (both available and reliable) in the presence of certain component or subsystem failures.

Engineering 215

Engineering Systems Scalability Availability 215

cdemi

DECEMBER 4, 2016

Apache Kafka - High-Throughput, Low-Latency, Uses Apache ZooKeeper for Distribution, Written in Scala and Java. Apache Kafka - High-Throughput, Low-Latency, Uses Apache ZooKeeper for Distribution, Written in Scala and Java. A choice of Message Broker requires careful consideration as each has their pros and cons.

Design 47

Design Azure Scalability Latency 47

Dynatrace

MAY 27, 2021

We’re excited to announce Dynatrace has been named as a select launch partner for a newly launched Amazon Web Services (AWS) offering, Amazon ECS Anywhere. This new extension allows customers to deploy native Amazon ECS tasks in any target environment including traditional AWS managed infrastructure and now customer-managed infrastructure.

AWS 131

AWS IoT Virtualization Infrastructure 131

Percona

JULY 1, 2023

Now that Database-as-a-service (DBaaS) is in high demand, there are multiple questions regarding AWS services that cannot always be answered easily: When should I use Aurora and when should I use RDS MySQL ? Amazon Aurora is a proprietary, cloud-native, fully managed relational database service developed by Amazon Web Services (AWS).

AWS 52

AWS Database Serverless Storage 52

Dynatrace

APRIL 7, 2021

But your infrastructure teams don’t see any issue on their AWS or Azure monitoring tools, your platform team doesn’t see anything too concerning in Kubernetes logging, and your apps team says there are green lights across the board. Imagine you’re in a war room. So, what happens next? Kiosks, mobile apps, websites, and QR codes.

Cloud 158

Cloud Monitoring Latency Games 158

The Netflix TechBlog

AUGUST 29, 2023

A brief history of IPC at Netflix Netflix was early to the cloud, particularly for large-scale companies: we began the migration in 2008, and by 2010, Netflix streaming was fully run on AWS. To improve availability, we designed systems where components could fail separately and avoid single points of failure.

Traffic 220

Traffic Latency Cloud C++ 220

The Netflix TechBlog

DECEMBER 21, 2020

By Anupom Syam Background At Netflix, our current data warehouse contains hundreds of Petabytes of data stored in AWS S3 , and each day we ingest and create additional Petabytes. This article will list some of the use cases of AutoOptimize, discuss the design principles that help enhance efficiency, and present the high-level architecture.

Storage 203

Storage Latency Efficiency Data Engineering 203

All Things Distributed

JULY 14, 2015

In this blog post, I will explain how these three new capabilities empower you to build applications with distributed systems architecture and create responsive, reliable, and high-performance applications using DynamoDB that work at any scale. Let me expand on each one of them. DynamoDB Streams.

Database 167

Database Lambda AWS IoT 167

The Netflix TechBlog

OCTOBER 27, 2020

The processed data is typically stored as data warehouse tables in AWS S3. At Netflix, we also heavily embrace a microservice architecture that emphasizes separation of concerns. The data warehouse is not designed to serve point requests from microservices with low latency.

Latency 243

Latency Storage Big Data Tuning 243

Dynatrace

OCTOBER 1, 2021

As dynamic systems architectures increase in complexity and scale, IT teams face mounting pressure to track and respond to conditions and issues across their multi-cloud environments. Dynatrace news. But what is observability? Why is it important, and what can it actually help organizations achieve? What is observability?

Metrics 363

Metrics Open Source Monitoring Infrastructure 363

The Morning Paper

FEBRUARY 6, 2020

On the Cloudburst design teams’ wish list: A running function’s ‘hot’ data should be kept physically nearby for low-latency access. The canononical cloud platform architecture decouples storage and compute services so that each can be scaled and operated independently, i.e., they are disaggregated. Consistency.

Lambda 98

Lambda Serverless Cache Latency 98

The Netflix TechBlog

FEBRUARY 20, 2019

Today we are excited to announce latency heatmaps and improved container support for our on-host monitoring solution?—?Vector?—?to Remotely view real-time process scheduler latency and tcp throughput with Vector and eBPF What is Vector? to the broader community. Vector is open source and in use by multiple companies.

Performance 211

Performance Latency Open Source Metrics 211

High Scalability

JULY 20, 2018

That’s mapping applications to the specific architectural choices. The third wing of the architecture piece is the “domain specific system-on-chip.” tef : You can use a message broker to glue systems together, but never use one to cut systems apart. There are a few more quotes.

Internet 121

Internet Internet Scalability Automotive 121

High Scalability

SEPTEMBER 8, 2018

RISELabs , those wonderfully innovative folks over at Berkeley, have uplifted their Anna datatabase —a shared-nothing, thread-per-core architecture to achieve lightning-fast speeds by avoiding all coordination mechanisms—to become cloud-aware. No, I don’t think that is because AWS is earning a 355x margin on DynamoDB!

Storage 199

Storage Performance AWS Efficiency 199

The Morning Paper

MAY 12, 2019

An open-source benchmark suite for microservices and their hardware-software implications for cloud & edge systems Gan et al., Systems built with lots of microservices have different operational characteristics to those built from a small number of monoliths, we’d like to study and better understand those differences.

Open Source 61

Open Source Hardware Benchmarking Systems 61