All Things Distributed

JUNE 20, 2017

Today, I am very excited to announce our plans to open a new AWS Region in Hong Kong! The new region will give Hong Kong-based businesses, government organizations, non-profits, and global companies with customers in Hong Kong, the ability to leverage AWS technologies from data centers in Hong Kong.

AWS 146

AWS Logistics Cloud Social Media 146

Scalegrid

MARCH 14, 2024

Providers such as AWS Outposts, Microsoft Azure Stack, Google Anthos, and VMware Cloud on AWS offer compatible private clouds that integrate seamlessly with established public cloud platforms — a key feature for constructing hybrid cloud systems. Elevate your cloud strategy today with ScaleGrid!

Strategy 130

Strategy Cloud Artificial Intelligence Infrastructure 130

High Scalability

SEPTEMBER 8, 2018

New databases used to be announced seemingly every week. While database neogenesis has slowed down considerably, it has not gone necrotic. No, I don’t think that is because AWS is earning a 355x margin on DynamoDB! Our monitoring engine automatically moves data between tiers based on access patterns.

Storage 199

Storage Performance AWS Efficiency 199

Percona

MAY 2, 2023

That’s why it’s essential to implement the best practices and strategies for MongoDB database backups. Why are MongoDB database backups important? Regular database backups are essential to protect against data loss caused by system failures, human errors, natural disasters, or cyber-attacks.

Best Practices 52

Best Practices Database Storage Servers 52

The Netflix TechBlog

OCTOBER 27, 2020

Data scientists and engineers collect this data from our subscribers and videos, and implement data analytics models to discover customer behaviour with the goal of maximizing user joy. The processed data is typically stored as data warehouse tables in AWS S3. How Bulldozer leverages Spark, Protobuf and KV DAL for moving the data.

Latency 243

Latency Storage Big Data Tuning 243

The Netflix TechBlog

OCTOBER 19, 2020

a Netflix member via Twitter This is an example of a question our on-call engineers need to answer to help resolve a member issue?—?which We needed to increase engineering productivity via distributed request tracing. That is the first question our engineering teams asked us when integrating the tracer library.

Infrastructure 233

Infrastructure Transportation Storage Open Source 233

Adrian Cockcroft

NOVEMBER 18, 2019

AWS Developer Relations on how the shift from Robot Operating System (ROS) 1 to ROS 2 will change the landscape for all robot lovers. OPN205-R Contributing to the AWS Construct Library (repeats) Using and loving the AWS Cloud Development Kit and want to help make it better? Alejandra Olvera-Novack?—?AWS

Open Source 40

Open Source AWS Lambda Serverless 40

Adrian Cockcroft

NOVEMBER 18, 2019

AWS Developer Relations on how the shift from Robot Operating System (ROS) 1 to ROS 2 will change the landscape for all robot lovers. OPN205-R Contributing to the AWS Construct Library (repeats) Using and loving the AWS Cloud Development Kit and want to help make it better? Alejandra Olvera-Novack?—?AWS

Open Source 40

Open Source AWS Lambda Serverless 40

The Netflix TechBlog

FEBRUARY 18, 2022

Personalized Experience Refresh Netflix Recommendation engine continuously refreshes recommendations for every member. Scaling Policies To address the thundering herd problem and to keep latencies under acceptable thresholds, the cluster scale-up policies are configured to be more aggressive than the scale-down policies.

Systems 334

Systems Traffic Architecture Mobile 334

Brendan Gregg

SEPTEMBER 25, 2021

A Cassandra database cluster had switched to Ubuntu and noticed write latency increased by over 30%. top(1) showed that only the Cassandra database was consuming CPU. I happened to be speaking at a technical confering while still debugging this, and mentioned what I was working on to a processor engineer.

Speed 126

Speed Java AWS Virtualization 126

Scalegrid

JANUARY 12, 2024

Simply put, it’s the set of computational tasks that cloud systems perform, such as hosting databases, enabling collaboration tools, or running compute-intensive algorithms. What is workload in cloud computing? This trend features running multiple distinct variants simultaneously, each relying on services delivered by various vendors.

Cloud 130

Cloud Virtualization Storage Efficiency 130

The Netflix TechBlog

DECEMBER 21, 2020

By Anupom Syam Background At Netflix, our current data warehouse contains hundreds of Petabytes of data stored in AWS S3 , and each day we ingest and create additional Petabytes. Sometimes Data Engineers write downstream ETLs on ingested data to optimize the data/metadata layouts to make other ETL processes cheaper and faster.

Storage 203

Storage Latency Efficiency Data Engineering 203

The Morning Paper

FEBRUARY 6, 2020

On the Cloudburst design teams’ wish list: A running function’s ‘hot’ data should be kept physically nearby for low-latency access. Only now the database is itself a distributed KVS, and the slices of application functionality are much finer-grained. Updates should be allowed at any function invocation site.

Lambda 98

Lambda Serverless Cache Latency 98

Dynatrace

SEPTEMBER 30, 2021

Cloud services platforms like AWS, Azure, and GCP are reshaping how organizations deliver value to their customers, making cloud migration an increasingly attractive option for running applications. This can dramatically decrease network latency and its effect on the end-user experience. Migrate databases intelligently.

Cloud 160

Cloud Traffic Best Practices Strategy 160

All Things Distributed

NOVEMBER 26, 2013

As I discussed in my re:Invent keynote earlier this month, I am now happy to announce the immediate availability of Amazon RDS Cross Region Read Replicas , which is another important enhancement for our customers using or planning to use multiple AWS Regions to deploy their applications. Cross Region Read Replicas are available for MySQL 5.6

Cloud 145

Cloud AWS Traffic Latency 145

High Scalability

DECEMBER 21, 2018

2) Cloud will decentralise in terms of provision not power i.e. Amazon will "invade" more of those holdouts with AWS Outpost. Whether it’s database or message queues it’s a really weird combo of licenses and features for hostage. And if you know someone with hearing problems they might find Live CC useful.

Internet 137

Internet Internet Scalability Serverless 137

The Netflix TechBlog

AUGUST 13, 2020

Over the years we’ve learned from on-call engineers about the pain points of application monitoring: too many alerts, too many dashboards to scroll through, and too much configuration and maintenance. It usually has dependencies, talks to other services, and lives in different AWS regions. Client metrics and QoE changes.

Monitoring 210

Monitoring Tuning Traffic Metrics 210

All Things Distributed

SEPTEMBER 25, 2012

Following the huge success of being able to provision a consistent, user-requested I/O rate for DynamoDB and Elastic Block Store (EBS), the AWS Database Services team has now released Provisioned IOPS, a new high performance storage option for the Amazon Relational Database Service (Amazon RDS). Reducing Complexity.

Cloud 117

Cloud AWS Storage Database 117

The Netflix TechBlog

DECEMBER 17, 2019

Andreas Andreakis , Ioannis Papapanagiotou Overview Change-Data-Capture (CDC) allows capturing committed changes from a database in real-time and propagating those changes to downstream consumers [1][2]. In databases like MySQL and PostgreSQL, transaction logs are the source of CDC events. Designed with High Availability in mind.

Database 197

Database Traffic Transportation Open Source 197

The Netflix TechBlog

DECEMBER 17, 2019

Andreas Andreakis , Ioannis Papapanagiotou Overview Change-Data-Capture (CDC) allows capturing committed changes from a database in real-time and propagating those changes to downstream consumers [1][2]. In databases like MySQL and PostgreSQL, transaction logs are the source of CDC events. Designed with High Availability in mind.

Database 212

Database Traffic Transportation Open Source 212

Smashing Magazine

AUGUST 3, 2023

These pages serve as a pivotal tool in our digital marketing strategy, not only providing valuable information about our services but also designed to be easily discoverable through search engines. Large preview ) We’ve known for a long time that fast page performance influences search engine rankings. SEO is key to our success.

Performance 74

Performance Cache Traffic Metrics 74

All Things Distributed

OCTOBER 25, 2013

Many of you know Thorsten von Eicken as the founder of Rightscale , the company that has helped numerous organizations find their way onto AWS. The lack of low-latency made that distributed systems (e.g. database replication, fault tolerance protocols) could not benefit from these advances at the network level.

Network 96

Network Latency Operating System Speed 96

Percona

APRIL 17, 2023

Note that the intent of tuning the settings is not exclusively about improving performance but also enhancing the high availability and resilience of the MongoDB database. The Linux default is usually 60 , which is not ideal for database usage. The CFQ works well for many general use cases but lacks latency guarantees.

Best Practices 52

Best Practices Design Tuning Database 52

All Things Distributed

APRIL 28, 2010

Expanding the Cloud - Opening the AWS Asia Pacific (Singapore) Region. Today Amazon Web Services has taken another important step in serving customers worldwide: the AWS Asia Pacific (Singapore) Region is now launched. The new Singapore Region offers customers in APAC lower-latency access to AWS services. Comments ().

AWS 61

AWS Cloud Latency Storage 61

Brendan Gregg

SEPTEMBER 25, 2021

A Cassandra database cluster had switched to Ubuntu and noticed write latency increased by over 30%. top(1) showed that only the Cassandra database was consuming CPU. I happened to be speaking at a technical confering while still debugging this, and mentioned what I was working on to a processor engineer.

Speed 52

Speed Java AWS Virtualization 52

All Things Distributed

FEBRUARY 22, 2012

Today AWS launched an exciting new service for developers: the Amazon Simple Workflow Service. They must deal with the increased latency and unreliability inherent in remote communication. Tasks can be long-running, may fail, may timeout and may complete with varying throughputs and latencies. Expanding the Cloud â??

Cloud 130

Cloud AWS Java Scalability 130

Brendan Gregg

SEPTEMBER 25, 2021

A Cassandra database cluster had switched to Ubuntu and noticed write latency increased by over 30%. top(1) showed that only the Cassandra database was consuming CPU. I happened to be speaking at a technical confering while still debugging this, and mentioned what I was working on to a processor engineer.

Speed 40

Speed Java AWS Virtualization 40

The Netflix TechBlog

MARCH 6, 2019

Mounting object storage in Netflix’s media processing platform By Barak Alon (on behalf of Netflix’s Media Cloud Engineering team) MezzFS (short for “Mezzanine File System”) is a tool we’ve developed at Netflix that mounts cloud objects as local files via FUSE. Netflix operates in multiple AWS regions. Regional caching? —?Netflix

Media 214

Media Storage Processing Cache 214

All Things Distributed

JULY 13, 2010

In particular this has been true for applications based on algorithms - often MPI-based - that depend on frequent low-latency communication and/or require significant cross sectional bandwidth. Driving Storage Costs Down for AWS Customers. Expanding the Cloud - The AWS Storage Gateway. Countdown to What is Next in AWS.

Cloud 118

Cloud AWS Automotive Latency 118

All Things Distributed

DECEMBER 5, 2010

Route 53 has the business properties that you have come to expect from an AWS service: fully self-service and programmable, with transparent pay-as-you-go pricing and no minimum usage commitments. This achieves very low-latency for queries which is crucial for the overall performance of internet applications. No lock-in. Contact Info.

Cloud 117

Cloud Internet Internet AWS 117

All Things Distributed

NOVEMBER 19, 2010

Science & Engineering. Understanding Throughput-Oriented Architectures - background article in CACM on massively parallel and throughput vs latency oriented architectures. an engineering adventure to break the 1,000 mph barrier in a car. Driving Storage Costs Down for AWS Customers. Countdown to What is Next in AWS.

AWS 67

AWS Cloud Benchmarking Storage 67

Adrian Cockcroft

NOVEMBER 11, 2019

In addition to the common financial calculation of risk as the product of probability and severity, engineering risk includes detectability. A learning organization, disaster recovery testing, game days, and chaos engineering tools are all important components of a continuously resilient system.

Latency 52

Latency Engineering Systems Hardware 52

Adrian Cockcroft

NOVEMBER 11, 2019

In addition to the common financial calculation of risk as the product of probability and severity, engineering risk includes detectability. A learning organization, disaster recovery testing, game days, and chaos engineering tools are all important components of a continuously resilient system.

Latency 53

Latency Engineering Systems Hardware 53

All Things Distributed

JULY 14, 2015

I am excited to share with you that today we are expanding DynamoDB with streams, cross-region replication, and database triggers. In traditional database architectures, database engines often run a small search engine or data warehouse engines on the same hardware as the database. DynamoDB Streams.

Database 167

Database Lambda AWS IoT 167

All Things Distributed

APRIL 4, 2017

Today, I am very excited to announce our plans to open a new AWS Region in the Nordics! The new region will give Nordic-based businesses, government organisations, non-profits, and global companies with customers in the Nordics, the ability to leverage the AWS technology infrastructure from data centers in Sweden.

AWS 116

AWS Airlines Latency Games 116

All Things Distributed

JUNE 21, 2018

A common question that I get is why do we offer so many database products? To do this, they need to be able to use multiple databases and data models within the same application. Seldom can one database fit the needs of multiple distinct use cases. Seldom can one database fit the needs of multiple distinct use cases.

Database 167

Database AWS Games Latency 167

All Things Distributed

JANUARY 18, 2012

a Fast and Scalable NoSQL Database Service Designed for Internet Scale Applications. Today is a very exciting day as we release Amazon DynamoDB , a fast, highly reliable and cost-effective NoSQL database service designed for internet scale applications. Amazon DynamoDB offers low, predictable latencies at any scale. Comments ().

Scalability 167

Scalability Database Ecommerce Latency 167