All Things Distributed

JUNE 7, 2017

Unlocking the value of data is a primary goal that AWS helps our customers to pursue. Some applications – medical equipment, industrial machinery, and building automation are just a few – can't rely exclusively on the cloud for control, and require some form of local storage and execution. Law of the Land.

AWS 118

AWS IoT Lambda Storage 118

Brendan Gregg

JULY 4, 2021

It's an exciting time for developments in computer performance, not just for the BPF technology (which I often [write about]) but also for processors with 3D stacking and cloud vendor CPUs (e.g., Ford, et al., “TCP on Upcoming Sapphire Rapids CPUs,” [link] Oct 2020 - [Liu 20] Linda Liu, “Samsung QVO vs EVO vs PRO: What’s the Difference?

Performance 140

Performance Latency Hardware Storage 140

The Netflix TechBlog

OCTOBER 27, 2020

Usually Data scientists and engineers write Extract-Transform-Load (ETL) jobs and pipelines using big data compute technologies, like Spark or Presto , to process this data and periodically compute key information for a member or a video. The processed data is typically stored as data warehouse tables in AWS S3.

Latency 243

Latency Storage Big Data Tuning 243

Scalegrid

MARCH 14, 2024

Integrating technology from private and public clouds and on-premises resources within one hybrid cloud platform creates an integrated IT infrastructure that leverages the strengths of each component. We will examine each of these elements in more detail.

Strategy 130

Strategy Cloud Artificial Intelligence Infrastructure 130

Scalegrid

JANUARY 12, 2024

These include popular technologies such as web servers and web applications, along with advanced solutions like distributed data stores and containerized microservices. Storage is a critical aspect to consider when working with cloud workloads. Storage is a critical aspect to consider when working with cloud workloads.

Cloud 130

Cloud Virtualization Storage Efficiency 130

Scalegrid

JANUARY 8, 2024

Discover how AI is reshaping the cloud and what this means for the future of technology. Infrastructure Excellence ScaleGrid’s infrastructure is designed to facilitate hosting in your cloud account and provides cost-saving options with AWS or Azure Reserved Instances or GCP.

Artificial Intelligence 147

Artificial Intelligence Cloud Scalability Analytics 147

Brendan Gregg

FEBRUARY 28, 2023

My personal opinion is that I don't see a widespread need for more capacity given horizontal scaling and servers that can already exceed 1 Tbyte of DRAM; bandwidth is also helpful, but I'd be concerned about the increased latency for adding a hop to more memory. It's an important vendor-neutral space to share the latest in technology.

Performance 110

Performance Latency Cache Virtualization 110

All Things Distributed

DECEMBER 14, 2011

Introducing the AWS South America (Sao Paulo) Region. Today, Amazon Web Services is expanding its worldwide coverage with the launch of a new AWS Region in Sao Paulo, Brazil. Several prominent South American customers have been using AWS since the early days. RAMA , a Brazilian financial services firm and AWS customer : â??The

AWS 114

AWS Latency Storage Big Data 114

All Things Distributed

MARCH 2, 2011

Expanding the Cloud - Introducing the AWS Asia Pacific (Tokyo) Region. Today Amazon Web Services is expanding its world-wide coverage with the launch of a new AWS Region located in Tokyo, Japan. The advanced Asia Pacific network infrastructure also makes the AWS Tokyo Region a viable low-latency option for customers from South Korea.

AWS 112

AWS Cloud Games Latency 112

The Netflix TechBlog

NOVEMBER 22, 2019

by Shefali Vyas Dalal AWS re:Invent is a couple weeks away and our engineers & leaders are thrilled to be in attendance yet again this year! Technology advancements in content creation and consumption have also increased its data footprint. Please stop by our “Living Room” for an opportunity to connect or reconnect with Netflixers.

AWS 37

AWS Entertainment Open Source Benchmarking 37

All Things Distributed

AUGUST 20, 2015

Today, we are releasing a plugin that allows customers to use the Titan graph engine with Amazon DynamoDB as the backend storage layer. It opens up the possibility to enjoy the value that graph databases bring to relationship-centric use cases, without worrying about managing the underlying storage. The importance of relationships.

Database 134

Database Logistics Availability Social Media 134

Brendan Gregg

JULY 4, 2021

It's an exciting time for developments in computer performance, not just for the BPF technology (which I often [write about]) but also for processors with 3D stacking and cloud vendor CPUs (e.g., Ford, et al., “TCP on Upcoming Sapphire Rapids CPUs,” [link] Oct 2020 - [Liu 20] Linda Liu, “Samsung QVO vs EVO vs PRO: What’s the Difference?

Performance 52

Performance Latency Hardware Storage 52

All Things Distributed

AUGUST 16, 2011

Expanding the Cloud - The AWS GovCloud (US) Region. Today AWS announced the launch of the AWS GovCloud (US) Region. The concept of regions gives AWS customers control over the placement of their resources and services. All Things Distributed. Werner Vogels weblog on building scalable and robust distributed systems.

AWS 111

AWS Government Big Data Cloud 111

All Things Distributed

OCTOBER 2, 2017

The Dynamo paper was well-received and served as a catalyst to create the category of distributed database technologies commonly known today as "NoSQL." " Of course, no technology change happens in isolation, and at the same time NoSQL was evolving, so was cloud computing.

Internet 128

Internet Internet AWS Performance 128

All Things Distributed

APRIL 9, 2015

At Amazon we have hundreds of teams using machine learning and by making use of the Machine Learning Service we can significantly speed up the time they use to bring their technologies into production. Details on the AWS Blog. AWS has been offering a range of storage solutions: objects, block storage, databases, archiving, etc.

Lambda 122

Lambda Cloud IoT AWS 122

High Scalability

JULY 20, 2018

DonHopkins : NeWS differs from the current technology stack in that it was all coherently designed at once by James Gosling and David Rosenthal, by taking several steps back and thinking deeply about all the different problems it was trying to solve together. lowrykoz : Stolen from a co-worker "Every company has a test environment.

Internet 121

Internet Internet Scalability Automotive 121

Brendan Gregg

FEBRUARY 28, 2023

My personal opinion is that I don't see a widespread need for more capacity given horizontal scaling and servers that can already exceed 1 Tbyte of DRAM; bandwidth is also helpful, but I'd be concerned about the increased latency for adding a hop to more memory. It was a great privilege. Ford, et al., “TCP

Performance 52

Performance Latency Cache Virtualization 52

All Things Distributed

JANUARY 18, 2012

s Dynamo technology , which was one of the first non-relational databases developed at Amazon. Amazon DynamoDB offers low, predictable latencies at any scale. This was not our technology vendorsâ?? Dynamo might have been the best technology in the world at the time but it was still software you had to run yourself.

Scalability 167

Scalability Database Ecommerce Latency 167

All Things Distributed

DECEMBER 3, 2009

Expanding the Cloud - New AWS Region: US-West (Northern California). We have expanded the AWS footprint in the US and starting today a new AWS Region is available for use: US-West (Northern California). AWS is committed to making its services available at low cost. Driving Storage Costs Down for AWS Customers.

AWS 60

AWS Cloud Latency Storage 60

All Things Distributed

NOVEMBER 29, 2017

Though the AWS Cloud gives you access to the storage and processing power required for ML, the process for building, training, and deploying ML models has unique challenges that often block successful use of this powerful new technology. At AWS, we believe in giving choices, so Amazon SageMaker removes that problem.

Tuning 94

Tuning AWS Scalability Infrastructure 94

All Things Distributed

APRIL 28, 2010

Expanding the Cloud - Opening the AWS Asia Pacific (Singapore) Region. Today Amazon Web Services has taken another important step in serving customers worldwide: the AWS Asia Pacific (Singapore) Region is now launched. The new Singapore Region offers customers in APAC lower-latency access to AWS services. Comments ().

AWS 61

AWS Cloud Latency Storage 61

Percona

MAY 2, 2023

The speed of backup also depends on allocated IOPS and type of storage since lots of read/writes would be happening during this process. Back up anywhere – to the cloud (use any S3-compatible storage) or on-premise with a locally-mounted remote file system It allows you to choose which compression algorithms to use.

Best Practices 52

Best Practices Database Storage Servers 52

All Things Distributed

NOVEMBER 15, 2010

For example, the most fundamental abstraction trade-off has always been latency versus throughput. Modern CPUs strongly favor lower latency of operations with clock cycles in the nanoseconds and we have built general purpose software architectures that can exploit these low latencies very well. Where to go from here?

AWS 136

AWS Latency Programming Architecture 136

All Things Distributed

JULY 13, 2010

During my academic career, I spent many years working on HPC technologies such as user-level networking interfaces, large scale high-speed interconnects, HPC software stacks, etc. When instances are placed in a cluster they have access to low latency, non-blocking 10 Gbps networking when communicating the other instances in the cluster.

Cloud 118

Cloud AWS Automotive Latency 118

O'Reilly

DECEMBER 2, 2019

For the past year, 5G cell technology has generated a lot of excitement–and a lot of hype. If carriers won’t build cell towers for current technology, they aren’t likely to build even more towers for 5G. There are impressive estimates for latency for 5G, but reality has a tendency to be harsh on such predictions.

Wireless 53

Wireless Serverless IoT Artificial Intelligence 53

All Things Distributed

DECEMBER 5, 2010

Route 53 has the business properties that you have come to expect from an AWS service: fully self-service and programmable, with transparent pay-as-you-go pricing and no minimum usage commitments. The Domain Name System is a wonderful practical piece of technology; it is a fundamental building block of our modern internet. No lock-in.

Cloud 117

Cloud Internet Internet AWS 117

All Things Distributed

JULY 11, 2011

As a part of that process, we also realized that there were a number of latency sensitive or location specific use cases like Hadoop, HPC, and testing that would be ideal for Spot. To get started using Spot or for more details visit the Amazon EC2 Spot Instance web page, the AWS developer blog , and the EC2 Release Notes. Contact Info.

AWS 85

AWS Storage Cloud Big Data 85

All Things Distributed

NOVEMBER 19, 2010

Understanding Throughput-Oriented Architectures - background article in CACM on massively parallel and throughput vs latency oriented architectures. Driving Storage Costs Down for AWS Customers. Expanding the Cloud - The AWS Storage Gateway. Countdown to What is Next in AWS. All postings. Recent Entries.

AWS 67

AWS Cloud Benchmarking Storage 67

O'Reilly

NOVEMBER 12, 2019

When we asked if respondents’ organizations had adopted serverless (defining “adopted” as entering into a contract with a vendor to provide serverless resources), we expected a low take rate for this relatively new and developing technology. latency, startup, mocking, etc.) changing the integration landscape—at least for now.

Serverless 88

Serverless Architecture FinTech Infrastructure 88

All Things Distributed

FEBRUARY 24, 2010

Achieving strict consistency can come at a cost in update or read latency, and may result in lower throughput. Amazon S3 and Amazon SimpleDB are examples of AWS services that are designed using these high-availability principles using an eventual consistency model. Lowest read latency. Higher read latency. No stale reads.

AWS 69

AWS Latency Database Scalability 69

Brendan Gregg

SEPTEMBER 5, 2017

Here's some output from my zfsdist tool, in bcc/BPF, which measures ZFS latency as a histogram on Linux: # zfsdist. Tracing ZFS operation latency. And namespaces, used for Linux containers, are also a relevant technology. I suspect at some point there'll be more jobs supporting the AWS cloud than there will be supporting Linux.

Virtualization 76

Virtualization AWS Engineering Hardware 76

Dynatrace

AUGUST 5, 2019

In this role, I am leading a global team that works closely with our strategic partners such as AWS, Microsoft, Google, Pivotal, Red Hat and others. It covers these key areas: Technology & Dependency Analysis. Step 1: Get to Know your Technology & Service Stack. Which technologies are candidates to be moved?

Cloud 31

Cloud Traffic Database Network 31

Smashing Magazine

JANUARY 7, 2019

Estimated Input Latency tells us if we are hitting that threshold, and ideally, it should be below 50ms. The seemingly obvious benefit of these technologies for users is guaranteed performance , so at times they might even prefer AMP-/Apple News/Instant Pages-links over "regular" and potentially bloated pages.

Performance 104

Performance Cache Network Metrics 104

Smashing Magazine

JANUARY 11, 2021

Estimated Input Latency tells us if we are hitting that threshold, and ideally, it should be below 50ms. Designed for the modern web, it responds to actual congestion, rather than packet loss like TCP does, it is significantly faster , with higher throughput and lower latency — and the algorithm works differently.

Performance 145

Performance Cache Media Metrics 145

Smashing Magazine

JANUARY 6, 2020

Estimated Input Latency tells us if we are hitting that threshold, and ideally, it should be below 50ms. Designed for the modern web, it responds to actual congestion, rather than packet loss like TCP does, it is significantly faster , with higher throughput and lower latency — and the algorithm works differently.

Performance 145

Performance Cache Network Metrics 145