All Things Distributed

JANUARY 23, 2012

Expanding the Cloud - The AWS Storage Gateway. Today Amazon Web Services has launched the AWS Storage Gateway, making the power of secure and reliable cloud storage accessible from customersâ?? With the launch of the AWS Storage Gateway our customers can now integrate their on-premises IT environment with AWSâ??s

Storage 115

Storage AWS Virtualization Cloud 115

Percona

JUNE 22, 2023

Let’s dive in and learn how (and what) to effectively monitor MySQL performance, along with examples from PMM, by understanding the critical KPIs to watch for. This is not an exhaustive list but an example of what we can watch for.

Performance 108

Performance Monitoring Traffic Database 108

The Morning Paper

OCTOBER 4, 2020

We are standing on the eve of the 5G era… 5G, as a monumental shift in cellular communication technology, holds tremendous potential for spurring innovations across many vertical industries, with its promised multi-Gbps speed, sub-10 ms low latency, and massive connectivity. Throughput and latency. Application performance.

Energy 130

Energy Latency Performance Network 130

The Morning Paper

JANUARY 30, 2020

Edge servers are the middle ground – more compute power than a mobile device, but with latency of just a few ms. The kind of edge server envisaged here might, for example, be integrated with your WiFi access point. One example from the paper is an application using the ammo.js Why would we want to live migrate web workers?

Mobile 104

Mobile Cloud Latency Games 104

Percona

SEPTEMBER 1, 2023

This results in expedited query execution, reduced resource utilization, and more efficient exploitation of the available hardware resources. This reduction in latency ensures that applications and websites provide a more rapid and responsive user experience. Avoid over-indexing, which can bloat storage and slow writes.

Tuning 52

Tuning Database Performance Hardware 52

The Morning Paper

JUNE 13, 2019

As we saw with the SOAP paper last time out, even with a fixed model variant and hardware there are a lot of different ways to map a training workload over the available hardware. The following figure highlights how just one of these variables, batch size, impacts throughput and latency on ResNet50.

Hardware 58

Hardware Latency Serverless Energy 58

Dotcom-Montior

MAY 12, 2020

Websites are now more than just the storage and retrieval of information to present content to users. Network latency. Hardware resources. Network Latency. Network latency can be affected due to. Hardware Resources. Website and web application technologies have grown tremendously over the years. Wi-Fi usage.

Monitoring 63

Monitoring Entertainment Hardware Latency 63

Percona

JULY 1, 2023

Understanding DBaaS DBaaS cloud services allow users to use databases without configuring physical hardware and infrastructure or installing software. It also supports auto-scaling of compute and storage resources, dynamically adjusting capacity based on chosen utilization thresholds to optimize performance and costs.

AWS 52

AWS Database Serverless Storage 52

The Morning Paper

MARCH 17, 2020

higher latency and lower bandwidth)… We have found the actual behavior of Optane DIMMs to be more complicated and nuanced than the "slower, persistent DRAM" label would suggest. The read latency for Optane is 2x-3x higher than DRAM. Optane DIMMs are here! Optane memory is also more pattern-dependent than DRAM.

Scalability 59

Scalability Latency Cache Media 59

The Morning Paper

DECEMBER 8, 2019

Byte-addressable non-volatile memory,) NVM will fundamentally change the way hardware interacts, the way operating systems are designed, and the way applications operate on data. Therefore any programming abstraction must be low latency and the kernel needs to be kept off the path of persistent data access as much as possible.

Hardware 56

Hardware Programming Media Storage 56

Dotcom-Montior

OCTOBER 6, 2021

For example, a site reliability engineer may be involved with help desk tickets, on-call incidents, manual tasks, etc. trying to reduce the amount of manual work and ensuring all the components (infrastructure/hardware, middleware, software, etc.) What Does a Site Reliability Engineer Do? What are Some Common SRE Responsibilities?

Engineering 67

Engineering DevOps Monitoring Google 67

Abhishek Tiwari

DECEMBER 27, 2017

faster access to external storage and data locality (I/O, bandwidth). A good example will be something like Pachyderm [8] [9] - a relatively young project which implements a distributed Pachyderm File System (PFS) and a data-aware scheduler Pachyderm Pipeline System (PPS) on top of Kubernetes. Storage provisioning.

Big Data 63

Big Data Storage Benchmarking Hardware 63

Percona

APRIL 17, 2023

The CFQ works well for many general use cases but lacks latency guarantees. The deadline excels at latency-sensitive use cases ( like databases ), and noop is closer to no schedule at all. For example, on a 128GB memory host, this can allow up to 38.4GB of dirty pages. Two other schedulers are deadline and noop.

Best Practices 52

Best Practices Design Tuning Database 52

The Morning Paper

MAY 28, 2019

The goal is to produce a low-energy hardware classifier for embedded applications doing local processing of sensor data. The resulting system can integrate seamlessly into a scikit-learn based development process, and dramatically reduces the total energy usage required for classification with very low latency. Introducing race logic.

Energy 52

Energy Hardware Efficiency Architecture 52

SQL Server According to Bob

JANUARY 11, 2020

Stable Media Stable media is often confused with physical storage. SQL Server defines stable media as storage that can survive system restart or common failure. Stable media is commonly physical disk storage, but other devices and certain caching facilities qualify as well.

Servers 40

Servers Cache Media Hardware 40

Abhishek Tiwari

JULY 12, 2018

These nodes and edges require a good amount of compute and storage which is typically distributed across a large number servers either running in the cloud or your own data center. A data pipeline is a software which runs on hardware. The software is error-prone and hardware failures are inevitable.

Latency 63

Latency Analytics Scalability Engineering 63

The Morning Paper

MARCH 3, 2020

It takes you through the thinking processes and engineering practices behind the design of a key part of the control plane for AWS Elastic Block Storage (EBS): the Physalia database that stores configuration information. This work is latency critical, because volume IO is blocked until it is complete. This paper is a real joy to read.

Database 106

Database AWS Network Design 106

Highly Scalable

SEPTEMBER 18, 2012

Historically, NoSQL paid a lot of attention to tradeoffs between consistency, fault-tolerance and performance to serve geographically distributed systems, low-latency or highly available applications. A database should accommodate itself to different data distributions, cluster topologies and hardware configurations. Data Placement.

Database 213

Database Latency C++ Scalability 213

HammerDB

OCTOBER 12, 2018

As with the previous guides as an Intel employee (#IAMINTEL) the examples are taken from a MySQL 8 on Linux on Intel system and the approach is the same for whatever system you are testing although some of the settings you see may be different. an example is shown of the shortest of these: . hardware limits: 1000 MHz - 3.80

Best Practices 40

Best Practices Scalability Performance C++ 40

All Things Distributed

JULY 13, 2010

Further computationally intensive, highly parallel workloads have found their way to Amazon EC2 as businesses have explored using HPC types of algorithms for other application categories, for example to to process very large unstructured data sets for Business Intelligence applications. Driving Storage Costs Down for AWS Customers.

Cloud 118

Cloud AWS Automotive Latency 118

HammerDB

DECEMBER 5, 2020

For example HammerDB has not used tpmC terminology to report TPC-C based metrics instead using TPM and NOPM nomenclature. A full understanding of why this is important requires some knowledge of the evolution of database hardware and software. This was both expensive and time consuming to configure.

C++ 40

C++ Benchmarking Virtualization Open Source 40

Alex Russell

MARCH 6, 2021

A then-representative $200USD device had 4-8 slow (in-order, low-cache) cores, ~2GiB of RAM, and relatively slow MLC NAND flash storage. The Moto G4 , for example. Hardware Past As Performance Prologue. Sadly, data on latency is harder to get, even from Google's perch, so progress there is somewhat more difficult to judge.

Performance 124

Performance Network Cache Metrics 124

Adrian Cockcroft

NOVEMBER 11, 2019

This discussion focuses on hardware, software and operational failure modes. Collecting some critical metrics at one second intervals, with a total observability latency of ten seconds or less matches the human attention span much better. We’ll use FMEA to work through the sign-up flow example in a later section.

Latency 52

Latency Engineering Systems Hardware 52

Alex Russell

APRIL 29, 2021

Chrome has missed several APIs for 3+ years: Storage Access API. For heavily latency-sensitive use-cases like WebXR, this is a critical component in delivering a good experience. A tool for reliably sending data — for example, chat and email messages — in the face of intermittent network connections. Where Chrome Has Lagged.

Media 145

Media Games Education Engineering 145

Adrian Cockcroft

NOVEMBER 11, 2019

This discussion focuses on hardware, software and operational failure modes. Collecting some critical metrics at one second intervals, with a total observability latency of ten seconds or less matches the human attention span much better. We’ll use FMEA to work through the sign-up flow example in a later section.

Latency 53

Latency Engineering Systems Hardware 53

SQL Server According to Bob

JANUARY 11, 2020

As ​​ a ​​ simple ​​ example, ​​ take the following ​​ 2-byte string value of “AB” and apply a formula to it (Value = Position + ASCII value). ​​ Checksum ​​ is ​​ the sum of the values.

Servers 40

Servers Cache Database Media 40

Dynatrace

AUGUST 5, 2019

Lift & Shift is where you basically just move physical or virtual hosts to the cloud – essentially you just run your host on somebody else’s hardware. Remember: This is a critical aspect as you do not want to migrate a service and suddenly introduce high latency or costs to a system that you forgot about having a dependency with!

Cloud 31

Cloud Traffic Database Network 31

Alex Russell

OCTOBER 22, 2017

If our example document wasn’t reliant on JavaScript to construct the <my-app> custom element, the contents of the document would likely be interactive as soon as enough CSS and content was available to render meaningfully. Simulated packet loss and variable latency, however, can make benchmarking extremely difficult and slow.

Performance 68

Performance Benchmarking Network Mobile 68

Highly Scalable

AUGUST 20, 2013

This system has been designed to supplement and succeed the existing Hadoop-based system that had too high latency of data processing and too high maintenance costs. The pipelines can be stateful and the engine’s middleware should provide a persistent storage to enable state checkpointing. Interoperability with Hadoop.

Big Data 154

Big Data Processing Lambda Database 154

Brendan Gregg

SEPTEMBER 5, 2017

Here's some output from my zfsdist tool, in bcc/BPF, which measures ZFS latency as a histogram on Linux: # zfsdist. Tracing ZFS operation latency. For example, although it's very minor, the /dev/*random devices were faster on Solaris last time I checked, as Linux was spending more effort on entropy for improving security. (Or,

Virtualization 76

Virtualization AWS Engineering Hardware 76

Smashing Magazine

JANUARY 6, 2020

Run performance experiments and measure outcomes — both on mobile and on desktop (for example, with Google Analytics ). Rather than focusing on full page loading time (via onLoad and DOMContentLoaded timings, for example), prioritize page loading as perceived by your customers. What does it mean?

Performance 145

Performance Cache Network Metrics 145

Smashing Magazine

JANUARY 7, 2019

One way or another, rather than focusing on full page loading time (via onLoad and DOMContentLoaded timings, for example), prioritize page loading as perceived by your customers. Estimated Input Latency tells us if we are hitting that threshold, and ideally, it should be below 50ms.

Performance 103

Performance Cache Metrics Network 103

Smashing Magazine

JANUARY 11, 2021

For example, to make performance more tangible, you could expose the revenue performance impact by showing the correlation between the conversion rate and time to application load, as well as rendering performance. Run performance experiments and measure outcomes — both on mobile and on desktop (for example, with Google Analytics ).

Performance 145

Performance Cache Media Metrics 145

The Netflix TechBlog

FEBRUARY 9, 2021

Server-generated assets, since client-side generation would require the retrieval of many individual images, which would increase latency and time-to-render. To reduce latency, assets should be generated in an offline fashion and not in real time. This requires an asset storage solution. We call these adaptive fields.

Engineering 165

Engineering Storage Latency Entertainment 165

All Things Distributed

MARCH 11, 2016

This is a given, whether you are using the highest quality hardware or lowest cost components. This becomes an even more important lesson at scale: for example, as S3 processes trillions and trillions of storage transactions, anything that has even the slightest probability of error will become realistic.

AWS 144

AWS Hardware Retail Virtualization 144

Alex Russell

JULY 14, 2021

Here, native apps are doing work related to their core function; storage and tracking of user data are squarely within the four corners of the app's natural responsibilities. Hardware access APIs, notably: Geolocation. Nathan Schloss @n8Schloss @slightlylate @gsnedders @ericlaw @jyasskin Good example! Web Bluetooth. Web Serial.

Google 121

Google Mobile Engineering Internet 121

Abhishek Tiwari

DECEMBER 27, 2017

It offers reliability and performance of a data warehouse, real-time and low-latency characteristics of a streaming system, and scale and cost-efficiency of a data lake. Databricks Delta is a perfect example of this class. In contrast, Alluxio a middleware for data access - think Alluxio storage layer as fast cache.

Big Data 49

Big Data Artificial Intelligence Storage Hardware 49