Brendan Gregg

FEBRUARY 28, 2023

Titus, the Netflix container management platform, is now open source,” [link] Apr 2018 - [Cutress 19] Dr. [link] 2021 - [Trader 21] Tiffany Trader, “Cerebras Second-Gen 7nm Wafer Scale Engine Doubles AI Performance Over First-Gen Chip ,” [link] Apr 2021 - [Ghigoff 21] Yoann Ghigoff, et al., "BMC:

Performance 110

Performance Latency Cache Virtualization 110

Brendan Gregg

APRIL 14, 2022

The Netflix stack is more diverse than I was expecting, and is explained in detail in the [Netflix tech blog]: The production cloud is AWS EC2, Ubuntu Linux, Intel x86, mostly Java with some Node.js (and other languages), microservices, Cassandra (storage), EVCache (caching), Spinnaker (deployment), Titus (containers), Apache Spark (analytics), Atlas (..)

Java 145

Java Cloud Cache AWS 145

Uber Engineering

FEBRUARY 22, 2018

For context, we deployed more than 40 Schemaless instances and many thousands of storage nodes in 2016 alone. In 2014, Uber Engineering built Schemaless , our fault-tolerant and scalable datastore, to facilitate the rapid growth of our company.

Code 94

Code Storage Scalability Engineering 94

CSS Wizardry

MARCH 3, 2019

Tip: There are a number of directives that Clear-Site-Data will accept: "cookies" , "storage" , "executionContexts" , and "*" (which, naturally, means ‘all of the above’). a CDN), to always serve the freshest possible copy, and not to persist anything to storage. Jake Archibald , 2016. Patrick McManus , 2016.

Cache 264

Cache Latency Strategy Servers 264

All Things Distributed

MAY 19, 2017

That database engine is now known as Amazon Aurora and launched in 2014 for MySQL and in 2016 for PostgreSQL. We believe the central constraint in high throughput data processing has moved from compute and storage to the network.

Database 120

Database Design Cloud Storage 120

Brendan Gregg

JULY 4, 2021

AWS Graviton2); for memory with the arrival of DDR5 and High Bandwidth Memory (HBM) on-processor; for storage including new uses for 3D Xpoint as a 3D NAND accelerator; for networking with the rise of QUIC and eXpress Data Path (XDP); and so on.

Performance 52

Performance Latency Hardware Storage 52

Brendan Gregg

FEBRUARY 28, 2023

Titus, the Netflix container management platform, is now open source,” [link] , Apr 2018 [Cutress 19] Dr. [link] , 2021 [Trader 21] Tiffany Trader, “Cerebras Second-Gen 7nm Wafer Scale Engine Doubles AI Performance Over First-Gen Chip ,” [link] , Apr 2021 [Ghigoff 21] Yoann Ghigoff, et al.,

Performance 52

Performance Latency Cache Virtualization 52

All Things Distributed

OCTOBER 17, 2016

In the remainder of 2016 and inq 2017, we will launch another four AWS regions in Canada, China, the United Kingdom, and France, adding another nine AZs to our global infrastructure footprint. It brings the worldwide total of AWS Availability Zones (AZs) to 38, and the number of regions globally to 14.

AWS 127

AWS Cloud Infrastructure Analytics 127

All Things Distributed

DECEMBER 8, 2016

It relies on the scale and power of Amazon Simple Storage Service (Amazon S3) to deliver in-region storage options to businesses and organizations across the world in Canada, Japan, Singapore, Australia, Ireland, Germany, and the U.S., Par Werner Vogels, le 8 décembre 2016. as part of its Box Zones ecosystem.

AWS 155

AWS Cloud Lambda Innovation 155

Dynatrace

OCTOBER 15, 2021

The data is incredibly plentiful and difficult to store over long periods due to capacity limitations — a reason why private and public cloud storage services have been a boon to DevOps teams. This information is gathered from remote, often inaccessible points within your ecosystem and processed by some sort of tool or equipment.

Open Source 224

Open Source Metrics Cloud Transportation 224

SQL Performance

MAY 1, 2020

In a recent tip , I described a scenario where a SQL Server 2016 instance seemed to be struggling with checkpoint times. I was a bit perplexed by this issue, since the system was certainly no slouch — plenty of cores, 3TB of memory, and XtremIO storage. SQL 2016 – It Just Runs Faster: Indirect Checkpoint Default.

C++ 58

C++ Database Servers Storage 58

Percona

SEPTEMBER 19, 2023

This issue most often occurs when there is significant MySQL server activity during the backup operation, and the redo log file storage media operates faster than the backup storage media. Copyright (c) 2016, 2023, Oracle and/or its affiliates. root@ip-xxx-xx-xx-xx ~]# mysqlsh -uroot -p MySQL Shell 8.0.33 Type 'help' or '?'

C++ 101

C++ Servers Storage Speed 101

SQL Server According to Bob

NOVEMBER 8, 2016

SQL Server 2016 Service Pack 1 (all SKUs) , in combination with Windows Server 2016 (All SKUs) or Windows 10 Client introduces non-volatile memory support for the tail of the log file (LDF) which can significantly increase transaction throughput. Block storage is what you think of today as disk access. Tail Of Log Caching.

Cache 40

Cache Servers Media Storage 40

All Things Distributed

DECEMBER 13, 2016

In just this past month we've had HSBC , ARM , Missguided , and most recently at re:Invent 2016 , Trainline , talking with us about how they are using AWS to transform and scale their businesses. I have been humbled by just how much our UK customers have been able to achieve using AWS technology so far.

AWS 166

AWS Cloud Artificial Intelligence IoT 166

Scalegrid

JANUARY 19, 2024

Data-bearing members face a higher risk of encountering issues caused by rollbacks, compared to others who utilize different storage methods. The files look something like this: <dbname> <collectionname> 2016-02-08T19-34-44.0.bson

Database 130

Database Network Servers Monitoring 130

Alex Russell

MARCH 6, 2021

Back in 2016, I gave a talk outlining the causes and effects of the terrible performance of web apps built using popular tools on the fastest-growing device segment: low-end to mid-range Android phones. A then-representative $200USD device had 4-8 slow (in-order, low-cache) cores, ~2GiB of RAM, and relatively slow MLC NAND flash storage.

Performance 124

Performance Network Cache Metrics 124

Brendan Gregg

NOVEMBER 29, 2017

In this configuration, the AMI and boot is paravirt (PV), the kernel is making hypercalls instead of privileged instructions, and the system is using paravirt network and storage drivers. But not all workloads: some are network bound (proxies) and storage bound (databases). ## 5. The AMI and boot are now HVM.

Virtualization 111

Virtualization AWS Hardware Storage 111

Percona

JANUARY 24, 2023

Some of the built-in features ( wal_compression ) have been there since 2016, and almost all backup tools do the WAL compression before taking it to the backup repository. Attempts to compress PostgreSQL WAL at different levels have always been around since the beginning. The actual benefit of compression depends on many factors.

Database 114

Database Benchmarking Open Source Latency 114

Percona

AUGUST 18, 2023

And I agree that in the modern age of cloud and huge flash storage, the vast majority of companies will never need to consider doing this in prod, but there is always a chance of its need. That said, other performance motivations mentioned above did play into it, especially before the days of multi-threaded replication. 10-Linux.x86_64.ssl101.tar.gz

Programming 82

Programming Servers Database Hardware 82

Percona

JANUARY 17, 2019

now has a version which will support parallelism for SELECT queries (utilizing the read capacity of storage nodes underneath the Aurora cluster). Aurora PQ works by doing a full table scan (parallel reads are done on the storage level). SSD) we can’t utilize the full potential of IOPS with just one thread. Documentation: [link].

Cache 46

Cache C++ AWS Airlines 46

Percona

JUNE 4, 2023

MariaDB retains compatibility with MySQL, offers support for different programming languages, including Python, PHP, Java, and Perl, and works with all major open source storage engines such as MyRocks, Aria, and InnoDB. GA 23 February 2016 5.7.10-3 MariaDB Server includes many storage engines beyond the default InnoDB.

Open Source 52

Open Source Servers Storage Scalability 52

SQL Performance

APRIL 8, 2019

Back on December 5, 2017, Microsoft announced that they were using AMD EPYC 7551 processors in their storage-optimized Lv2-Series virtual machines. The slightly newer Intel Xeon E5-26xx v4 (Broadwell) series which was introduced in Q1 of 2016, increased that to 2400MHz. They also only have 40 PCIe 3.0 lanes per processor. Memory (GiB).

Azure 42

Azure Virtualization Storage Benchmarking 42

All Things Distributed

DECEMBER 12, 2018

The second platform is a managed IoT cloud with customer-facing applications and data management, which went live in 2016. We are excited to offer a comprehensive portfolio of services, from foundational technologies such as compute, storage, and networking to more advanced services such as containers and serverless computing.

AWS 124

AWS Cloud Games Serverless 124

SQL Performance

MAY 29, 2019

From SQL Server 2016 onward, FastLoadContext is enabled by default ; the trace flag is not required. For more background, see the Data Performance Loading Guide and the Tiger Team notes on the behaviour changes for SQL Server 2016. FastLoadContext can be disabled on SQL Server 2016 using documented trace flag 692. I >= 100.

Cache 64

Cache Testing Servers Code 64

Smashing Magazine

FEBRUARY 8, 2022

In 2016, I released the first version of the tool, then called not Statoscope, but Webpack Runtime Analyzer. We can import information from our metric storage into data. For example, we can record the daily average bundle build time, send it to storage with the metrics and then embed it into the custom report. Sergey Melukov.

Storage 79

Storage Code Metrics C++ 79

SQL Performance

SEPTEMBER 5, 2019

You will need SQL Server 2016 (or later) and Developer Edition (or equivalent) to reproduce the results shown here. This table uses columnstore for its primary storage to produce batch mode execution later on. You might call this arrangement "off-row" or "out-of-batch" storage. DROP TABLE IF EXISTS #T ; GO. c1 = @Start + N.

Performance 81

Performance C++ Storage Servers 81

Tim Kadlec

JANUARY 9, 2019

Cennydd also makes the case that performance also has an impact on energy consumption: In 2016, video, tracking scripts and sharing buttons caused the average website to swell to the same size as the original version of Doom. Ballooning bandwidth and storage have fostered complacency that we can do without. Performance is conservation.

Performance 111

Performance Energy Internet Internet 111

SQL Server According to Bob

DECEMBER 18, 2018

Device level flushing may have an impact on your I/O caching, read ahead or other behaviors of the storage system. Neal, Matt, and others from Windows Storage, Windows Azure Storage, Windows Hyper-V, … validating Windows behaviors. · Any storage device that can survive a power outage. Starting with the Linux 4.18

Servers 90

Servers Media Cache Storage 90

Brendan Gregg

NOVEMBER 28, 2017

In this configuration, the AMI and boot is paravirt (PV), the kernel is making hypercalls instead of privileged instructions, and the system is using paravirt network and storage drivers. But not all workloads: some are network bound (proxies) and storage bound (databases). ## 5. The AMI and boot are now HVM.

Virtualization 40

Virtualization AWS Hardware Storage 40

SQL Performance

MAY 1, 2019

The summary rows for heap tables without indexes are the same in both documents (no changes for SQL Server 2016): An explicit TABLOCK hint is not the only way to meet the requirement for table-level locking. This is reflected in the summary tables included in the Data Loading Performance Guide and the Tiger Team post.

Servers 47

Servers Database Testing Performance 47

Testsigma

FEBRUARY 12, 2021

billion in 2016. By 2021, a distributed cloud would help companies physically put all services closely together, thereby addressing low-latency challenges, minimising the expense of storage and ensuring that data standards are consistent with the laws in a given geographical region. In 2021 what can we expect?

Artificial Intelligence 92

Artificial Intelligence Software Software IoT 92

Brendan Gregg

SEPTEMBER 5, 2017

LTS (April 2016). I wrote about it in a previous post, [DTrace for Linux 2016]. I wrote a page on it: [perf]. - **eBPF**: tracing features completed in 2016, this provides efficient programmatic tracing to existing kernel frameworks. The hardest part on Linux is now done: kernel support. It's the official profiler.

Virtualization 76

Virtualization AWS Engineering Hardware 76

SQL Performance

AUGUST 23, 2019

SQL Server 2016 introduced the ability to create a bitmap for general predicates on dictionary-encoded segment data. This 2016 optimization can be applied to any predicate pushed into a columnstore scan, including a bitmap 'predicate' created by an upstream hash join.

Servers 75

Servers Engineering Efficiency Retail 75

SQL Performance

APRIL 23, 2019

SQL Server 2016 introduced serial batch mode processing and aggregate pushdown. Columnstore storage. To understand this, it is necessary to first review how columnstore storage works at a high level: A compressed row group contains a column segment for each column. Test 1: Pushdown, 9-bit Impure Keys.

Servers 67

Servers Testing Metrics Storage 67

Percona

JANUARY 14, 2019

In a partitioned massively parallel database system, the storage format and sorting algorithm may not be optimized for that operation as we are reading multiple partitions in parallel. voting','maven'] ? ? compose','xamarin','markdown','scrum','comic'] ? ? china','sourceforge','subscription','chinese','kotlin'] ? ?

Database 40

Database Analytics Blockchain Healthcare 40

SQL Performance

MAY 16, 2019

The calculated row size (61 bytes) differs from the true row storage size (60 bytes) by the extra one byte of internal metadata present in the insert stream. These correspond directly to the new facilities added under trace flag 610 to SQL Server 2008, then changed to be on by default from SQL Server 2016 onward.

Cache 67

Cache Servers Testing Storage 67

Smashing Magazine

JANUARY 15, 2019

An obvious metric here is CPU usage, but memory usage and other forms of data storage also play their part. In 2016, O’Reilly published “ Designing For Sustainability ” by Tim Frick. These include data transfer (i.e. Data transfer is one thing that we can measure quite easily.

Website Performance 73

Website Performance Website Energy Performance 73