Adrian Cockcroft

JUNE 2, 2023

My talk was on Innovation and Tipping Points, the first half was based on some content I’ve given before on how to get out of the way of innovation by speeding up time to value or idea to implementation.

Social Media 98

Social Media Innovation Open Source Entertainment 98

SQL Performance

MARCH 17, 2020

Back in 2014, I wrote an article called Performance Tuning the Whole Query Plan. This machine has four i7 CPUs (hyperthreaded to 8) with a base speed of 2.4GHz. The Sort only has 19 rows to process, so it consumes only 1ms or so. Having a wide range of tools available is generally speaking a good thing when tuning queries.

Servers 86

Servers Tuning Database Efficiency 86

KeyCDN

AUGUST 15, 2019

Before you begin tuning your website or application, you must first figure out which metrics matter most to your users and establish some achievable benchmarks. When the word “performance” is heard, most people immediately think of speed. Memory utilization: The amount of memory required to process a request.

Testing Tools 55

Testing Tools Best Practices Performance Testing Testing 55

Brendan Gregg

SEPTEMBER 25, 2021

What about short-lived processes, like a service restarting in a loop? As a Xen guest, this profile was gathered using perf(1) and the kernel's software cpu-clock soft interrupts, not the hardware NMI. Measuring the speed of time Is there already a microbenchmark for os::javaTimeMillis()? These can be invisible to top(8).

Speed 126

Speed Java AWS Virtualization 126

Percona

MARCH 23, 2023

Hardware Memory The amount of RAM to be provisioned for database servers can vary greatly depending on the size of the database and the specific requirements of the company. As datasets continue to grow in size, the amount of RAM required to store and process these datasets also increases. Refer to innodb_redo_log_capacity below.

Performance 123

Performance Hardware Tuning Storage 123

Percona

AUGUST 29, 2018

In this post, we will discuss some important kernel parameters that can affect database server performance and how these should be tuned. SHMMAX is a kernel parameter used to define the maximum size of a single shared memory segment a Linux process can allocate. A page is a chunk of RAM that is allocated to a process.

Tuning 53

Tuning Database Servers Processing 53

Brendan Gregg

SEPTEMBER 25, 2021

What about short-lived processes, like a service restarting in a loop? As a Xen guest, this profile was gathered using perf(1) and the kernel's software cpu-clock soft interrupts, not the hardware NMI. Measuring the speed of time Is there already a microbenchmark for os::javaTimeMillis()? These can be invisible to top(8).

Speed 40

Speed Java AWS Virtualization 40

Dynatrace

NOVEMBER 15, 2021

Logs can include data about user inputs, system processes, and hardware states. Log monitoring is a process by which developers and administrators continuously observe logs as they’re being recorded. Log analytics is the process of evaluating and interpreting log data so teams can quickly detect and resolve issues.

Analytics 209

Analytics Monitoring Artificial Intelligence DevOps 209

O'Reilly

NOVEMBER 28, 2023

Users and Nonusers AI adoption is in the process of becoming widespread, but it’s still not universal. Until AI reaches 100%, it’s still in the process of adoption. Automating the process of building complex prompts has become common, with patterns like retrieval-augmented generation (RAG) and tools like LangChain.

Open Source 111

Open Source Artificial Intelligence Google Infrastructure 111

Dynatrace

AUGUST 12, 2020

As companies strive to innovate and deliver faster, modern software architecture is evolving at near the speed of light. With Azure Functions, engineers don’t have to worry about provisioning and maintaining underlying hardware; they simply upload their code, and it’s up and running seconds later. So stay tuned! Dynatrace news.

Serverless 236

Serverless Azure Monitoring Lambda 236

Percona

JANUARY 4, 2024

Flexible location : Data files can reside within the MySQL data directory or an independent location, enabling finer control over storage management and performance tuning. Balancing I/O load: Distribute tables across multiple general tablespaces located on different disks to avoid I/O bottlenecks and improve query execution speed.

Storage 83

Storage Engineering Database Hardware 83

All Things Distributed

MARCH 19, 2018

Bill Simmons, CTO of Dataxu, states, "We process 3 million ad requests a second - 100,000 features per request. In other words, processing the 10th gigabyte and 1000th gigabyte is conceptually the same. To seamlessly switch between CPU and GPU machines, we use Apache MXNet to interface with the underlying hardware.

Scalability 159

Scalability Hardware AWS Tuning 159

Dynatrace

AUGUST 12, 2020

As companies strive to innovate and deliver faster, modern software architecture is evolving at near the speed of light. With Azure Functions, engineers don’t have to worry about provisioning and maintaining underlying hardware; they simply upload their code, and it’s up and running seconds later. So stay tuned! Dynatrace news.

Serverless 142

Serverless Azure Monitoring Lambda 142

The Morning Paper

NOVEMBER 10, 2019

” That’s 4-8x the speed of evolution and feedback cycles. To get that release speed, Snap needs to be a user space solution. …with a gradual upgrade process across a cluster, we have found that our existing applications do not notice hundred millisecond blips in communication that occur once per week.

Network 92

Network Transportation Latency Entertainment 92

Percona

AUGUST 5, 2023

Each partition holds data that falls within a specific range, optimizing data handling and query speed. This version also notably integrates native backing for range and list partitioning of spatial indexes, amplifying geospatial query speed for substantial datasets. Additionally, MySQL 8.0

Strategy 40

Strategy Storage Database Best Practices 40

Dynatrace

OCTOBER 22, 2019

Such behavior not only limits speed but also your ability to increase throughput by adding resources. Ultimately, it leads to a state where your system won’t be able to process more data even if you add more hardware. Now, let’s look at a Cassandra process group and see if we can find any CPU-hungry threads and thread groups.

Java 103

Java Scalability Hardware Architecture 103

The Morning Paper

MAY 21, 2019

Using zswap means that no new hardware solutions are required, enabling rapid deployment across clusters. ML-based auto-tuning. This model is used by a Gaussian Process (GP) Bandit machine learning model to guide the parameter search towards an optimal point with a minimal number of trials.

Software 78

Software Software Google Hardware 78

Dynatrace

OCTOBER 26, 2020

With the rich set of features in Dynatrace for diagnostics (check out my Advanced Diagnostics with Dynatrace YouTube Tutorial ) it speeds up analysis and diagnostics for Christian significantly. So stay tuned. If you want to replicate Christians work – here are the software and hardware specs: Hardware. Python 3.7.3.

Innovation 154

Innovation Metrics Hardware Games 154

HammerDB

JANUARY 5, 2019

Some opinions claim that “Benchmarks are meaningless”, “benchmarks are irrelevant” or “benchmarks are nothing like your real applications” However for others “Benchmarks matter,” as they “account for the processing architecture and speed, memory, storage subsystems and the database engine.”

Benchmarking 40

Benchmarking Database Social Media Scalability 40

SQL Performance

JUNE 21, 2019

Prior to SQL Server 2012, the maximum server memory value only applied to the bufferpool, so customers needed to limit the amount of memory the bufferpool could use to save memory for the operating system and other processes. Part of that process is his Index Optimize procedure, which can take additional parameters to update statistics.

Servers 49

Servers Azure Hardware Database 49

HammerDB

OCTOBER 18, 2019

cpupower frequency-info analyzing CPU 0: driver: intel_pstate CPUs which run at the same hardware frequency: 0 CPUs which need to have their frequency coordinated by software: 0 maximum transition latency: Cannot determine or is not supported. hardware limits: 1000 MHz - 4.00 hardware limits: 1000 MHz - 4.00

Benchmarking 63

Benchmarking Performance Hardware C++ 63

Smashing Magazine

AUGUST 22, 2021

A Primer on Speed. Discussing performance and “speed” can quickly get complex, because many underlying aspects contribute to a web-page loading “slowly”. It is physically limited by the speed of light or, practically, how fast signals can travel in wires or in the open air.

Performance 107

Performance Network Latency Servers 107

SQL Server According to Bob

JANUARY 11, 2020

Many high-end disk subsystems provide high-speed cache facilities to reduce the latency of read and write operations. Example 1: ​​ Hardware failure (CPU board) Battery backup on the caching controller maintained the data. Important Always consult with your hardware manufacturer for proper stable media strategies.

Servers 40

Servers Cache Media Hardware 40

Brendan Gregg

SEPTEMBER 5, 2017

There's also a ZFS send/recv code path that should try to use the TASK_INTERRUPTIBLE flag (as suggested by a coworker), to avoid a kernel hang (can't kill -9 the process). On Linux, containers are a combination of namespaces (restricting what a process sees) and cgroups (similar to Solaris resource controls). amazon").

Virtualization 76

Virtualization AWS Engineering Hardware 76

HammerDB

OCTOBER 12, 2018

As is also the case this limitation is at the database level (especially the storage engine) rather than the hardware level. driver: intel_pstate CPUs which run at the same hardware frequency: 0 . hardware limits: 1000 MHz - 3.80 hardware limits: 1000 MHz - 3.80 current CPU frequency: Unable to call hardware .

Best Practices 40

Best Practices Scalability Performance C++ 40

SQL Performance

MARCH 8, 2021

This can make it difficult to draw sound performance-tuning conclusions. This uses hardware, typically a crystal oscillator, that produces ticks at a very high constant rate regardless of processor speed, power settings, or anything of that nature. This operator consumes no CPU time, and only processes 524 rows.

Servers 95

Servers Tuning Architecture Performance 95

SQL Server According to Bob

JANUARY 11, 2020

DBCC Page Auditing DBCC page auditing is the process of executing common DBCC consistency checks as pages are read from disk. ​​ This is a serious I/O subsystem problem that should be addressed immediately to avoid SQL Server and overall system stability problems ​​ and ​​ data loss.

Servers 40

Servers Cache Database Media 40

Percona

NOVEMBER 3, 2023

About the cluster Following a step-by-step process, the objective is to create a four-node cluster consisting of: PostgreSQL version 15 Citus extension (I’ll be using version 11, but there are newer ones available.) In order to speed up the benchmark indexes must be added. psql pgbench <<_eof1_ qecho adding node citus3.

Database 101

Database Benchmarking Latency C++ 101

High Scalability

NOVEMBER 25, 2019

Your Laptop has a filesystem used by hundreds of processes. It is limited by the disk space; it can’t expand storage elastically; it chokes if you run few I/O intensive processes or try collaborating with 100 other users. On prem data processing. Locks used for strong consistency across processes. Document store.

Architecture 52

Architecture Cache Azure Storage 52