Adrian Cockcroft

JUNE 21, 2022

So in 1988 I joined the Sun sales force and became a Systems Engineer (nowadays we would say Solutions Architect). I became the Sun UK local specialist in performance and hardware, and as Sun transitioned from a desktop workstation company to sell high end multiprocessor servers I was helping customers find and fix scalability problems.

Tuning 52

Tuning Benchmarking Engineering C++ 52

Adrian Cockcroft

JUNE 2, 2023

FSD has a sense of purpose, a planning capability, has real time agency and responds to its environment via an ego model, predicting the behavior of pedestrians and other road users, and is being tuned to drive in a very human way, so that other road users interact with it as a predicable normal driver. Tristan Zajonc, CEO of Continual.ai

Social Media 98

Social Media Innovation Open Source Entertainment 98

HammerDB

APRIL 19, 2023

Please refer to this tuning guide to tune the system for HammerDB: Open Source Database Tuning Guide on 3rd Generation Intel® Xeon® Scalable Processors Based Platform. For the experiments in this blog, we did not tune the system. on identical hardware, with identical settings, but at different load levels.

Performance 54

Performance Benchmarking Cache Storage 54

The Morning Paper

JUNE 13, 2019

As we saw with the SOAP paper last time out, even with a fixed model variant and hardware there are a lot of different ways to map a training workload over the available hardware. Different hardware architectures (CPUs, GPUs, TPUs, FPGAs, ASICs, …) offer different performance and cost trade-offs.

Hardware 58

Hardware Latency Serverless Energy 58

Alex Podelko

OCTOBER 20, 2016

System’s configuration is not given anymore and often can’t be easily mapped to hardware. As already mentioned, performance testing is rather a performance engineering process (with tuning, optimization, troubleshooting and fixing multi-user issues) eventually bringing the system to the proper state rather than just testing.

Performance Testing 100

Performance Testing Architecture Testing Performance 100

Sutter's Mill

NOVEMBER 11, 2023

Instead, you want a library that is tuned for your target hardware architecture and ready for par_unseq vectorized algorithms, for blazing speed. If you want to do efficient linear algebra, you don’t want to write your own code by hand; that would be slow. This is that library.

C++ 107

C++ Lambda Network Programming 107

Percona

JANUARY 19, 2023

You can find more information in this good blog from Marco: MySQL Sharding with ProxySQL Vitess: It is an open source database clustering solution created by PlanetScale that is compatible with the MySQL engine. MyRocks: MyRocks is a storage engine developed by Facebook and made open source. It supports native sharding.

Open Source 87

Open Source Storage Database Big Data 87

Abhishek Tiwari

JULY 12, 2018

These data pipelines can process data at petabytes scale and to some extent, their success can be attributed to an army of engineers devoted to build and maintain internal data pipelines. Not everyone is operating at Netflix or Spotify scale data engineering function. A data pipeline is a software which runs on hardware.

Latency 63

Latency Analytics Scalability Engineering 63

The Agile Manager

JUNE 30, 2015

We have to look at skills and capabilities: we're not going to be much of a software company if we don't employ any software engineers. Operations and execution matter to the bottom line, but ultimately dance to the tune called by finance. Is every company destined to be a software company?

Software 40

Software Software Innovation Hardware 40

Percona

MARCH 23, 2023

Hardware Memory The amount of RAM to be provisioned for database servers can vary greatly depending on the size of the database and the specific requirements of the company. If you see concurrency issues, you can tune this variable. have been released since then with some major changes. I hope this helps! References How MySQL 8.0.21

Performance 124

Performance Hardware Tuning Storage 124

Dynatrace

SEPTEMBER 29, 2021

Container technology is very powerful as small teams can develop and package their application on laptops and then deploy it anywhere into staging or production environments without having to worry about dependencies, configurations, OS, hardware, and so on. The time and effort saved with testing and deployment are a game-changer for DevOps.

Open Source 302

Open Source Traffic DevOps Cloud 302

Percona

JANUARY 4, 2024

Flexible location : Data files can reside within the MySQL data directory or an independent location, enabling finer control over storage management and performance tuning. Creating and managing general tablespaces You can create general tablespaces using the CREATE TABLESPACE statement, specifying data file locations and engine options.

Storage 87

Storage Engineering Database Hardware 87

The Netflix TechBlog

APRIL 29, 2019

Such applications track the inventory of our network gear: what devices, of which models, with which hardware components, located in which sites. Python has long been a popular programming language in the networking space because it’s an intuitive language that allows engineers to quickly solve networking problems.

Open Source 279

Open Source Network Infrastructure Big Data 279

O'Reilly

NOVEMBER 28, 2023

Even with cloud-based foundation models like GPT-4, which eliminate the need to develop your own model or provide your own infrastructure, fine-tuning a model for any particular use case is still a major undertaking. That pricing won’t be sustainable, particularly as hardware shortages drive up the cost of building infrastructure.

Open Source 108

Open Source Artificial Intelligence Google Infrastructure 108

SQL Performance

JANUARY 23, 2020

Tom Davidson, Opening Microsoft's Performance-Tuning Toolbox SQL Server Pro Magazine, December 2003. Waits and Queues has been used as a SQL Server performance tuning methodology since Tom Davidson published the above article as well as the well-known SQL Server 2005 Waits and Queues whitepaper in 2006. The Top Queries That Weren't.

Tuning 115

Tuning Servers Monitoring Hardware 115

Dynatrace

AUGUST 12, 2020

With Azure Functions, engineers don’t have to worry about provisioning and maintaining underlying hardware; they simply upload their code, and it’s up and running seconds later. So stay tuned! These functions are usually triggered by events, therefore, Microsoft Azure is also commonly described as “event-driven FaaS.”

Serverless 236

Serverless Azure Monitoring Lambda 236

O'Reilly

SEPTEMBER 19, 2023

Doubly so as hardware improved, eating away at the lower end of Hadoop-worthy work. The data scientist—sorry, “machine learning engineer” or “AI specialist”—job interview now involves one of those toolkits, or one of the higher-level abstractions such as HuggingFace Transformers.

Hardware 99

Hardware Storage Big Data Blockchain 99

The Morning Paper

NOVEMBER 10, 2019

You need a lot of software engineers and the willingness to rewrite a lot of software to entertain that idea. Data plane operations are handled by pluggable engines (Pony Express is an engine). Upgrades are performed incrementally, one engine at a time. The little engine that could. Enter Google!

Network 92

Network Transportation Latency Entertainment 92

Dynatrace

AUGUST 12, 2020

With Azure Functions, engineers don’t have to worry about provisioning and maintaining underlying hardware; they simply upload their code, and it’s up and running seconds later. So stay tuned! These functions are usually triggered by events, therefore, Microsoft Azure is also commonly described as “event-driven FaaS.”

Serverless 141

Serverless Azure Monitoring Lambda 141

Dynatrace

OCTOBER 26, 2020

Dynatrace has recently enhanced its Metrics APIs, allowing everyone to send any type of metric with any set of data dimension to Davis, Dynatrace’s AI engine. All your JMeter results in Dynatrace for better performance engineering. So stay tuned. Dynatrace news. Goal: sending metrics to Dynatrace. Raspberry Pi Model 3 B.

Innovation 154

Innovation Metrics Hardware Games 154

All Things Distributed

MARCH 19, 2018

This post lifts the veil on some of the scientific, system design, and engineering decisions we made along the way. Amazon SageMaker training supports powerful container management mechanisms that include spinning up large numbers of containers on different hardware with fast networking and access to the underlying hardware, such as GPUs.

Scalability 159

Scalability Hardware AWS Tuning 159

Percona

APRIL 17, 2023

The main objective of this post is to share my experience over the past years tuning MongoDB and centralize the diverse sources that I crossed in this journey in a unique place. systemctl stop tuned $ systemctl disable tuned Dirty ratio The dirty_ratio is the percentage of total system memory that can hold dirty pages.

Best Practices 52

Best Practices Design Tuning Database 52

All Things Distributed

MARCH 7, 2013

We have also reduced our underlying costs through significant technical innovations from our engineering team. This allows us to tune both our hardware and our software to ensure that the end-to-end service is both cost-efficient and highly performant. As with our other services, as weâ?? In the end, youâ??ll

Ecommerce 139

Ecommerce Storage Scalability Hardware 139

Percona

AUGUST 5, 2023

Otherwise, the storage engine does a scatter-gather and queries ALL partitions in a UNION that is not concurrent. As the data is distributed into smaller partitions, the database engine only needs to scan relevant partitions, leading to faster query responses. Streamlined Maintenance : Partitioning simplifies maintenance operations.

Strategy 40

Strategy Storage Database Best Practices 40

Brendan Gregg

SEPTEMBER 25, 2021

As a Xen guest, this profile was gathered using perf(1) and the kernel's software cpu-clock soft interrupts, not the hardware NMI. I happened to be speaking at a technical confering while still debugging this, and mentioned what I was working on to a processor engineer. But I'm not completely sure.

Speed 126

Speed Java AWS Virtualization 126

The Morning Paper

MAY 21, 2019

Using zswap means that no new hardware solutions are required, enabling rapid deployment across clusters. ML-based auto-tuning. The big advantage of the ML based approach is that it can continuously adapt to changes in the workload and WSC configuration without needing constant manual tuning. Evaluation.

Software 78

Software Software Google Hardware 78

DZone

AUGUST 16, 2019

It’s not just a simple tweak you can turn on/off; it’s a long-time process that touches almost every single item in your stack, including both hardware and software sides of the system. Application scalability is the potential of an application to grow in time, being able to efficiently handle more and more requests per minute (RPM).

Scalability 145

Scalability Efficiency Hardware Software 145

SQL Performance

JUNE 21, 2019

While hardware such as intelligent SANs, Solid State Disk, and other advancements have helped speed things up, wasted space in index can translate to wasted space in the buffer pool as well as wasting more I/O. The engine takes care of placing logical processors from the same physical core into different soft-NUMA nodes.

Servers 49

Servers Azure Hardware Database 49

HammerDB

JANUARY 5, 2019

Some opinions claim that “Benchmarks are meaningless”, “benchmarks are irrelevant” or “benchmarks are nothing like your real applications” However for others “Benchmarks matter,” as they “account for the processing architecture and speed, memory, storage subsystems and the database engine.”

Benchmarking 40

Benchmarking Database Social Media Scalability 40

Dotcom-Montior

OCTOBER 12, 2017

Dotcom-Monitor’s LoadView and Dynatrace are performance-engineering platforms by heart. This showcase demonstrates how the combination of both tools will close remaining gaps in the performance engineering chain. Verify hardware sizing. Make application tuning much easier. Understand communication pattern.

AWS 40

AWS Tuning Monitoring Servers 40

Brendan Gregg

SEPTEMBER 5, 2017

This includes many great engineers who I'm sure will excel in whatever they choose to work on next. Linux has been adding tracing technologies over the years: kprobes (kernel dynamic tracing), uprobes (user-level dynamic tracing), tracepoints (static tracing), and perf_events (profiling and hardware counters).

Virtualization 76

Virtualization AWS Engineering Hardware 76

Percona

AUGUST 29, 2023

Its query engine not only parallelizes incoming SQL queries for time series but also has the ability to create column-wise tables making it ideal for OLAP analysis duties. The distributed query engine can route and parallelize SELECT, DML, and other operations on distributed tables across the cluster.

C++ 79

C++ Database Storage Testing 79

HammerDB

OCTOBER 12, 2018

As is also the case this limitation is at the database level (especially the storage engine) rather than the hardware level. InnoDB is the storage engine that will deliver the best OLTP throughput and should be chosen for this test. . driver: intel_pstate CPUs which run at the same hardware frequency: 0 .

Best Practices 40

Best Practices Scalability Performance C++ 40

O'Reilly Software

APRIL 4, 2018

The data shape will dictate capacity planning, tuning of the backbone, and scalability analysis for individual components. The output expectations will assist in the choice of processing engine while the process tolerance will add restrictions in terms of processing semantics and error handling. At least once? At most once?

Architecture 80

Architecture Scalability Google Operating System 80

Brendan Gregg

SEPTEMBER 25, 2021

As a Xen guest, this profile was gathered using perf(1) and the kernel's software cpu-clock soft interrupts, not the hardware NMI. I happened to be speaking at a technical confering while still debugging this, and mentioned what I was working on to a processor engineer. But I'm not completely sure.

Speed 40

Speed Java AWS Virtualization 40

O'Reilly

OCTOBER 11, 2022

Andrew Ng , Christopher Ré , and others have pointed out that in the past decade, we’ve made a lot of progress with algorithms and hardware for running AI. Our current set of AI algorithms are good enough, as is our hardware; the hard problems are all about data. Is special-purpose training to fine-tune a model’s behavior an option?

Games 116

Games Healthcare Processing Retail 116

John McCalpin

JANUARY 22, 2018

Hardware performance counter results for a simple benchmark code calling Intel’s optimized DGEMM implementation for this processor (from the Intel MKL library) show that about 20% of the dynamic instruction count consists of instructions that are not packed SIMD operations (i.e., addl $1, %eax vfmadd213pd %zmm16, %zmm17, %zmm29.

Latency 40

Latency Hardware Code Testing 40