Percona

MARCH 28, 2023

” This acts as a step to ensure durability by recovering lost data from the same journal files in case of crashes, power, and hardware failures between the checkpoints (see below) Here’s what the process looks like. So, what happens when there’s an unexpected crash or hardware failure?

Hardware 92

Hardware C++ Storage Cache 92

Percona

JULY 7, 2023

Unfortunately, using certain open source database software as part of an HA architecture can present significant challenges. Therefore, DBAs must take a piecemeal approach to achieve HA with PostgreSQL, bringing together open source extensions and tools from multiple sources. there cannot be high availability.

Availability 88

Availability Open Source Architecture Database 88

Brendan Gregg

JULY 4, 2021

AWS Graviton2); for memory with the arrival of DDR5 and High Bandwidth Memory (HBM) on-processor; for storage including new uses for 3D Xpoint as a 3D NAND accelerator; for networking with the rise of QUIC and eXpress Data Path (XDP); and so on. I also wrote about these topics in detail for my recent [Systems Performance 2nd Edition] book.

Performance 140

Performance Latency Hardware Storage 140

Percona

FEBRUARY 24, 2023

Use reduction at scale requires a cultural shift, and engineers must consider the cost as they think of memory or bandwidth as another deployment KPI. From AWS documentation , Amazon EBS is an easy-to-use, scalable, high-performance block-storage service designed for Amazon EC2.

Cloud 123

Cloud Serverless AWS Storage 123

Percona

JUNE 22, 2023

A monitoring tool like Percona Monitoring and Management (PMM) is a popular choice among open source options for effectively monitoring MySQL performance. They include backup success rate, backup duration, recovery time objective (RTO), recovery point objective (RPO), and backup storage utilization.

Performance 108

Performance Monitoring Traffic Database 108

HammerDB

JUNE 27, 2022

It enables the user to measure database performance and make comparative judgements about database hardware and software. These factors meant that often when looking for database performance information, the results for a particular combination of software and hardware were not available. How to build HammerDB from source.

Benchmarking 62

Benchmarking Open Source C++ Cache 62

Percona

JANUARY 19, 2023

It can help us to save costs on storage and backup times. While MySQL can handle large data sets, it is always recommended to keep only the used data in the databases, as this will make data access more efficient, and also will help to save costs on storage and backups. It supports native sharding being transparent for the application.

Open Source 92

Open Source Storage Database Big Data 92

HammerDB

APRIL 19, 2023

As database performance is heavily influenced by the performance of storage, network, memory, and processors, we must understand the upper limit of these key components. For storage, FIO is generally used. Storage: The system has a SATA drive for the operating system and one NVMe (Intel SSD D7-P5510 (3.84 Database: MySQL 8.0.31

Performance 54

Performance Benchmarking Cache Storage 54

Percona

JULY 1, 2023

Understanding DBaaS DBaaS cloud services allow users to use databases without configuring physical hardware and infrastructure or installing software. What we should really compare is the MySQL and Aurora database engines provided by Amazon RDS. Considering a Fully Managed DBaaS Offering For Your Business?

AWS 52

AWS Database Serverless Storage 52

Percona

NOVEMBER 1, 2023

MySQL backups play a pivotal role in safeguarding the integrity of your data, providing defense against various unforeseen calamities, hardware malfunctions, data loss, corruption, and inadvertent deletions. Encryption Backups have sensitive data, so it’s highly recommended to encrypt, especially for offsite storage.

Best Practices 79

Best Practices Storage Strategy Database 79

Brendan Gregg

JULY 4, 2021

AWS Graviton2); for memory with the arrival of DDR5 and High Bandwidth Memory (HBM) on-processor; for storage including new uses for 3D Xpoint as a 3D NAND accelerator; for networking with the rise of QUIC and eXpress Data Path (XDP); and so on. I also wrote about these topics in detail for my recent [Systems Performance 2nd Edition] book.

Performance 52

Performance Latency Hardware Storage 52

Brendan Gregg

FEBRUARY 28, 2023

link] 2021 - [Trader 21] Tiffany Trader, “Cerebras Second-Gen 7nm Wafer Scale Engine Doubles AI Performance Over First-Gen Chip ,” [link] Apr 2021 - [Ghigoff 21] Yoann Ghigoff, et al., "BMC:

Performance 110

Performance Latency Cache Virtualization 110

Alex Russell

APRIL 29, 2021

Apple forces developers of competing browsers to use their engine for all browsers on iOS , restricting their ability to deliver a better version of the web platform. They are, pound for pound, some of the best engine developers globally and genuinely want good things for the web. So is speedy resolution and agreement.

Media 145

Media Games Education Engineering 145

Abhishek Tiwari

DECEMBER 27, 2017

Kubernetes has emerged as go to container orchestration platform for data engineering teams. faster access to external storage and data locality (I/O, bandwidth). To address this, the open source community is now trying to develop Kubernetes custom schedulers specifically optimised for big data workloads.

Big Data 63

Big Data Storage Benchmarking Hardware 63

Testsigma

MAY 3, 2021

We, engineers, love automation! From operating systems to versions to hardware specs, mobile devices stand unique even after being billions in number. But testing this is essential too because data communication over the network cannot be eliminated by putting all the files into the device storage.

Mobile 77

Mobile Testing Operating System Hardware 77

Percona

AUGUST 29, 2023

Its query engine not only parallelizes incoming SQL queries for time series but also has the ability to create column-wise tables making it ideal for OLAP analysis duties. The distributed query engine can route and parallelize SELECT, DML, and other operations on distributed tables across the cluster.

C++ 84

C++ Database Storage Testing 84

Brendan Gregg

FEBRUARY 28, 2023

link] , 2021 [Trader 21] Tiffany Trader, “Cerebras Second-Gen 7nm Wafer Scale Engine Doubles AI Performance Over First-Gen Chip ,” [link] , Apr 2021 [Ghigoff 21] Yoann Ghigoff, et al.,

Performance 52

Performance Latency Cache Virtualization 52

The Morning Paper

MAY 28, 2019

The goal is to produce a low-energy hardware classifier for embedded applications doing local processing of sensor data. Race logic has four primary operations that are easy to implement in hardware: MAX, MIN, ADD-CONSTANT, and INHIBIT. One efficient way of doing that in analog hardware is the use of current-starved inverters.

Energy 52

Energy Hardware Efficiency Architecture 52

Percona

APRIL 17, 2023

Since MongoDB usually works with a replica set, it is possible to increase this parameter to get better performance: # edit /etc/mongod.conf storage: journal: enabled: true commitIntervalMs: 300 WiredTiger cache For dedicated servers, it is possible to increase the WiredTiger(WT) cache. By default, it uses 50% of the memory + 1 GB.

Best Practices 52

Best Practices Design Tuning Database 52

HammerDB

JANUARY 5, 2019

Some opinions claim that “Benchmarks are meaningless”, “benchmarks are irrelevant” or “benchmarks are nothing like your real applications” However for others “Benchmarks matter,” as they “account for the processing architecture and speed, memory, storage subsystems and the database engine.”

Benchmarking 40

Benchmarking Database Social Media Scalability 40

HammerDB

DECEMBER 5, 2020

From an engineering perspective this may be all that you need to know. Additionally both commercial and open source tools based on the specifications also continued to use TPC-C and TPC-H to describe these workloads. However a reasonable follow-on question is then why change the name at all? But why is this important?

C++ 40

C++ Benchmarking Virtualization Open Source 40

All Things Distributed

SEPTEMBER 6, 2013

Shell''s scientists, especially the geophysicists and drilling engineers, frequently use cloud computing to run models. Due to the exponential growth of the biology and informatics fields, Unilever needs to maintain this new program within a highly-scalable environment that supports parallel computation and heavy data storage demands.

Cloud 129

Cloud Energy AWS Healthcare 129

Abhishek Tiwari

JULY 12, 2018

These data pipelines can process data at petabytes scale and to some extent, their success can be attributed to an army of engineers devoted to build and maintain internal data pipelines. Not everyone is operating at Netflix or Spotify scale data engineering function. A data pipeline is a software which runs on hardware.

Latency 63

Latency Analytics Scalability Engineering 63

The Netflix TechBlog

MARCH 8, 2021

As an engineer, I can work anywhere with a standard laptop as long as I have an IDE and access to Stack Overflow. They need specialized hardware, access to petabytes of images, and digital content creation applications with controlled licenses. Spinnaker uses “pipelines” as instructions for creating the pools.

Entertainment 306

Entertainment Storage Hardware Open Source 306

Alex Russell

JULY 14, 2021

The 85% global-share OS (Android) has historically facilitated browser choice and diversity in browser engines. Engine diversity is essential, as it is the mechanism that causes competition to deliver better performance, capability, privacy, security, and user controls. If so, it's a browser regardless of the underlying engine.

Google 121

Google Mobile Engineering Internet 121

Adrian Cockcroft

NOVEMBER 11, 2019

In addition to the common financial calculation of risk as the product of probability and severity, engineering risk includes detectability. A learning organization, disaster recovery testing, game days, and chaos engineering tools are all important components of a continuously resilient system.

Latency 52

Latency Engineering Systems Hardware 52

Adrian Cockcroft

NOVEMBER 11, 2019

In addition to the common financial calculation of risk as the product of probability and severity, engineering risk includes detectability. A learning organization, disaster recovery testing, game days, and chaos engineering tools are all important components of a continuously resilient system.

Latency 53

Latency Engineering Systems Hardware 53

O'Reilly

SEPTEMBER 19, 2023

It progressed from “raw compute and storage” to “reimplementing key services in push-button fashion” to “becoming the backbone of AI work”—all under the umbrella of “renting time and storage on someone else’s computers.” Cloud computing? ” scenarios at industrial scale.

Hardware 99

Hardware Storage Big Data Blockchain 99

Smashing Magazine

JANUARY 6, 2020

Performance isn’t just a technical concern: it affects everything from accessibility to usability to search engine optimization, and when baking it into the workflow, design decisions have to be informed by their performance implications. Looking back now, things seem to have changed quite significantly. Large preview ).

Performance 145

Performance Cache Network Metrics 145

Smashing Magazine

JANUARY 11, 2021

Performance isn’t just a technical concern: it affects everything from accessibility to usability to search engine optimization, and when baking it into the workflow, design decisions have to be informed by their performance implications. Looking back now, things seem to have changed quite significantly. Large preview ).

Performance 145

Performance Cache Media Metrics 145

Smashing Magazine

JANUARY 7, 2019

dashboard (open source), SpeedCurve and Calibre are just a few of them, and you can find more tools on perf.rocks. Deviation metrics As noted by Wikipedia engineers , data of how much variance exists in your results could inform you how reliable your instruments are, and how much attention you should pay to deviations and outlers.

Performance 103

Performance Cache Metrics Network 103