HammerDB

APRIL 19, 2023

As database performance is heavily influenced by the performance of storage, network, memory, and processors, we must understand the upper limit of these key components. For the network, we can use Iperf to assess the network bandwidth between the client and the database server to ensure it will be enough to meet our peak requirement.

Performance 54

Performance Benchmarking Cache Storage 54

The Netflix TechBlog

JUNE 4, 2019

Because microprocessors are so fast, computer architecture design has evolved towards adding various levels of caching between compute units and the main memory, in order to hide the latency of bringing the bits to the brains. The predictions are then fed into a MIP which is solved on the fly.

Cache 251

Cache Latency Airlines Logistics 251

Brendan Gregg

FEBRUARY 28, 2023

DDR6: Here's What to Expect in RAM Modules,” [link] Nov 2020 - [Salter 20] Jim Salter, “Western Digital releases new 18TB, 20TB EAMR drives,” [link] Jul 2020 - [Spier 20] Martin Spier, Brendan Gregg, et al.,

Performance 110

Performance Latency Cache Virtualization 110

Brendan Gregg

JULY 4, 2021

AWS Graviton2); for memory with the arrival of DDR5 and High Bandwidth Memory (HBM) on-processor; for storage including new uses for 3D Xpoint as a 3D NAND accelerator; for networking with the rise of QUIC and eXpress Data Path (XDP); and so on. I also wrote about these topics in detail for my recent [Systems Performance 2nd Edition] book.

Performance 52

Performance Latency Hardware Storage 52

Brendan Gregg

FEBRUARY 28, 2023

DDR6: Here's What to Expect in RAM Modules,” [link] , Nov 2020 [Salter 20] Jim Salter, “Western Digital releases new 18TB, 20TB EAMR drives,” [link] , Jul 2020 [Spier 20] Martin Spier, Brendan Gregg, et al.,

Performance 52

Performance Latency Cache Virtualization 52

Percona

AUGUST 30, 2023

In general terms, here are potential trouble spots: Hardware failure: Manufacturing defects, wear and tear, physical damage, and other factors can cause hardware to fail. heat) can damage hardware components and prompt data loss. Human mistakes: Incorrect configuration is an all-too-common cause of hardware and software failure.

Database 84

Database Hardware Open Source Strategy 84

Scalegrid

MAY 13, 2020

Greenplum Database is an open-source , hardware-agnostic MPP database for analytics, based on PostgreSQL and developed by Pivotal who was later acquired by VMware. Greenplum uses an MPP database design that can help you develop a scalable, high performance deployment. Greenplum Architectural Design. Open Source.

Big Data 321

Big Data Database Artificial Intelligence Open Source 321

The Morning Paper

MAY 12, 2019

An open-source benchmark suite for microservices and their hardware-software implications for cloud & edge systems Gan et al., The paper examines the implications of microservices at the hardware, OS and networking stack, cluster management, and application framework levels, as well as the impact of tail latency.

Open Source 61

Open Source Hardware Benchmarking Systems 61

Brendan Gregg

AUGUST 18, 2019

bpftrace is a new open source tracer for Linux for analyzing production performance problems and troubleshooting software. hardware Hardware counter-based instrumentation. On, say, the network engineering team, they have been using BCC to develop an agent for their needs. usdt User-level statically defined tracing.

Latency 68

Latency C++ Cache Programming 68

The Netflix TechBlog

APRIL 29, 2019

We use and contribute to many open-source Python packages, some of which are mentioned below. Open Connect Open Connect is Netflix’s content delivery network (CDN). video streaming) takes place in the Open Connect network. If any of this interests you, check out the jobs site or find us at PyCon.

Open Source 279

Open Source Network Infrastructure Big Data 279

Percona

AUGUST 22, 2023

We’ll wrap it up by suggesting high availability open source solutions, and we’ll introduce you to support options for ensuring continuous high performance from your systems. Without enough infrastructure (physical or virtualized servers, networking, etc.), What does high availability mean? What is fault tolerance?

Availability 76

Availability Hardware Open Source Database 76

ACM Sigarch

NOVEMBER 4, 2019

Krste Asanovic from UC Berkeley kicked off the main program sharing his experience on “ Rejuvenating Computer Architecture Research with Open-Source Hardware ”. He ended the keynote with a call to action for open hardware and tools to start the next wave of computing innovation.

Hardware 18

Hardware Architecture Programming Innovation 18

O'Reilly

NOVEMBER 28, 2023

16% of respondents working with AI are using open source models. Many of the new open source models are much smaller and not as resource intensive but still deliver good results (especially when trained for a specific application). But 18% already have applications in production. Or is there still room for growth?

Open Source 111

Open Source Artificial Intelligence Google Infrastructure 111

Abhishek Tiwari

JULY 12, 2018

A data pipeline is a software which runs on hardware. The software is error-prone and hardware failures are inevitable. For a telecom company, the ability to monitor network congestion and failures is part of their core value proposition. A data pipeline can process data in a different order than they were received.

Latency 63

Latency Analytics Scalability Engineering 63

Percona

JUNE 22, 2023

Defining high availability In general terms, high availability refers to the continuous operation of a system with little to no interruption to end users in the event of hardware or software failures, power outages, or other disruptions. In addition to the disaster recovery site, this design includes an external layer of nodes.

Availability 103

Availability Database Open Source Hardware 103

Percona

APRIL 17, 2023

For example: UUID=f41e390f-835b-4223-a9bb-9b45984ddf8d / xfs rw,noatime,attr2,inode64,noquota 0 0 Unix ulimit settings Most UNIX-like operating systems, including Linux and macOS, provide ways to limit and control the usage of system resources such as threads, files, and network connections on a per-process and per-user basis.

Best Practices 52

Best Practices Design Tuning Database 52

Percona

JULY 7, 2023

Unfortunately, using certain open source database software as part of an HA architecture can present significant challenges. Therefore, DBAs must take a piecemeal approach to achieve HA with PostgreSQL, bringing together open source extensions and tools from multiple sources. there cannot be high availability.

Availability 81

Availability Open Source Architecture Database 81

High Scalability

OCTOBER 24, 2022

Each cloud-native evolution is about using the hardware more efficiently. Network effects are not the same as monopoly control. Maybe start investing into open source alternatives rather than complaining they don't already exist? Do you think those evolutions would have happened with a structural separation in place?

Cloud 187

Cloud Hardware Infrastructure Innovation 187

Percona

FEBRUARY 24, 2023

Figure 1: PMM Home Dashboard From the Amazon Web Services (AWS) documentation , an instance is considered over-provisioned when at least one specification of your instance, such as CPU, memory, or network, can be sized down while still meeting the performance requirements of your workload and no specification is under-provisioned.

Cloud 114

Cloud Serverless AWS Storage 114

Testlodge

JUNE 16, 2021

To secure consumer data, organizations need to make sure that their networks are updated and protected against any malicious activity. Penetration testing is comprehensively performed over a fully-functional system’s software and hardware. It is available both as open source and as a commercial version.

Testing 67

Testing Hardware Open Source Internet 67

O'Reilly

SEPTEMBER 19, 2023

Doubly so as hardware improved, eating away at the lower end of Hadoop-worthy work. And that brings our story to the present day: Stage 3: Neural networks High-end video games required high-end video cards. Google goes a step further in offering compute instances with its specialized TPU hardware.

Hardware 101

Hardware Storage Big Data Blockchain 101

All Things Distributed

MARCH 19, 2018

So, when designing Amazon SageMaker we took on a challenge: to build machine learning algorithms that can handle an infinite amount of data. This post lifts the veil on some of the scientific, system design, and engineering decisions we made along the way. What does that even mean though? This sounds like a pipe dream.

Scalability 159

Scalability Hardware AWS Tuning 159

Alex Russell

APRIL 29, 2021

Apple Corporate is at fault, not Open Source engineers or the line managers who support them. Real-time network protocols for enabling videoconferencing, desktop sharing, and game streaming applications. Modern, asynchronous network APIs that dramatically improve performance in some situations. Delayed five years.

Media 145

Media Games Education Engineering 145

The Morning Paper

MARCH 1, 2020

Firecracker is open source, and there are a number of projects that make it easy to work with outside of the AWS environment too, including Weave Firekube (disclaimer: Accel is an investor in Weaveworks). The design of Firecracker. These simplifying assumptions are reflected in Firecracker’s design and implementation.

Serverless 101

Serverless Virtualization Lambda AWS 101

Testsigma

MAY 3, 2021

Web developers are moving their focus from desktop-first to mobile-first web design which further solidifies our understanding of how important this pocket device has become. From operating systems to versions to hardware specs, mobile devices stand unique even after being billions in number. Side-By-Side View for multi-tasking.

Mobile 77

Mobile Testing Operating System Hardware 77

All Things Distributed

JULY 20, 2015

The pool of resources, at this time, is the CPU, memory, and networking resources of Amazon EC2 instances as partitioned by containers. networks ports, memory, CPU, etc). This key/value store acts as the single source of truth for all information on the cluster (state, and all changes to state transitions) are entered and stored here.

Latency 166

Latency Architecture AWS Open Source 166

Smashing Magazine

AUGUST 22, 2021

As we will see, QUIC and HTTP/3 indeed have great web performance potential, but mainly for users on slow networks. If your average visitor is on a fast cabled or cellular network, they probably won’t benefit from the new protocols all that much. An often used metaphor is that of a pipe used to transport water. Congestion Control.

Performance 106

Performance Network Latency Servers 106

The Morning Paper

MAY 28, 2019

The goal is to produce a low-energy hardware classifier for embedded applications doing local processing of sensor data. Race logic has four primary operations that are easy to implement in hardware: MAX, MIN, ADD-CONSTANT, and INHIBIT. One efficient way of doing that in analog hardware is the use of current-starved inverters.

Energy 52

Energy Hardware Efficiency Architecture 52

Percona

JULY 26, 2023

To make this process work more efficiently and ensure a smooth failover, it is important to have the same hardware configuration on all the nodes of the replica set. When you run multiple MongoD processes on the same server, it’s very difficult to monitor them and the resources they are consuming (CPU, RAM, Network, etc.).

Database 77

Database Servers Storage Strategy 77

HammerDB

FEBRUARY 25, 2021

As we have seen databases are designed to handle multiple database sessions at the same time. For simplicity, we do not include networking or transaction management middleware in this discussion because although important in the real world they do not affect the key concepts. Basic Benchmarking Concepts. Why Tcl as a Glue Language.

Programming 67

Programming Benchmarking Virtualization C++ 67

Adrian Cockcroft

NOVEMBER 11, 2019

Another problem is that a design control, intended to mitigate a failure mode, may not work as intended. This discussion focuses on hardware, software and operational failure modes. STPA is based on a functional control diagram of the system, and the safety constraints and requirements for each component in the design.

Latency 52

Latency Engineering Systems Hardware 52

Alex Russell

JUNE 20, 2020

full TCP/IP & UDP networking, including local network discovery. Java arrived with a focus on structured programming (including threads), GUI controls , networking, device access, and graphics. In each quantum of time, hardware and OS vendors press ahead, adding features. high-color image raster. floppy/CD/DVD).

Hardware 55

Hardware Games Java Media 55

Adrian Cockcroft

NOVEMBER 11, 2019

Another problem is that a design control, intended to mitigate a failure mode, may not work as intended. This discussion focuses on hardware, software and operational failure modes. STPA is based on a functional control diagram of the system, and the safety constraints and requirements for each component in the design.

Latency 53

Latency Engineering Systems Hardware 53

Alex Russell

JULY 14, 2021

The benefits to apps that adopt WebView-based IABs are numerous: WebViews are system components designed for use within other apps. On Android today and early iOS versions, WebViews allow embedders to observe and modify all network traffic (regardless of encryption). Hardware access APIs, notably: Geolocation. Web Bluetooth.

Google 121

Google Mobile Engineering Internet 121

ACM Sigarch

APRIL 8, 2019

Communities in other fields have made a substantial effort towards “democratization,” where tools are not just made open source, but also accessible to everyone. For example, the existence and support of open-source frameworks such as LLVM or Tensorflow/Pytorch are an attractive element to many newcomers. uarch@gmail.com.

Architecture 74

Architecture Open Source Hardware Software Engineering 74

Smashing Magazine

JANUARY 11, 2021

Performance isn’t just a technical concern: it affects everything from accessibility to usability to search engine optimization, and when baking it into the workflow, design decisions have to be informed by their performance implications. Networking, HTTP/2, HTTP/3 OCSP stapling, EV/DV certificates, packaging, IPv6, QUIC, HTTP/3.

Performance 145

Performance Cache Media Metrics 145

Smashing Magazine

JANUARY 6, 2020

Performance isn’t just a technical concern: it affects everything from accessibility to usability to search engine optimization, and when baking it into the workflow, design decisions have to be informed by their performance implications. Large preview ). There are many tools allowing you to achieve that: SiteSpeed.io thanks, Guy Podjarny! ).

Performance 145

Performance Cache Network Metrics 145