The Morning Paper

MARCH 8, 2020

Building an elastic query engine on disaggregated storage , Vuppalapati, NSDI’20. This paper describes the design decisions behind the Snowflake cloud-based data warehouse. have altered the many assumptions that guided the design and optimization of the Snowflake system. From shared-nothing to disaggregation.

Storage 112

Storage Engineering Cache Serverless 112

Dynatrace

DECEMBER 15, 2022

Complex cloud computing environments are increasingly replacing traditional data centers. In fact, Gartner estimates that 80% of enterprises will shut down their on-premises data centers by 2025. The IT help desk creates a ticketing system and resolves service request issues. So, what is ITOps? Why is IT operations important?

Artificial Intelligence 198

Artificial Intelligence DevOps Hardware Virtualization 198

High Scalability

DECEMBER 16, 2022

Cloud computing is a model of computing that delivers computing services over the internet, including storage, data processing, and networking. It allows users to access and use shared computing resources, such as servers, storage, and applications, on demand and without the need to manage the underlying infrastructure.

Cloud 201

Cloud Serverless Internet Internet 201

Dynatrace

JANUARY 16, 2023

The study analyzes factual Kubernetes production data from thousands of organizations worldwide that are using the Dynatrace Software Intelligence Platform to keep their Kubernetes clusters secure, healthy, and high performing. Kubernetes is emerging as the “operating system” of the cloud. Kubernetes moved to the cloud in 2022.

Open Source 340

Open Source Java Operating System Programming 340

Dynatrace

AUGUST 26, 2020

“ Dynatrace just makes observability easy—it works out-of-the-box, no silos of data, no DIY stitching together tools, no wasted time, and no wasted resources.” There’s no other competing software that can provide this level of value with minimum effort and optimal hardware utilization that can scale up to web-scale!

Performance 150

Performance Hardware Java Storage 150

Dynatrace

AUGUST 5, 2022

A message queue is a form of middleware used in software development to enable communications between services, programs, and dissimilar components, such as operating systems and communication protocols. A message queue enables the smooth flow of information to make complex systems work. What is the Advanced Message Queuing Protocol?

Monitoring 196

Monitoring Serverless Programming Speed 196

Dynatrace

AUGUST 5, 2022

A message queue is a form of middleware used in software development to enable communications between services, programs, and dissimilar components, such as operating systems and communication protocols. A message queue enables the smooth flow of information to make complex systems work. What is the Advanced Message Queuing Protocol?

Monitoring 195

Monitoring Serverless Programming Speed 195

ACM Sigarch

OCTOBER 4, 2018

This is the first post in a series of posts on different approaches to systems security especially as they apply to hardware and architectural security. In this post, we will consider the use of mathematics/cryptography as an approach to improving systems security.

Systems 47

Systems Hardware Storage Healthcare 47

All Things Distributed

JANUARY 23, 2012

Werner Vogels weblog on building scalable and robust distributed systems. Expanding the Cloud - The AWS Storage Gateway. Today Amazon Web Services has launched the AWS Storage Gateway, making the power of secure and reliable cloud storage accessible from customersâ?? s storage infrastructure. Comments ().

Storage 115

Storage AWS Virtualization Cloud 115

All Things Distributed

SEPTEMBER 20, 2013

Traditionally records in a database were stored as such: the data in a row was stored together for easy and fast retrieval. Combined with the rise of data warehouse workloads, where there is often significant redundancy in the values stored in columns, and database models based on column oriented storage took off.

Storage 87

Storage Hardware Cache C++ 87

Percona

SEPTEMBER 1, 2023

This results in expedited query execution, reduced resource utilization, and more efficient exploitation of the available hardware resources. This not only enhances performance but also enables you to make more efficient use of your hardware resources, potentially resulting in cost savings on infrastructure.

Tuning 52

Tuning Database Performance Hardware 52

Brendan Gregg

JULY 4, 2021

AWS Graviton2); for memory with the arrival of DDR5 and High Bandwidth Memory (HBM) on-processor; for storage including new uses for 3D Xpoint as a 3D NAND accelerator; for networking with the rise of QUIC and eXpress Data Path (XDP); and so on. Note that my predictions in this talk may be wrong, but they should be thought provoking.

Performance 140

Performance Latency Hardware Storage 140

Abhishek Tiwari

DECEMBER 27, 2017

Kubernetes has emerged as go to container orchestration platform for data engineering teams. In 2018, a widespread adaptation of Kubernetes for big data processing is anitcipated. Organisations are already using Kubernetes for a variety of workloads [1] [2] and data workloads are up next. better cluster resource utilization.

Big Data 63

Big Data Storage Benchmarking Hardware 63

Percona

JUNE 22, 2023

Database uptime and availability Monitoring database uptime and availability is crucial as it directly impacts the availability of critical data and the performance of applications or websites that rely on the MySQL database. Disk space usage Monitor the disk space usage of MySQL data files, log files, and temporary files.

Performance 106

Performance Monitoring Traffic Database 106

Scalegrid

JANUARY 25, 2024

Effective management of memory stores with policies like LRU/LFU proactive monitoring of the replication process and advanced metrics such as cache hit ratio and persistence indicators are crucial for ensuring data integrity and optimizing Redis’s performance. A reduced throughput could generally indicate that the server gets fewer queries.

Metrics 130

Metrics Monitoring Latency Cache 130

Highly Scalable

AUGUST 20, 2013

The shortcomings and drawbacks of batch-oriented data processing were widely recognized by the Big Data community quite a long time ago. This system has been designed to supplement and succeed the existing Hadoop-based system that had too high latency of data processing and too high maintenance costs.

Big Data 154

Big Data Processing Lambda Database 154

ACM Sigarch

AUGUST 3, 2023

Embodied carbon is due to Scope 3, e.g., manufacturing IT hardware and datacenter construction. Hence, significant additional investments into capacity and energy storage would be needed to match a fab’s daily load. Chien’s Arguments Against Embodied Carbon Prof.

Energy 98

Energy Architecture Hardware Metrics 98

Percona

JANUARY 4, 2024

Managing storage and performance efficiently in your MySQL database is crucial, and general tablespaces offer flexibility in achieving this. In contrast to the single system tablespace that holds system tables by default, general tablespaces are user-defined storage containers for multiple InnoDB tables.

Storage 90

Storage Engineering Database Hardware 90

HammerDB

APRIL 19, 2023

Therefore, before we attempt to measure our database performance, we should know the system or cloud instance to be tested in detail. As database performance is heavily influenced by the performance of storage, network, memory, and processors, we must understand the upper limit of these key components. Operating System: Ubuntu 22.04

Performance 54

Performance Benchmarking Cache Storage 54

Abhishek Tiwari

JULY 12, 2018

Building data pipelines can offer strategic advantages to the business. Often companies underestimate the necessary effort and cost involved to build and maintain data pipelines. Data pipeline initiatives are generally unfinished projects. In this post, we will discuss why you should avoid building data pipelines in first place.

Latency 63

Latency Analytics Scalability Engineering 63

Brendan Gregg

FEBRUARY 28, 2023

This talk originated from my updates to [Systems Performance 2nd Edition], and this was the first time I've given this talk in person! CXL in a way allows a custom memory controller to be added to a system, to increase memory capacity, bandwidth, and overall performance. Ford, et al., “TCP

Performance 110

Performance Latency Cache Virtualization 110

Percona

MAY 1, 2023

It is primarily the responsibility of the database administrator or developer to tune PostgreSQL according to their system’s workload. Hardware optimization : You need to ensure that the CPU, memory, and storage components meet the performance requirements of the database workload. What is PostgreSQL performance tuning?

Tuning 52

Tuning Efficiency Database Performance 52

Percona

DECEMBER 27, 2023

In our conversations with developers and engineers during the conference, we confirmed the CNCF GItOps Microsurvey data – 91% are already using GitOps. Creating repositories for memory-safe patterns in cloud systems to improve overall security. of respondents are currently utilizing databases in Kubernetes (k8s).

Cloud 82

Cloud Open Source Strategy Best Practices 82

Brendan Gregg

JULY 4, 2021

AWS Graviton2); for memory with the arrival of DDR5 and High Bandwidth Memory (HBM) on-processor; for storage including new uses for 3D Xpoint as a 3D NAND accelerator; for networking with the rise of QUIC and eXpress Data Path (XDP); and so on. Note that my predictions in this talk may be wrong, but they should be thought-provoking.

Performance 52

Performance Latency Hardware Storage 52

The Morning Paper

MAY 27, 2019

Last week we saw the benefits of rethinking memory and pointer models at the hardware level when it came to object storage and compression ( Zippads ). The protections are hardware implemented and cannot be forged in software. At hardware reset the boot code is granted maximally permissive architectural capabilities.

C++ 61

C++ Hardware Virtualization Benchmarking 61

The Morning Paper

FEBRUARY 18, 2020

When it comes to leveraging ML in enterprise applications, especially in regulated environments, the level of scrutiny for data handling, model fairness, user privacy, and debuggability will be substantially higher than in the first wave of ML applications. The key insight this led to is that ML models are software , derived from data.

Government 95

Government Software Engineering Storage Database 95

Testsigma

JULY 3, 2021

But for the uninitiated, on-premise testing is a form of testing where testers perform tests on local machines, systems, or devices set up at an office. More control: While performing on-premise testing, organizations have more control over configurations, setup, hardware, and software. As the name suggests, it happens on-premise.

Cloud 57

Cloud Testing Storage DevOps 57

All Things Distributed

AUGUST 24, 2012

Werner Vogels weblog on building scalable and robust distributed systems. The AWS team launched this week Amazon Glacier , a cold storage archive service at the very low price point of $0.01 Which makes this week a good moment to read up on some of the historical work around the costs of data engineering. Comments ().

Storage 108

Storage Hardware AWS Data Engineering 108

Scalegrid

FEBRUARY 1, 2024

Regarding MySQL backups , knowing how to secure your data is crucial. Dive in to uncover the essentials of different backup types, command-line tools, and strategic practices for robust data protection. MySQL is a popular open-source relational database management system for online applications and data warehousing.

Best Practices 130

Best Practices Storage Strategy Database 130

O'Reilly

SEPTEMBER 19, 2023

It progressed from “raw compute and storage” to “reimplementing key services in push-button fashion” to “becoming the backbone of AI work”—all under the umbrella of “renting time and storage on someone else’s computers.” Cloud computing? And Hadoop rolled in. Until it wasn’t.

Hardware 99

Hardware Storage Big Data Blockchain 99

Scalegrid

MAY 13, 2020

It can scale towards a multi-petabyte level data workload without a single issue, and it allows access to a cluster of powerful servers that will work together within a single SQL interface where you can view all of the data. This feature-packed database provides powerful and rapid analytics on data that scales up to petabyte volumes.

Big Data 321

Big Data Database Artificial Intelligence Open Source 321

High Scalability

NOVEMBER 14, 2018

New topics range from additional workloads like video streaming, machine learning, and public cloud to specialized silicon accelerators, storage and network building blocks, and a revised discussion of data center power and cooling, and uptime.

Design 54

Design Education Google Internet 54

Brendan Gregg

FEBRUARY 28, 2023

This talk originated from my updates to Systems Performance 2nd Edition , and this was the first time I've given this talk in person! CXL in a way allows a custom memory controller to be added to a system, to increase memory capacity, bandwidth, and overall performance. Ford, et al., “TCP

Performance 52

Performance Latency Cache Virtualization 52

Dynatrace

JUNE 25, 2020

With Dynatrace actively managing business-critical applications, some of our globally distributed enterprise customers require Dynatrace Managed to continue operating even when an entire data center goes down. Our Premium High Availability comes with the following features: Active-active deployment model for optimum hardware utilization.

Availability 244

Availability Hardware Latency Traffic 244

Percona

AUGUST 30, 2023

These days, “Data is king,” and a lot more. Data powers everything, and unlike coal and coal combustion, data and databases aren’t going away. Any organization that uses personal and other sensitive data must have a firm, proven plan for business continuity in the event of a disaster or cyberattack.

Database 90

Database Hardware Open Source Strategy 90

Adrian Cockcroft

JUNE 21, 2022

Photo by Adrian I spent six years at Cambridge Consultants, building some interesting systems, managing our Sun workstations and learning a lot, but by then Sun had opened a sales office across the street, and I wanted to find out what they were going to release next, before everyone else.

Tuning 52

Tuning Benchmarking Engineering C++ 52

All Things Distributed

MARCH 11, 2015

From retail recommendations to genomics based product development, from financial risk management to start-ups measuring the effect of their new products, from digital marketing to fast processing of clinical trial data, all are taken to the next level by cloud based analytics. Cloud enables self-service analytics.

Analytics 135

Analytics Cloud Healthcare Retail 135