The Morning Paper

MARCH 8, 2020

Building an elastic query engine on disaggregated storage , Vuppalapati, NSDI’20. This paper describes the design decisions behind the Snowflake cloud-based data warehouse. have altered the many assumptions that guided the design and optimization of the Snowflake system. From shared-nothing to disaggregation.

Storage 112

Storage Engineering Cache Serverless 112

Percona

FEBRUARY 1, 2023

This post will look at using The Oversized-Attribute Storage Technique (TOAST) to improve performance and scalability. Therefore, TOAST is a storage technique used in PostgreSQL to handle large data objects such as images, videos, and audio files.

Storage 94

Storage Scalability Strategy Performance 94

Scalegrid

MARCH 28, 2024

This article will explore how they handle data storage and scalability, perform in different scenarios, and, most importantly, how these factors influence your choice. Redis Data Types and Structures The design of Redis’s data structures emphasizes versatility. Memcached’s primary strength lies in its simplicity.

Cache 130

Cache Storage Scalability Architecture 130

Percona

AUGUST 22, 2023

With so much at stake, database high availability and fault tolerance have become must-have items, but many companies just aren’t certain which one they must have. This blog article will examine shared attributes of high availability (HA) and fault tolerance (FT). What does high availability mean?

Availability 83

Availability Hardware Open Source Database 83

Scalegrid

MAY 13, 2020

It’s architecture was specially designed to manage large-scale data warehouses and business intelligence workloads by giving you the ability to spread your data out across a multitude of servers. Greenplum uses an MPP database design that can help you develop a scalable, high performance deployment. Greenplum Architectural Design.

Big Data 321

Big Data Database Artificial Intelligence Open Source 321

Dynatrace

JANUARY 31, 2024

Designed to reach beyond petabyte (PB) scale, OpenPipeline will—at its launch— quintuple the ingest throughput from 100 TB a day per tenant to 500 TB a day per tenant. OpenPipeline high-performance filtering and preprocessing provides full ingest and storage control for the Dynatrace platform. Commitment to privacy.

Analytics 195

Analytics Processing Transportation Storage 195

Dynatrace

APRIL 28, 2023

Optimize cost and availability while staying compliant Observability data like logs and metrics provide automated answers, root cause detection, and security issues. This means compromising between keeping data available as long as possible for analysis while juggling the costs and overhead of storage, archiving, and retrieval.

Analytics 216

Analytics Storage Infrastructure Metrics 216

DZone

OCTOBER 11, 2022

Making Google’s CalDAV and CardDAV APIs available for everyone ( Google Developers Blog). Pandora launches new HTML5 site for TVs and gaming consoles, available now on PS3 and Xbox 360 ( The Next Web). Simpler UI Testing with CasperJS ( Architects Zone – Architectural Design Patterns & Best Practices). Hacker News).

Java 244

Java Best Practices Google Analytics 244

DZone

JANUARY 23, 2022

Hardware - servers/storage hardware/software faults such as disk failure, disk full, other hardware failures, servers running out of allocated resources, server software behaving abnormally, intra DC network connectivity issues, etc. . – this is addressed through monitoring and redundancy. Redundancy by building additional data centers.

Hardware 243

Hardware DevOps Network Storage 243

Dynatrace

OCTOBER 4, 2022

A data lakehouse addresses these limitations and introduces an entirely new architectural design. This architecture offers rich data management and analytics features (taken from the data warehouse model) on top of low-cost cloud storage systems (which are used by data lakes). This scenario is a thing of the past.

Artificial Intelligence 270

Artificial Intelligence Analytics Storage Architecture 270

Scalegrid

JANUARY 8, 2019

In Part I , we introduced a High Availability (HA) framework for MySQL hosting and discussed various components and their functionality. Simply put, in a MySQL semisynchronous replication configuration, the master commits transactions to the storage engine only after receiving acknowledgement from at least one of the slaves.

Availability 113

Availability Tuning Speed Network 113

Dynatrace

OCTOBER 7, 2021

This means you no longer have to provision, scale, and maintain servers to run your applications, databases, and storage systems. Instead of worrying about infrastructure management functions, such as capacity provisioning and hardware maintenance, teams can focus on application design, deployment, and delivery. Reliability.

Serverless 283

Serverless AWS Lambda Storage 283

Dynatrace

OCTOBER 6, 2023

Grail: Enterprise-ready data lakehouse Grail, the Dynatrace causational data lakehouse, was explicitly designed for observability and security data, with artificial intelligence integrated into its foundation. Buckets are similar to folders, a physical storage location. There is a default bucket for each table.

Artificial Intelligence 263

Artificial Intelligence Analytics Metrics Storage 263

Particular Software

APRIL 12, 2021

Azure Storage Queues is a basic yet robust queueing service available on the Azure platform. For example, it lacks concepts like topics and subscriptions to build publisher and subscriber topologies—these are capabilities NServiceBus adds on top of Azure Storage Queues. Here’s how it works.

Azure 52

Azure Storage Transportation Java 52

Percona

MARCH 11, 2024

Volume Expansion, a feature that became generally available since Kubernetes version 1.24, allows users to increase the capacity of their Persistent Volumes and underlying storage within Kubernetes. In our Operators, we use Stateful Sets – a Kubernetes resource that was designed for […]

Storage 92

Storage Cloud Design Availability 92

All Things Distributed

MARCH 23, 2021

Storage was one of our biggest pain points, and the traditional systems we used just weren’t fitting the needs of the Amazon.com retail business. When we took a hard look at our storage for the Amazon ecommerce web site in 2005, we realized that the majority of our data needed an object (or key-value) store.

Ecommerce 216

Ecommerce Retail Storage Scalability 216

Percona

APRIL 17, 2023

Note that the intent of tuning the settings is not exclusively about improving performance but also enhancing the high availability and resilience of the MongoDB database. There is an issue with this, which causes the OS to swap even with memory available. Spoiler alert: This post focuses on MongoDB 3.6.X 25.84 - Total 21.04

Best Practices 52

Best Practices Design Tuning Database 52

Dynatrace

AUGUST 11, 2022

Infrastructure as a service (IaaS) handles compute, storage, and network resources. Increased availability. Because FaaS is a cloud-native approach, it makes great use of multisite cloud architecture to improve availability and reliability. Consider a monolithic application, for example, designed to perform a host of functions.

Development 253

Development Serverless Best Practices Lambda 253

Scalegrid

JANUARY 12, 2024

All rely heavily on utilizing allocated portions from existing pools made available through specific providers as part of their service offerings. There are numerous choices available for deploying these workloads on various cloud provider platforms that offer unique capabilities.

Cloud 130

Cloud Virtualization Storage Efficiency 130

Percona

MAY 29, 2023

With the Percona Operators, businesses can manage multi-cloud or hybrid-cloud PostgreSQL deployments with ease, ensuring that critical data is always available and secure, no matter what happens. pgBackrest on the Main site streams backups and Write Ahead Logs (WALs) to the object storage.

Storage 124

Storage Cloud Network Traffic 124

Scalegrid

MARCH 8, 2024

Executing PITR requires restoring from the full backup and then applying binary log events in sequence up to the desired point in time, with advanced techniques and third-party tools available to optimize large dataset handling and automate the recovery process. Use the ‘–log-bin’ option to designate a base name for the binary log files.

Database 162

Database Strategy Servers Best Practices 162

Percona

MARCH 29, 2023

Compression in any database is necessary as it has many advantages, like storage reduction, data transmission time, etc. Storage reduction alone results in significant cost savings, and we can save more data in the same space. By default, MongoDB provides a snappy block compression method for storage and network communication.

Storage 113

Storage Network Open Source Latency 113

Dynatrace

JUNE 6, 2023

This key feature helps in maintaining availability and reduces the need for manual intervention. Therefore, they are built to be non-persistent by design. Traditional storage solutions were not created to address these requirements, which are common among modern deployments. Extensibility and technology ecosystem.

Open Source 195

Open Source Storage Analytics Innovation 195

Percona

FEBRUARY 24, 2023

Let’s take a look at how to get the benefits you need while spending less, based on the recommendations presented by Dani Guzmán Burgos, our Percona Monitoring and Management (PMM) Tech Lead, on this webinar (now available on demand) hosted in November last year. These color-coded states on panels are available in PMM 2.32.0

Cloud 123

Cloud Serverless AWS Storage 123

All Things Distributed

AUGUST 20, 2012

Managing Cold Storage with Amazon Glacier. With the introduction of Amazon Glacier , IT organizations now have a solution that removes the headaches of digital archiving and provides extremely low cost storage. All Things Distributed. Werner Vogels weblog on building scalable and robust distributed systems. Expanding the Cloud â??

Storage 122

Storage Cloud AWS Media 122

Scalegrid

JANUARY 5, 2024

It was initially designed to mitigate the limitations of file management systems, including slow operations, inadequate security, and substantial data redundancy. It comprises a collection of interrelated data and a set of software tools that aid in the access, processing, and management of data.

Efficiency 147

Efficiency Storage Database Scalability 147

Scalegrid

JULY 17, 2020

JSONB storage has some drawbacks vs. traditional columns: PostreSQL does not store column statistics for JSONB columns. JSONB storage results in a larger storage footprint. JSONB storage does not deduplicate the key names in the JSON. If that doesn’t work, the data is moved to out-of-line storage.

Storage 321

Storage Database Efficiency Processing 321

Dynatrace

JUNE 27, 2023

Logs complement out-of-the-box metrics and enable automated actions for responding to availability, security, and other service events. Many AWS services and third party solutions use AWS S3 for log storage. A Dynatrace S3 log forwarder has been available for some time to early adopters, with community support only.

AWS 201

AWS Cloud Lambda Analytics 201

High Scalability

DECEMBER 16, 2022

Cloud computing is a model of computing that delivers computing services over the internet, including storage, data processing, and networking. It allows users to access and use shared computing resources, such as servers, storage, and applications, on demand and without the need to manage the underlying infrastructure.

Cloud 201

Cloud Serverless Internet Internet 201

The Netflix TechBlog

NOVEMBER 29, 2023

HDR was launched at Netflix in 2016 and the number of titles available in HDR has been growing ever since. Bitrate versus quality comparison HDR-VMAF is designed to be format-agnostic — it measures the perceptual quality of HDR video signal regardless of its container format, for example, Dolby Vision or HDR10. The graphic below (Fig.

Open Source 252

Open Source Software Engineering Internet Internet 252

Percona

DECEMBER 11, 2023

A Dedicated Log Volume (DLV) is a specialized storage volume designed to house database transaction logs separately from the volume containing the database tables. DLVs are particularly advantageous for databases with large allocated storage, high I/O per second (IOPS) requirements, or latency-sensitive workloads.

AWS 101

AWS Benchmarking Performance Traffic 101

The Netflix TechBlog

OCTOBER 28, 2021

During earlier years of my career, I primarily worked as a backend software engineer, designing and building the backend systems that enable big data analytics. Thanks to the migration to V2, we were able to improve data availability and usability for our consumers significantly.

Data Engineering 173

Data Engineering Engineering Big Data Software Engineering 173

Scalegrid

NOVEMBER 29, 2023

A DBaaS automates several processes, such as using, erasing, and spinning up storage without interventions from IT staff. Automated backups mean your database will never be lost and it will always be available for your company. You don’t need to purchase additional storage when you have a self-managed cloud database.

Healthcare 130

Healthcare Hardware Database Scalability 130

Dynatrace

NOVEMBER 11, 2020

” Meaning, you don’t need to worry about OneAgent updates because they’re performed automatically, safely, and promptly as soon as new versions are available (no manual effort required). Easier rollout thanks to log storage best practices. Easier rollout thanks to log storage best practices.

Best Practices 199

Best Practices Storage Java Benchmarking 199

Percona

MARCH 28, 2023

33: Sun Mar 26 08:35:59 2022 (size 8 KB) file-size: 8 KB, checkpoint-size: 4 KB offset, size, checksum root : 8192, 4096, 3824871989 (0xe3faea35) alloc: 12288, 4096, 4074814944 (0xf2e0bde0) discard : 0, 0, 0 (0) available : 0, 0, 0 (0) WiredTigerCheckpoint.34: . $ wt list -c WiredTigerCheckpoint.33:

Hardware 92

Hardware C++ Storage Cache 92

Dynatrace

AUGUST 27, 2020

Dynatrace’s ability to ingest metrics from all 95 AWS services will be available within the next 60 days. Those in the left column are readily available now, with those in the right available soon. Available Now. AWS Storage Gateway. Achieve full observability of all AWS services. Coming Soon. AWS AppSync.

AWS 271

AWS Cloud IoT Database 271

Dynatrace

SEPTEMBER 8, 2023

To achieve this, we’ll create a Grail bucket specifically designed to retain data for a duration of 10 years (3,657 days). In the above recording, we demonstrate an alert specifically designed to notify when there is a significant increase in pending transactions.

Database 272

Database Artificial Intelligence Metrics Monitoring 272