article thumbnail

Building an elastic query engine on disaggregated storage

The Morning Paper

Building an elastic query engine on disaggregated storage , Vuppalapati, NSDI’20. Snowflake is a data warehouse designed to overcome these limitations, and the fundamental mechanism by which it achieves this is the decoupling (disaggregation) of compute and storage. joins) during query processing. Disaggregation (or not).

Storage 112
article thumbnail

Log Analysis: How to Digest 15 Billion Logs Per Day and Keep Big Queries Within 1 Second

DZone

From an architectural perspective, the system should be able to undertake real-time analysis of various formats of logs, and of course, be scalable to support the huge and ever-enlarging data size.

Analytics 124
Insiders

Sign Up for our Newsletter

This site is protected by reCAPTCHA and the Google Privacy Policy and Terms of Service apply.

Trending Sources

article thumbnail

What is a Distributed Storage System

Scalegrid

A distributed storage system is foundational in today’s data-driven landscape, ensuring data spread over multiple servers is reliable, accessible, and manageable. Understanding distributed storage is imperative as data volumes and the need for robust storage solutions rise.

Storage 130
article thumbnail

A MyRocks Use Case

Percona

I wrote this post on MyRocks because I believe it is the most interesting new MySQL storage engine to have appeared over the last few years. I decided to use a virtual machine with two CPU cores, four GB of memory, and storage limited to a maximum of 1000 IOPs of 16KB. I emulated these limits using the KVM iotune settings in my lab. <iotune>

Storage 55
article thumbnail

Best MySQL DigitalOcean Performance – ScaleGrid vs. DigitalOcean Managed Databases

Scalegrid

ScaleGrid provides 30% more storage on average vs. DigitalOcean for MySQL at the same affordable price. As you can see above, ScaleGrid and DigitalOcean offer the same plan configurations across this plan size, apart from SSD where ScaleGrid provides over 20% more storage for the same price. Compare Latency. Compare Pricing.

Database 217
article thumbnail

Announcing the Availability of Percona Backup for MongoDB 2.2.0

Percona

PBM) introduced a GA version of incremental physical backups , which can greatly impact both the recovery time and the cost of backup (considering storage and data transfer cost). Of course, each release also includes bug fixes and refining some of the existing features. In the previous minor release, Percona Backup for MongoDB 2.1.0

article thumbnail

Follower Clusters – 3 Major Use Cases for Syncing SQL & NoSQL Deployments

Scalegrid

Note: ScaleGrid implements follower clusters using storage snapshots. Not actual production of course – that will be flirting with disaster. And since the entire import is performed using storage snapshots, rather than a logical dump, the process is nearly instantaneous. This is not something you can do on a replica node.

Analytics 185