article thumbnail

Storage Types Used on Cloud Computing Platforms

DZone

Because of the emergence of cloud services, a broad range of storage choices are now easily available to fulfill the different demands of both organizations and people. These storage alternatives have been designed to meet a range of requirements, including performance, scalability, durability, and price.

Storage 266
article thumbnail

Building and operating a pretty big storage system called S3

All Things Distributed

Three distinct perspectives on scale that come along with building and operating a storage system the size of S3.

Storage 147
Insiders

Sign Up for our Newsletter

This site is protected by reCAPTCHA and the Google Privacy Policy and Terms of Service apply.

article thumbnail

Introduction to Azure Data Lake Storage Gen2

DZone

Built on Azure Blob Storage, Azure Data Lake Storage Gen2 is a suite of features for big data analytics. Azure Data Lake Storage Gen1 and Azure Blob Storage's capabilities are combined in Data Lake Storage Gen2. For instance, Data Lake Storage Gen2 offers scale, file-level security, and file system semantics.

Azure 243
article thumbnail

Article: Magic Pocket: Dropbox’s Exabyte-Scale Blob Storage System

InfoQ

A horizontally scalable exabyte-scale blob storage system which operates out of multiple regions, Magic Pocket is used to store all of Dropbox’s data. Adopting SMR technology and erasure codes, the system has extremely high durability guarantees but is cheaper than operating in the cloud. By Facundo Agriel

Storage 107
article thumbnail

What is log management? How to tame distributed cloud system complexities

Dynatrace

Log management is an organization’s rules and policies for managing and enabling the creation, transmission, analysis, storage, and other tasks related to IT systems’ and applications’ log data. Distributed cloud systems are complex, dynamic, and difficult to manage without the proper tools. Dynatrace news.

Cloud 197
article thumbnail

Why Compute and Storage Should Be Decoupled for Log Management at Scale

DZone

On a small scale, this isn’t problematic but when dealing with large-scale deployments, organizations end up using lots of computing, storage, and human resources just to manage their indexes. When companies are handling terabytes of data every day, the database-backed log management system becomes untenable.

Storage 189
article thumbnail

The Block Allocation Policy of Virtual Distributed File System at the Source Code Level

DZone

Users can allocate different storage tiers as the resources for Alluxio workers, including MEM/SSD/HDD, which are further composed of directories. Alluxio workers are responsible for managing local resources, and they store data as blocks.

Code 246