DZone

JUNE 4, 2019

NVMe Storage Use Cases. NVMe storage's strong performance, combined with the capacity and data availability benefits of shared NVMe storage over local SSD, makes it a strong solution for AI/ML infrastructures of any size. There are several AI/ML focused use cases to highlight.

Storage 100

Storage FinTech Artificial Intelligence Performance 100

Dynatrace

MAY 14, 2020

By vastly increasing the number of PurePaths that are processed by a Dynatrace Managed cluster, your initial sizing considerations for Dynatrace Managed nodes and clusters may however end up being inadequate for supporting such volume. A Dynatrace Managed cluster may lack the necessary hardware to process all the additional incoming data.

Processing 153

Processing Hardware Traffic Storage 153

Percona

FEBRUARY 1, 2023

This post will look at using The Oversized-Attribute Storage Technique (TOAST) to improve performance and scalability. This process is done automatically and does not significantly impact how the database is used. text, bytea), and “strategy” is one of the four TOAST storage strategies (PLAIN, EXTENDED, EXTERNAL, MAIN).

Storage 88

Storage Scalability Strategy Performance 88

The Morning Paper

MARCH 8, 2020

Building an elastic query engine on disaggregated storage , Vuppalapati, NSDI’20. Snowflake is a data warehouse designed to overcome these limitations, and the fundamental mechanism by which it achieves this is the decoupling (disaggregation) of compute and storage. joins) during query processing. Disaggregation (or not).

Storage 112

Storage Engineering Cache Serverless 112

Dynatrace

APRIL 24, 2024

To transparently manage expectations and maintain trust with our customers, we expanded the Dynatrace SLA beyond accessing the user interface to cover the full range of relevant product categories, such as processing and retaining incoming data, accessing and working with data, and triggering automations.

Azure 240

Azure Infrastructure Metrics AWS 240

DZone

NOVEMBER 25, 2023

We often dwell on the technical aspects of database selection, focusing on performance metrics , storage capacity, and querying capabilities. But if your application primarily revolves around batch processing of large datasets, then focusing on write speed could mislead your selection process.

Database 268

Database Latency Speed Metrics 268

KeyCDN

FEBRUARY 13, 2020

Push Zones are now seamlessly supporting Image Processing ! The complete feature set of Image Processing is now also available for Push Zones. Our edge servers are directly linked to our global storage cluster, which ensures faster loading times of images. It only takes a few steps to upload and transform images.

Processing 78

Processing Storage Latency Servers 78

DZone

OCTOBER 24, 2023

Logstash: a log-processing tool that collects logs from various sources, parses them, and sends them to Elasticsearch for storage and analysis. Kibana: A powerful visualization tool that allows you to explore and analyze the data stored in Elasticsearch using interactive charts, graphs, and dashboards.

Analytics 255

Analytics Storage Infrastructure Scalability 255

The Morning Paper

JANUARY 28, 2020

Narrowing the gap between serverless and its state with storage functions , Zhang et al., Shredder is " a low-latency multi-tenant cloud store that allows small units of computation to be performed directly within storage nodes. " In from of them is a networking layer, and the in-memory storage layer holds the actual data.

Serverless 99

Serverless Storage Latency Hardware 99

Scalegrid

MAY 13, 2020

Greenplum Database is a massively parallel processing (MPP) SQL database that is built and based on PostgreSQL. High performance, query optimization, open source and polymorphic data storage are the major Greenplum advantages. Greenplum uses an MPP database design that can help you develop a scalable, high performance deployment.

Big Data 321

Big Data Database Artificial Intelligence Open Source 321

DZone

JUNE 6, 2023

When dealing with IoT, one of the first things that come to mind is the limited processing, networking, and storage capabilities these devices operate with. A messaging protocol is a set of rules and formats that are agreed upon among entities that want to communicate with each other.

IoT 293

IoT Tuning Storage Network 293

DZone

MARCH 29, 2023

Data migration is the process of moving data from one location to another, which is an essential aspect of cloud migration. Data migration involves transferring data from on-premise storage to the cloud. With the rapid adoption of cloud computing , businesses are moving their IT infrastructure to the cloud.

Best Practices 304

Best Practices Cloud Storage Data Engineering 304

DZone

FEBRUARY 23, 2023

Introduction With big data streaming platform and event ingestion service Azure Event Hubs , millions of events can be received and processed in a single second. Any real-time analytics provider or batching/storage adaptor can transform and store data supplied to an event hub.

Azure 294

Azure Big Data Analytics Storage 294

High Scalability

JANUARY 11, 2022

There are two major processes which gets executed when a user posts a photo on Instagram. Firstly, the synchronous process which is responsible for uploading image content on file storage, persisting the media metadata in graph data-storage, returning the confirmation message to the user and triggering the process to update the user activity.

Design 334

Design Media Storage Logistics 334

Dynatrace

OCTOBER 30, 2023

Unified observability and security in the development process are crucial to defining ownership in the development process. Dynatrace AutomationEngine fully automates this entire process. AutomationEngine links these processes to your team’s progressive delivery pipelines. This context arrives automatically.

Engineering 245

Engineering DevOps Innovation Storage 245

DZone

AUGUST 16, 2023

Caching is the process of storing frequently accessed data or resources in a temporary storage location, such as memory or disk, to improve retrieval speed and reduce the need for repetitive processing.

Cache 246

Cache Scalability Performance Latency 246

DZone

APRIL 11, 2023

Data processing in the cloud has become increasingly popular due to its scalability, flexibility, and cost-effectiveness. This article will explore how these technologies can be used together to create an optimized data pipeline for data processing in the cloud.

Azure 246

Azure Analytics Storage Cloud 246

Scalegrid

FEBRUARY 1, 2024

Having MySQL backups for your database can speed up and simplify the recovery process. The biggest drawbacks are that a full backup can be time-consuming, and they require a significant amount of storage space. This backup type can save on storage space and reduce backup time by only capturing the changes made since the last backup.

Best Practices 130

Best Practices Storage Strategy Database 130

Dynatrace

OCTOBER 4, 2022

Data warehouses offer a single storage repository for structured data and provide a source of truth for organizations. However, organizations must structure and store data inputs in a specific format to enable extract, transform, and load processes, and efficiently query this data. Massively parallel processing. Query language.

Artificial Intelligence 208

Artificial Intelligence Analytics Storage Government 208

Dynatrace

OCTOBER 4, 2022

This architecture offers rich data management and analytics features (taken from the data warehouse model) on top of low-cost cloud storage systems (which are used by data lakes). This decoupling ensures the openness of data and storage formats, while also preserving data in context. Ingest and process with Grail. Retain data.

Artificial Intelligence 267

Artificial Intelligence Analytics Storage Architecture 267

Dynatrace

OCTOBER 17, 2023

As a result, IT organizations are overwhelmed as they strive to balance cost control processes with ensuring that their respective organizations have access to all the data required for their various use cases. All data is readily accessible without storage tiers, such as costly solid-state drives (SSDs). Ingest and process.

Retail 236

Retail Storage Best Practices Architecture 236

DZone

MAY 5, 2023

Apache Kafka and Apache Flink are increasingly joining forces to build innovative real-time stream processing applications. The core of Kafka is messaging at any scale in combination with a distributed storage (= commit log) for reliable durability, decoupling of applications, and replayability of historical data.

Open Source 246

Open Source Storage Innovation Engineering 246

DZone

JUNE 15, 2023

It is worth noting that this data collection process does not impact the performance of the application. Moreover, the process of collecting these profiles introduces overhead during application runtime and necessitates the storage and visualization of significantly large datasets.

Latency 246

Latency Storage Strategy Metrics 246

Scalegrid

JANUARY 5, 2024

Despite initial investment costs, DBMS presents long-term savings and improved efficiency through automated processes, efficient query optimizations, and scalability, contributing to enhanced decision-making and end-user productivity. It provides tools for organizing and retrieving data efficiently.

Efficiency 147

Efficiency Storage Database Scalability 147

Dynatrace

OCTOBER 7, 2021

This means you no longer have to provision, scale, and maintain servers to run your applications, databases, and storage systems. Speed is next; serverless solutions are quick to spin up or down as needed, and there are no delays due to limited storage or resource access. AWS offers four serverless offerings for storage.

Serverless 280

Serverless AWS Lambda Storage 280

Dynatrace

DECEMBER 5, 2023

Streamline privacy requirements with flexible retention periods Data retention is a critical aspect of data handling, and it’s not just about privacy compliance—it’s about having the flexibility to optimize data storage times in Grail for your Dynatrace use cases. Other data types will be available soon). What’s next?

Storage 166

Storage Healthcare Best Practices Speed 166

DZone

DECEMBER 14, 2021

Metadata synchronization (sync) is a core feature in Alluxio that keeps files and directories consistent with their source of truth in under-storage systems, thus making it simple for users to reason the data retrieved from Alluxio. Meanwhile, understanding the internal process is important in order to tune the performance.

Design 244

Design Storage Tuning Systems 244

Uber Engineering

MARCH 12, 2017

With the evolution of storage formats like Apache Parquet and Apache ORC and query engines like Presto and Apache Impala , the Hadoop ecosystem has the potential to become a general-purpose, unified serving layer for workloads that can tolerate latencies … The post Hudi: Uber Engineering’s Incremental Processing Framework on Apache Hadoop appeared (..)

Processing 105

Processing Latency Storage Engineering 105

Percona Community

JUNE 1, 2022

An important role is played by the time of their storage. Often, in order to understand certain processes and predict their development in the future, we need to analyze metrics over a fairly long period of time. Introduction Nowadays, the main tools for monitoring the operation of any application are metrics and logs.

Storage 52

Storage Metrics Monitoring Processing 52

Scalegrid

MARCH 28, 2024

This article will explore how they handle data storage and scalability, perform in different scenarios, and, most importantly, how these factors influence your choice. It uses a hash table to manage these pairs, divided into fixed-size buckets with linked lists for key-value storage. step in to simplify this task.

Cache 130

Cache Storage Scalability Architecture 130

Dynatrace

OCTOBER 23, 2023

Secondly, determining the correct allocation of resources (CPU, memory, storage) to each virtual machine to ensure optimal performance without over-provisioning can be difficult. Firstly, managing virtual networks can be complex as networking in a virtual environment differs significantly from traditional networking.

Efficiency 213

Efficiency Virtualization Hardware Performance 213

Dynatrace

JANUARY 4, 2024

Improving data quality is a strategic process that involves all organizational members who create and use data. It starts with implementing data governance practices, which set standards and policies for data use and management in areas such as quality, security, compliance, storage, stewardship, and integration.

Government 199

Government Analytics Benchmarking Storage 199

Dynatrace

MARCH 6, 2024

Manual monitoring processes are also too time-consuming, which distracts teams from tasks that create new value for customers and the business. The report shows that log analytics are a particular challenge, as the long-term storage cost of all this data has begun to overshadow the value organizations can unlock from querying it.

Analytics 190

Analytics Innovation Strategy Storage 190

DZone

JULY 3, 2023

Data engineering projects often require the setup and management of complex infrastructures that support data processing, storage, and analysis. Traditionally, this process involved manual configuration, leading to potential inconsistencies, human errors, and time-consuming deployments.

Data Engineering 141

Data Engineering Infrastructure Engineering Code 141

Dynatrace

JULY 29, 2022

Log analytics is the process of viewing, interpreting, and querying log data so developers and IT teams can quickly detect and resolve application and system issues. Cold storage and rehydration. Cold storage and rehydration. Data that organizations may need to access only once a quarter or year can reside in cold storage.

Analytics 214

Analytics Storage Retail DevOps 214

Highly Scalable

AUGUST 20, 2013

The shortcomings and drawbacks of batch-oriented data processing were widely recognized by the Big Data community quite a long time ago. It became clear that real-time query processing and in-stream processing is the immediate need in many practical applications. Fault-tolerance. Interoperability with Hadoop.

Big Data 154

Big Data Processing Lambda Database 154

Scalegrid

MARCH 8, 2024

Executing PITR requires restoring from the full backup and then applying binary log events in sequence up to the desired point in time, with advanced techniques and third-party tools available to optimize large dataset handling and automate the recovery process. Each caters to specific needs.

Database 162

Database Strategy Servers Best Practices 162