Scalegrid

MAY 13, 2020

It can scale towards a multi-petabyte level data workload without a single issue, and it allows access to a cluster of powerful servers that will work together within a single SQL interface where you can view all of the data. This feature-packed database provides powerful and rapid analytics on data that scales up to petabyte volumes.

Big Data 321

Big Data Database Artificial Intelligence Open Source 321

Dynatrace

JANUARY 23, 2024

AI data analysis can help development teams release software faster and at higher quality. So how can organizations ensure data quality, reliability, and freshness for AI-driven answers and insights? And how can they take advantage of AI without incurring skyrocketing costs to store, manage, and query data?

Performance 184

Performance DevOps Innovation Artificial Intelligence 184

Scalegrid

JULY 17, 2020

It is an open standard format which organizes data into key/value pairs and arrays detailed in RFC 7159. JSON is the most common format used by web services to exchange data, store documents, unstructured data, etc. You can also check out our Working with JSON Data in PostgreSQL vs. JSONB Patterns & Antipatterns.

Storage 321

Storage Database Efficiency Processing 321

The Netflix TechBlog

OCTOBER 28, 2021

Data Engineers of Netflix?—?Interview Interview with Pallavi Phadnis This post is part of our “ Data Engineers of Netflix ” series, where our very own data engineers talk about their journeys to Data Engineering @ Netflix. Pallavi Phadnis is a Senior Software Engineer on the Product Data Science and Engineering team.

Data Engineering 173

Data Engineering Engineering Big Data Software Engineering 173

Dynatrace

SEPTEMBER 7, 2022

Dynatrace and the Dynatrace Intelligent Observability Platform have added support for the newly introduced Amazon VPC Flow Logs to Amazon Kinesis Data Firehose. This support enables customers to define specific endpoint delivery of real-time streaming data to platforms such as Dynatrace. What is VPC Flow Logs? Why Dynatrace?

Traffic 214

Traffic AWS Network Cloud 214

Dynatrace

MAY 7, 2024

Fluent Bit is a telemetry agent designed to receive data (logs, traces, and metrics), process or modify it, and export it to a destination. Fluent Bit can serve as a proxy before you send data to Dynatrace or similar. However, you can also use Fluent Bit as a processor because you can perform various actions on the data.

Best Practices 241

Best Practices IoT Metrics Storage 241

Dynatrace

SEPTEMBER 23, 2020

This blog post highlights a group of improvements that were released with OneAgent version 1.199. Recent improvements in OneAgent runtime-data handling. Storage mount points in a system might be larger or smaller, local or remote, with high or low latency, and various speeds. Customizable location of large runtime files.

Storage 147

Storage Latency Operating System Network 147

The Netflix TechBlog

OCTOBER 15, 2019

Part I: Overview Andreas Andreakis , Falguni Jhaveri , Ioannis Papapanagiotou , Mark Cho , Poorna Reddy , Tongliang Liu Overview It is a commonly observed pattern for applications to utilize multiple datastores where each is used to serve a specific need such as storing the canonical form of data (MySQL etc.), caching (Memcached etc.),

Transportation 192

Transportation Architecture Processing Storage 192

DZone

MAY 5, 2023

This blog post explores the benefits of combining both open-source frameworks, shows unique differentiators of Flink versus Kafka, and discusses when to use a Kafka-native streaming engine like Kafka Streams instead of Flink. The Tremendous Adoption of Apache Kafka and Apache Flink Apache Kafka became the de facto standard for data streaming.

Open Source 246

Open Source Storage Innovation Engineering 246

Percona

DECEMBER 19, 2023

PG_TDE (PostgreSQL Transparent Data Encryption) is an experimental transparent data encryption access method for PostgreSQL 16, available on GitHub as an extension for vanilla PostgreSQL. Today, we’d like to share more details about the extension, including information about its internal architecture.

Open Source 84

Open Source Storage Database C++ 84

The Netflix TechBlog

APRIL 27, 2023

Atlas is an in-memory time-series database that ingests multiple billions of time-series per day and retains the last two weeks of data. To serve the increasing number of push down queries, the in-memory storage layer would need to scale up as well, and it became clear that this would push the already expensive storage costs far higher.

Storage 288

Storage Cache Metrics Database 288

Dynatrace

OCTOBER 17, 2023

In today’s data-driven landscape, businesses are grappling with an unprecedented surge in data volume. The data generated by applications to support operational teams has reached overwhelming proportions. This data overload also prevents customer-centric pricing models as users consider cost-effective technology platforms.

Retail 235

Retail Storage Best Practices Architecture 235

Percona

AUGUST 9, 2023

MySQL 8 brought a significant architectural transformation by replacing the traditional MyISAM-based system tables with the Transaction Data Dictionary (TDD), a more efficient and reliable approach. The primary goal of the Transaction Data Dictionary (TDD) is to enhance the overall performance, stability, and scalability of MySQL databases.

Database 79

Database Storage Scalability Games 79

Dynatrace

APRIL 8, 2022

Fluentd is an open-source data collector that unifies log collection, processing, and consumption. Output plugins deliver logs to storage solutions, analytics tools, and observability platforms like Dynatrace. Built-in resiliency ensures data completeness and consistency even if Fluentd or an endpoint service goes down temporarily.

Metrics 178

Metrics Analytics Software Architecture Open Source 178

Dynatrace

APRIL 19, 2023

In Part 1 of this blog post series , we introduced the concept of security policies in preparation for upcoming enhancements to the Dynatrace platform. ALLOW storage:system:read; The Storage All System Data Read policy grants access to Dynatrace internal data such as auditing events and query execution events. .*”;ALLOW

Storage 207

Storage Analytics Systems Architecture 207

Dynatrace

JUNE 25, 2020

With Dynatrace actively managing business-critical applications, some of our globally distributed enterprise customers require Dynatrace Managed to continue operating even when an entire data center goes down. Near-zero RPO and RTO—monitoring continues seamlessly and without data loss in failover scenarios.

Availability 246

Availability Hardware Latency Traffic 246

Dynatrace

NOVEMBER 28, 2023

In the previous installment of this blog series , we explored how to set up Dynatrace as a build-stage orchestrator to effectively address the challenges faced by Site Reliability Engineers (SREs). Davis AI can leverage this data to enable predictive analysis. To illustrate this concept, consider the scenario below. What’s next?

Traffic 261

Traffic Best Practices Strategy Engineering 261

Dynatrace

OCTOBER 4, 2022

Several pain points have made it difficult for organizations to manage their data efficiently and create actual value. Limited data availability constrains value creation. Modern IT environments — whether multicloud, on-premises, or hybrid-cloud architectures — generate exponentially increasing data volumes.

Analytics 233

Analytics Artificial Intelligence Storage Serverless 233

Dynatrace

AUGUST 1, 2022

Kubernetes was initially designed with a strong focus on stateless workloads, meaning these workloads do not need to store any persistent data. Interestingly, our partner RedHat reported in 2021 that around 80% of deployed workloads are databases or data caches, storing data in persistent volume claims (PVCs).

Storage 186

Storage Database Network Metrics 186

Dynatrace

APRIL 8, 2022

Fluentd is an open-source data collector that unifies log collection, processing, and consumption. Output plugins deliver logs to storage solutions, analytics tools, and observability platforms like Dynatrace. Built-in resiliency ensures data completeness and consistency even if Fluentd or an endpoint service goes down temporarily.

Metrics 130

Metrics Analytics Software Architecture Open Source 130

Dynatrace

NOVEMBER 20, 2023

Across the cloud operations lifecycle, especially in organizations operating at enterprise scale, the sheer volume of cloud-native services and dynamic architectures generate a massive amount of data. Generative AI brings data quality risks But generative AI also brings risks in terms of data quality. What is causal AI?

AWS 207

AWS Artificial Intelligence Analytics Cloud 207

Dynatrace

JULY 29, 2022

Log analytics is the process of viewing, interpreting, and querying log data so developers and IT teams can quickly detect and resolve application and system issues. At the same time, log analytics can present challenges as data volumes explode, particularly in traditional environments that lack end-to-end observability solutions.

Analytics 213

Analytics Storage Retail Monitoring 213

The Netflix TechBlog

NOVEMBER 29, 2023

As noted in an earlier blog post , we began developing an HDR variant of VMAF; let’s call it HDR-VMAF. A vital aspect of such development is subjective testing with HDR encodes in order to generate training data. The pandemic, however, posed unique challenges in conducting a conventional in-lab subjective test with HDR encodes.

Open Source 252

Open Source Software Engineering Internet Internet 252

Percona

APRIL 20, 2023

Data-at-rest encryption is essential for compliance with regulations that require the protection of sensitive data. It is also critical for securing sensitive data and avoiding data breaches. PostgreSQL does not natively support Transparent Data Encryption (TDE). apiVersion: storage.k8s.io/v1

Storage 121

Storage AWS Cloud Database 121

Percona

MARCH 21, 2023

In this blog, I’ll discuss the use case for replication. We want to improve our ability to replicate your data and limit replication to row-based events securely, wherein we do not have control over the source(s). PRIVILEGE CHECKS USER is helpful in multi-source replication to carefully gather data from various sources.

Storage 88

Storage Open Source Database Engineering 88

Dynatrace

OCTOBER 7, 2021

This means you no longer have to provision, scale, and maintain servers to run your applications, databases, and storage systems. Speed is next; serverless solutions are quick to spin up or down as needed, and there are no delays due to limited storage or resource access. Data Store. Improving data processing.

Serverless 279

Serverless AWS Lambda Storage 279

Percona

AUGUST 29, 2023

Among its many capabilities, a Citus cluster can: Create distributed tables that are sharded across a cluster of PostgreSQL nodes to combine their CPU, memory, storage, and I/O capacity. Columnar storage of tables can compress data, speeding up scans and supporting fast projections, both on regular and distributed tables.

C++ 81

C++ Database Storage Testing 81

Dynatrace

JULY 29, 2022

Log analytics is the process of viewing, interpreting, and querying log data so developers and IT teams can quickly detect and resolve application and system issues. At the same time, log analytics can present challenges as data volumes explode, particularly in traditional environments that lack end-to-end observability solutions.

Analytics 180

Analytics Storage Retail Monitoring 180

Percona

DECEMBER 20, 2018

Let’s look at some of the most popular Percona Database Performance Blog posts in 2018. With the Percona Database Performance Blog, Percona staff and leadership work hard to provide the open source community with insights, technical support, predictions and metrics around multiple open source database software technologies.

Database 40

Database Tuning Performance AWS 40

Dynatrace

DECEMBER 18, 2023

This challenge has given rise to the discipline of observability engineering, which concentrates on the details of telemetry data to fine-tune observability use cases. To get a more granular look into telemetry data, many analysts rely on custom metrics using Prometheus.

Metrics 176

Metrics Engineering Energy Tuning 176

Uber Engineering

AUGUST 14, 2019

Maintaining Uber’s large-scale data warehouse comes with an operational cost in terms of ETL functions and storage. Once identified, … The post Less is More: Engineering Data Warehouse Efficiency with Minimalist Design appeared first on Uber Engineering Blog.

Efficiency 99

Efficiency Engineering Design Storage 99

Dynatrace

FEBRUARY 10, 2023

Data proliferation—as well as a growing need for data analysis—has accelerated. According to recent Dynatrace data, 59% of CIOs say the increasing complexity of their technology stack could soon overload their teams without a more automated approach to IT operations. Digital transformation shows no signs of slowing down.

Efficiency 274

Efficiency Performance Analytics DevOps 274

Percona

MARCH 7, 2023

One approach is to separate compute and storage to allow for independent scaling. By separating storage and compute, Neon replaces the PostgreSQL storage layer with data nodes, and compute nodes are distributed across a cluster of nodes. Compute nodes are stateless PostgreSQL nodes backed by the Neon storage engine.

Serverless 112

Serverless Storage Open Source Architecture 112

The Netflix TechBlog

DECEMBER 21, 2020

By Anupom Syam Background At Netflix, our current data warehouse contains hundreds of Petabytes of data stored in AWS S3 , and each day we ingest and create additional Petabytes. We built AutoOptimize to efficiently and transparently optimize the data and metadata storage layout while maximizing their cost and performance benefits.

Storage 203

Storage Latency Efficiency Data Engineering 203

Percona

MARCH 29, 2023

Compression in any database is necessary as it has many advantages, like storage reduction, data transmission time, etc. Storage reduction alone results in significant cost savings, and we can save more data in the same space. In this blog, we will discuss both data and network-level compression offered in MongoDB.

Storage 110

Storage Network Open Source Latency 110

Dynatrace

SEPTEMBER 8, 2022

Log management is an organization’s rules and policies for managing and enabling the creation, transmission, analysis, storage, and other tasks related to IT systems’ and applications’ log data. In cloud-native environments, there can also be dozens of additional services and functions all generating data from user-driven events.

Systems 184

Systems Cloud Analytics DevOps 184

All Things Distributed

JANUARY 23, 2012

Expanding the Cloud - The AWS Storage Gateway. Today Amazon Web Services has launched the AWS Storage Gateway, making the power of secure and reliable cloud storage accessible from customersâ?? With the launch of the AWS Storage Gateway our customers can now integrate their on-premises IT environment with AWSâ??s

Storage 115

Storage AWS Virtualization Cloud 115