Scalegrid

JULY 17, 2020

It is an open standard format which organizes data into key/value pairs and arrays detailed in RFC 7159. JSON is the most common format used by web services to exchange data, store documents, unstructured data, etc. You can also check out our Working with JSON Data in PostgreSQL vs. JSONB Patterns & Antipatterns.

Storage 321

Storage Database Efficiency Processing 321

Dynatrace

SEPTEMBER 7, 2022

Dynatrace and the Dynatrace Intelligent Observability Platform have added support for the newly introduced Amazon VPC Flow Logs to Amazon Kinesis Data Firehose. This support enables customers to define specific endpoint delivery of real-time streaming data to platforms such as Dynatrace. What is VPC Flow Logs? Why Dynatrace?

Traffic 218

Traffic AWS Network Cloud 218

Uber Engineering

AUGUST 14, 2019

Maintaining Uber’s large-scale data warehouse comes with an operational cost in terms of ETL functions and storage. Once identified, … The post Less is More: Engineering Data Warehouse Efficiency with Minimalist Design appeared first on Uber Engineering Blog.

Efficiency 103

Efficiency Engineering Design Storage 103

DZone

MAY 5, 2023

This blog post explores the benefits of combining both open-source frameworks, shows unique differentiators of Flink versus Kafka, and discusses when to use a Kafka-native streaming engine like Kafka Streams instead of Flink.

Open Source 246

Open Source Storage Innovation Engineering 246

Dynatrace

NOVEMBER 28, 2023

In the previous installment of this blog series , we explored how to set up Dynatrace as a build-stage orchestrator to effectively address the challenges faced by Site Reliability Engineers (SREs). Davis AI can leverage this data to enable predictive analysis. This approach effectively combats configuration drift.

Traffic 264

Traffic Best Practices Strategy Engineering 264

Dynatrace

MARCH 23, 2020

A decade ago, while working for a large hosting provider, I led a team that was thrown into turmoil over the purchasing of server and storage hardware in preparation for a multi-million dollar super-bowl ad campaign. Our procurement decisions were based on trace data that was pulled from a handful of fragmented monitoring solutions.

Google 194

Google Engineering Metrics Hardware 194

Dynatrace

MAY 7, 2024

Fluent Bit is a telemetry agent designed to receive data (logs, traces, and metrics), process or modify it, and export it to a destination. Fluent Bit can serve as a proxy before you send data to Dynatrace or similar. However, you can also use Fluent Bit as a processor because you can perform various actions on the data.

Best Practices 203

Best Practices IoT Metrics Storage 203

The Netflix TechBlog

APRIL 27, 2023

Engineers want their alerting system to be realtime, reliable, and actionable. A few years ago, we were paged by our SRE team due to our Metrics Alerting System falling behind — critical application health alerts reached engineers 45 minutes late! Data expressions define what data needs to be sourced in order to evaluate a query.

Storage 288

Storage Cache Metrics Database 288

The Netflix TechBlog

NOVEMBER 29, 2023

As noted in an earlier blog post , we began developing an HDR variant of VMAF; let’s call it HDR-VMAF. A vital aspect of such development is subjective testing with HDR encodes in order to generate training data. The pandemic, however, posed unique challenges in conducting a conventional in-lab subjective test with HDR encodes.

Open Source 252

Open Source Software Engineering Internet Internet 252

Dynatrace

APRIL 8, 2022

Fluentd is an open-source data collector that unifies log collection, processing, and consumption. Output plugins deliver logs to storage solutions, analytics tools, and observability platforms like Dynatrace. Built-in resiliency ensures data completeness and consistency even if Fluentd or an endpoint service goes down temporarily.

Metrics 182

Metrics Analytics Software Architecture Open Source 182

Percona

DECEMBER 19, 2023

PG_TDE (PostgreSQL Transparent Data Encryption) is an experimental transparent data encryption access method for PostgreSQL 16, available on GitHub as an extension for vanilla PostgreSQL. Today, we’d like to share more details about the extension, including information about its internal architecture.

Open Source 87

Open Source Storage Database C++ 87

Percona

AUGUST 9, 2023

MySQL 8 brought a significant architectural transformation by replacing the traditional MyISAM-based system tables with the Transaction Data Dictionary (TDD), a more efficient and reliable approach. The primary goal of the Transaction Data Dictionary (TDD) is to enhance the overall performance, stability, and scalability of MySQL databases.

Database 82

Database Storage Scalability Games 82

Percona

SEPTEMBER 14, 2023

However, modern business is very much dependent on analytical data. I recommend you read our previous blog, Using ClickHouse as an Analytic Extension for MySQL , from Vadim to know more about this. In this blog post, I will show you how to replicate MySQL data in real-time to ClickHouse. We can do the filter as well.)

Engineering 92

Engineering Database Servers Analytics 92

Dynatrace

JULY 29, 2022

Log analytics is the process of viewing, interpreting, and querying log data so developers and IT teams can quickly detect and resolve application and system issues. At the same time, log analytics can present challenges as data volumes explode, particularly in traditional environments that lack end-to-end observability solutions.

Analytics 217

Analytics Storage Retail Monitoring 217

Dynatrace

OCTOBER 4, 2022

Several pain points have made it difficult for organizations to manage their data efficiently and create actual value. Limited data availability constrains value creation. Modern IT environments — whether multicloud, on-premises, or hybrid-cloud architectures — generate exponentially increasing data volumes.

Analytics 237

Analytics Artificial Intelligence Storage Serverless 237

Percona

MARCH 21, 2023

In this blog, I’ll discuss the use case for replication. We want to improve our ability to replicate your data and limit replication to row-based events securely, wherein we do not have control over the source(s). PRIVILEGE CHECKS USER is helpful in multi-source replication to carefully gather data from various sources.

Storage 91

Storage Open Source Database Engineering 91

Dynatrace

JUNE 25, 2020

With Dynatrace actively managing business-critical applications, some of our globally distributed enterprise customers require Dynatrace Managed to continue operating even when an entire data center goes down. Near-zero RPO and RTO—monitoring continues seamlessly and without data loss in failover scenarios.

Availability 249

Availability Hardware Latency Traffic 249

Dynatrace

JULY 29, 2022

Log analytics is the process of viewing, interpreting, and querying log data so developers and IT teams can quickly detect and resolve application and system issues. At the same time, log analytics can present challenges as data volumes explode, particularly in traditional environments that lack end-to-end observability solutions.

Analytics 183

Analytics Storage Retail Monitoring 183

The Netflix TechBlog

OCTOBER 19, 2020

a Netflix member via Twitter This is an example of a question our on-call engineers need to answer to help resolve a member issue?—?which In our previous blog post we introduced Edgar, our troubleshooting tool for streaming sessions. We needed to increase engineering productivity via distributed request tracing.

Infrastructure 233

Infrastructure Transportation Storage Open Source 233

Percona

APRIL 20, 2023

Data-at-rest encryption is essential for compliance with regulations that require the protection of sensitive data. It is also critical for securing sensitive data and avoiding data breaches. PostgreSQL does not natively support Transparent Data Encryption (TDE). apiVersion: storage.k8s.io/v1

Storage 124

Storage AWS Cloud Database 124

Dynatrace

APRIL 8, 2022

Fluentd is an open-source data collector that unifies log collection, processing, and consumption. Output plugins deliver logs to storage solutions, analytics tools, and observability platforms like Dynatrace. Built-in resiliency ensures data completeness and consistency even if Fluentd or an endpoint service goes down temporarily.

Metrics 130

Metrics Analytics Software Architecture Open Source 130

Dynatrace

OCTOBER 7, 2021

This means you no longer have to provision, scale, and maintain servers to run your applications, databases, and storage systems. Speed is next; serverless solutions are quick to spin up or down as needed, and there are no delays due to limited storage or resource access. Data Store. Improving data processing.

Serverless 283

Serverless AWS Lambda Storage 283

Dynatrace

FEBRUARY 10, 2023

Data proliferation—as well as a growing need for data analysis—has accelerated. According to recent Dynatrace data, 59% of CIOs say the increasing complexity of their technology stack could soon overload their teams without a more automated approach to IT operations. Digital transformation shows no signs of slowing down.

Efficiency 278

Efficiency Performance Analytics DevOps 278

Percona

MARCH 7, 2023

One approach is to separate compute and storage to allow for independent scaling. By separating storage and compute, Neon replaces the PostgreSQL storage layer with data nodes, and compute nodes are distributed across a cluster of nodes. Compute nodes are stateless PostgreSQL nodes backed by the Neon storage engine.

Serverless 115

Serverless Storage Open Source Architecture 115

Percona

MARCH 29, 2023

Compression in any database is necessary as it has many advantages, like storage reduction, data transmission time, etc. Storage reduction alone results in significant cost savings, and we can save more data in the same space. In this blog, we will discuss both data and network-level compression offered in MongoDB.

Storage 113

Storage Network Open Source Latency 113

Percona

OCTOBER 13, 2023

A common practice among DBAs and developers is to copy table data and.frm files from the data dictionary. In the case of InnoDB tablespaces, this information is incorporated into the tablespace itself, creating a fusion of metadata and data primarily to enhance performance. In MySQL 8.0, ibd files), general tablespace files (.ibd

Storage 98

Storage Database Engineering Open Source 98

Percona

JANUARY 5, 2023

Space constraint has been an endless and painstaking process for many of us, especially in systems that have a high number of transactions and data growth exceeding hundreds of GBs in a matter of days. MySQL won’t shrink tablespace and storage won’t be released to the file system. What is partitioning? 1 mysql mysql 9.0G

Traffic 61

Traffic Engineering Speed Storage 61

Percona

AUGUST 29, 2023

Its query engine not only parallelizes incoming SQL queries for time series but also has the ability to create column-wise tables making it ideal for OLAP analysis duties. The distributed query engine can route and parallelize SELECT, DML, and other operations on distributed tables across the cluster.

C++ 84

C++ Database Storage Testing 84

Dynatrace

MAY 26, 2021

From May 17 to May 18, 2021, the Open-Source Engineering team at Dynatrace attended the virtual observability conference, o11yfest. This blog dives into what we learned by listening to experts in the field and the key takeaways we gathered from the talks. A recurring theme was Site Reliability Engineering. We second that.

AWS 215

AWS Strategy Storage Metrics 215

Dynatrace

JULY 25, 2022

This new service enhances the user visibility of network details with direct delivery of Flow Logs for Transit Gateway to your desired endpoint via Amazon Simple Storage Service (S3) bucket or Amazon CloudWatch Logs. VPC Flow Logs is a feature that gives you the capability to capture more robust IP traffic data that traverses your VPCs.

AWS 220

AWS Transportation Network Traffic 220

Percona

DECEMBER 20, 2018

Let’s look at some of the most popular Percona Database Performance Blog posts in 2018. With the Percona Database Performance Blog, Percona staff and leadership work hard to provide the open source community with insights, technical support, predictions and metrics around multiple open source database software technologies.

Database 41

Database Tuning Performance AWS 41

Percona

JULY 13, 2023

With the rise of globalization, the need to store and process multilingual data has become essential. In this blog post, we will explore utf8mb4 and its advantages in MySQL 8.0, The first two rows contain simple text data, while the third row contains an emoji. version as a game-changer. backed by practical examples.

Database 95

Database Games Open Source Engineering 95

Percona

JANUARY 30, 2023

As Ibrar Ahmed noted in his blog post on Transparent Database Encryption (TDE). Instead, it seems PostgreSQL Developers are of the opinion that encryption is a storage-level problem and is better solved on the filesystem or block device level. Does PostgreSQL Need TDE (Transparent Data Encryption)? It adds defense in depth.

Database 129

Database Storage Government Open Source 129

Percona

DECEMBER 11, 2023

A Dedicated Log Volume (DLV) is a specialized storage volume designed to house database transaction logs separately from the volume containing the database tables. DLVs are particularly advantageous for databases with large allocated storage, high I/O per second (IOPS) requirements, or latency-sensitive workloads.

AWS 101

AWS Benchmarking Performance Traffic 101

The Netflix TechBlog

DECEMBER 21, 2020

By Anupom Syam Background At Netflix, our current data warehouse contains hundreds of Petabytes of data stored in AWS S3 , and each day we ingest and create additional Petabytes. We built AutoOptimize to efficiently and transparently optimize the data and metadata storage layout while maximizing their cost and performance benefits.

Storage 203

Storage Latency Efficiency Data Engineering 203

Dynatrace

JULY 29, 2021

The complexity of such deployments has accelerated with the adoption of emerging, open-source technologies that generate telemetry data, which is exploding in terms of volume, speed, and cardinality. All this data is then consumed by Dynatrace Davis® AI for more precise answers, thereby driving AIOps for cloud-native environments.

Analytics 246

Analytics Open Source Serverless Architecture 246

Scalegrid

JANUARY 20, 2020

Unlike cloning or replication, this allows you to maintain an active, point-in-time copy of your production data. In this blog post, we will cover the top three scenarios to leverage follower clusters for your application. In this blog post, we will cover the top three scenarios to leverage follower clusters for your application.

Analytics 185

Analytics Storage Database Systems 185