The Netflix TechBlog

APRIL 27, 2023

Engineers want their alerting system to be realtime, reliable, and actionable. A few years ago, we were paged by our SRE team due to our Metrics Alerting System falling behind — critical application health alerts reached engineers 45 minutes late! The internals here are outside the scope of this blog post.

Storage 288

Storage Cache Metrics Database 288

Uber Engineering

AUGUST 14, 2019

Maintaining Uber’s large-scale data warehouse comes with an operational cost in terms of ETL functions and storage. Once identified, … The post Less is More: Engineering Data Warehouse Efficiency with Minimalist Design appeared first on Uber Engineering Blog.

Efficiency 99

Efficiency Engineering Design Storage 99

Percona

SEPTEMBER 14, 2023

I recommend you read our previous blog, Using ClickHouse as an Analytic Extension for MySQL , from Vadim to know more about this. In this blog post, I will show you how to replicate MySQL data in real-time to ClickHouse. I am going to use the MaterializedMySQL engine for this purpose. The blog post contains the following topics.

Engineering 91

Engineering Database Servers Analytics 91

Dynatrace

JANUARY 23, 2024

Composite’ AI, platform engineering, AI data analysis through custom apps This focus on data reliability and data quality also highlights the need for organizations to bring a “ composite AI ” approach to IT operations, security, and DevOps. How can organizations use AI observability to optimize AI costs? What is explainable AI?

Performance 184

Performance DevOps Innovation Artificial Intelligence 184

DZone

OCTOBER 11, 2022

Making Google’s CalDAV and CardDAV APIs available for everyone ( Google Developers Blog). Email Reveals Google App Engine Search API About Ready For Preview Release, Charges Planned For Storage, Operations ( TechCrunch). Google Analytics Becomes A Robust Testing Platform With Content Experiments API ( Google Analytics Blog).

Java 244

Java Best Practices Google Analytics 244

Uber Engineering

NOVEMBER 1, 2016

Uber Engineering’s Schemaless storage system powers some of the biggest services at Uber, such as Mezzanine. Managing these clusters was fairly easy when we had … The post Dockerizing MySQL at Uber Engineering appeared first on Uber Engineering Blog.

Engineering 111

Engineering Storage Scalability Availability 111

Dynatrace

OCTOBER 4, 2022

This architecture offers rich data management and analytics features (taken from the data warehouse model) on top of low-cost cloud storage systems (which are used by data lakes). This decoupling ensures the openness of data and storage formats, while also preserving data in context. Grail is built for such analytics, not storage.

Artificial Intelligence 266

Artificial Intelligence Analytics Storage Architecture 266

Dynatrace

JULY 29, 2022

As development and site reliability engineering (SRE) teams strive to release software faster, log analytics can provide key insight into software quality as part of a broader DevOps observability and automation initiative. Cold storage and rehydration. Cold storage and rehydration. Better-quality code. Inadequate context.

Analytics 213

Analytics Storage Retail Monitoring 213

Dynatrace

MAY 7, 2024

New processors: Introducing new processors, including Metric Selector and Content Modifier, for selective data processing and metadata adjustment, improving data relevance and storage efficiency. By default, you have a storage type memory, but you may exceed this buffer limit if you have a lot of data. Learn more about Fluent Bit 3.0

Best Practices 241

Best Practices IoT Metrics Storage 241

Dotcom-Montior

OCTOBER 6, 2021

A site reliability engineer, or SRE, is a role that that encompasses aspects of both software engineering and operations/infrastructure. The term site reliability engineering first came into existence at Google in 2003 when a site reliability team was created. At that time, the team was made up of software engineers.

Engineering 67

Engineering DevOps Monitoring Google 67

Scalegrid

MAY 13, 2020

In this blog post, we explain what Greenplum is, and break down the Greenplum architecture, advantages, major use cases, and how to get started. High performance, query optimization, open source and polymorphic data storage are the major Greenplum advantages. Polymorphic Data Storage. What Exactly is Greenplum? Major Use Cases.

Big Data 321

Big Data Database Artificial Intelligence Open Source 321

Percona

DECEMBER 20, 2018

Let’s look at some of the most popular Percona Database Performance Blog posts in 2018. With the Percona Database Performance Blog, Percona staff and leadership work hard to provide the open source community with insights, technical support, predictions and metrics around multiple open source database software technologies.

Database 40

Database Tuning Performance AWS 40

The Netflix TechBlog

OCTOBER 19, 2020

a Netflix member via Twitter This is an example of a question our on-call engineers need to answer to help resolve a member issue?—?which In our previous blog post we introduced Edgar, our troubleshooting tool for streaming sessions. We needed to increase engineering productivity via distributed request tracing.

Infrastructure 233

Infrastructure Transportation Storage Open Source 233

Dynatrace

OCTOBER 4, 2022

Teams have introduced workarounds to reduce storage costs. Additionally, efforts such as lowered data retention times, two-tiered storage systems, shaky index management, sampled data, and data pipelines reduce the overall amount of stored data. Stop worrying about log data ingest and storage — start creating value instead.

Analytics 233

Analytics Artificial Intelligence Storage Serverless 233

Percona

MARCH 7, 2023

One approach is to separate compute and storage to allow for independent scaling. By separating storage and compute, Neon replaces the PostgreSQL storage layer with data nodes, and compute nodes are distributed across a cluster of nodes. Architecture A Neon installation consists of compute nodes and the Neon storage engine.

Serverless 114

Serverless Storage Open Source Architecture 114

Dynatrace

JULY 29, 2022

As development and site reliability engineering (SRE) teams strive to release software faster, log analytics can provide key insight into software quality as part of a broader DevOps observability and automation initiative. Cold storage and rehydration. Cold storage and rehydration. Better-quality code. Inadequate context.

Analytics 180

Analytics Storage Retail Monitoring 180

Percona

MARCH 29, 2023

Compression in any database is necessary as it has many advantages, like storage reduction, data transmission time, etc. Storage reduction alone results in significant cost savings, and we can save more data in the same space. In this blog, we will discuss both data and network-level compression offered in MongoDB.

Storage 112

Storage Network Open Source Latency 112

Percona

OCTOBER 13, 2023

MySQL writes a.sdi file to accommodate the serialized dictionary information for storage engines that lack support for this functionality. Storage engine compatibility Different MySQL storage engines handle.sdi files in various ways. The exact handling can vary depending on the storage engine.

Storage 97

Storage Database Engineering Open Source 97

Percona

APRIL 20, 2023

In this blog post, you will learn: how to leverage FDE on Kubernetes with Percona Operator for PostgreSQL how to start using encrypted storage for already running cluster Prepare In most public clouds, block storage is not encrypted by default. The configuration of the storage class depends on your storage plugin.

Storage 123

Storage AWS Cloud Database 123

Dynatrace

FEBRUARY 9, 2022

Cloud vendors such as Amazon Web Services (AWS), Microsoft, and Google provide a wide spectrum of serverless services for compute and event-driven workloads, databases, storage, messaging, and other purposes. Engineers often choose best-of-breed services from multiple sources to create a single application. recent blogs.

Serverless 186

Serverless Azure Lambda AWS 186

Dynatrace

OCTOBER 7, 2021

This means you no longer have to provision, scale, and maintain servers to run your applications, databases, and storage systems. Speed is next; serverless solutions are quick to spin up or down as needed, and there are no delays due to limited storage or resource access. AWS offers four serverless offerings for storage.

Serverless 279

Serverless AWS Lambda Storage 279

Percona

DECEMBER 11, 2023

A Dedicated Log Volume (DLV) is a specialized storage volume designed to house database transaction logs separately from the volume containing the database tables. DLVs are particularly advantageous for databases with large allocated storage, high I/O per second (IOPS) requirements, or latency-sensitive workloads.

AWS 100

AWS Benchmarking Performance Traffic 100

All Things Distributed

AUGUST 20, 2012

Managing Cold Storage with Amazon Glacier. With the introduction of Amazon Glacier , IT organizations now have a solution that removes the headaches of digital archiving and provides extremely low cost storage. All Things Distributed. Werner Vogels weblog on building scalable and robust distributed systems. Expanding the Cloud â??

Storage 122

Storage Cloud AWS Media 122

Percona

JANUARY 30, 2023

As Ibrar Ahmed noted in his blog post on Transparent Database Encryption (TDE). Instead, it seems PostgreSQL Developers are of the opinion that encryption is a storage-level problem and is better solved on the filesystem or block device level. It does not require encryption capabilities on the storage level to provide data security.

Database 128

Database Storage Government Open Source 128

Dynatrace

FEBRUARY 10, 2023

From data lakehouse to an analytics platform Traditionally, to gain true business insight, organizations had to make tradeoffs between accessing quality, real-time data and factors such as data storage costs. IT pros need a data and analytics platform that doesn’t require sacrifices among speed, scale, and cost. What is a data lakehouse?

Efficiency 274

Efficiency Performance Analytics DevOps 274

Dynatrace

MAY 26, 2021

From May 17 to May 18, 2021, the Open-Source Engineering team at Dynatrace attended the virtual observability conference, o11yfest. This blog dives into what we learned by listening to experts in the field and the key takeaways we gathered from the talks. A recurring theme was Site Reliability Engineering. Dynatrace news.

AWS 210

AWS Strategy Storage Metrics 210

Percona

FEBRUARY 24, 2023

Use reduction at scale requires a cultural shift, and engineers must consider the cost as they think of memory or bandwidth as another deployment KPI. From AWS documentation , Amazon EBS is an easy-to-use, scalable, high-performance block-storage service designed for Amazon EC2.

Cloud 122

Cloud Serverless AWS Storage 122

Dynatrace

OCTOBER 1, 2020

With all the data collected and powered by our Davis AI-driven causation engine, Dynatrace automatically identifies slowdowns in your applications and services and points you to their root cause. Network device visibility (hosts, switches, routers, storage devices). Average query response time.

Traffic 264

Traffic Network Infrastructure Artificial Intelligence 264

Dynatrace

JULY 25, 2022

This new service enhances the user visibility of network details with direct delivery of Flow Logs for Transit Gateway to your desired endpoint via Amazon Simple Storage Service (S3) bucket or Amazon CloudWatch Logs. The post Dynatrace adds support for AWS Transit Gateway with VPC Flow Logs appeared first on Dynatrace blog.

AWS 216

AWS Transportation Network Traffic 216

Percona

MARCH 23, 2023

This blog is in reference to our previous ones for ‘Innodb Performance Optimizations Basics’ 2007 and 2013. Although there have been many blogs about adjusting MySQL variables for better performance since then, I think this topic deserves a blog update since the last update was a decade ago, and MySQL 5.7

Performance 129

Performance Hardware Tuning Storage 129

Scalegrid

JULY 17, 2020

JSONPath brings a powerful JSON query engine to PostgreSQL. For the purposes of this blog, when we refer to JSON support in PostgreSQL, we will refer to JSONB going forward. JSONB storage has some drawbacks vs. traditional columns: PostreSQL does not store column statistics for JSONB columns. When Should You Use JSON vs. JSONB?

Storage 321

Storage Database Efficiency Processing 321

Dynatrace

MAY 26, 2021

From May 17 to May 18, 2021, the Open-Source Engineering team at Dynatrace attended the virtual observability conference, o11yfest. This blog dives into what we learned by listening to experts in the field and the key takeaways we gathered from the talks. A recurring theme was Site Reliability Engineering. Dynatrace news.

AWS 147

AWS Strategy Storage Metrics 147

Dynatrace

SEPTEMBER 7, 2022

With Amazon Web Services, the main sources from which to ingest logs—Simple Storage Service, or S3, and CloudWatch —come with an additional cost. Dynatrace uses your data and its sophisticated AI causation engine, Davis® , to automatically detect performance anomalies in your applications, services, and infrastructure.

Traffic 214

Traffic AWS Network Cloud 214

Percona

FEBRUARY 10, 2023

To take consistent and hot backup, it uses various locking methods, especially for non-transactional storage engine tables. This blog post discusses the cause and possible solution for queries with ​ Waiting for table flush state in processlist when taking backups using Percona XtraBackup. Only MySQL 5.7

Storage 104

Storage Engineering Open Source Servers 104

Dynatrace

JULY 29, 2021

Operations teams can leverage the same approach to improve analytics and insights into data storage, network devices, or even the room temperatures of specific server rooms. All Dynatrace enhancements mentioned in this blog post will be available within the next 90 days. Next steps. For more information visit our web page.

Analytics 246

Analytics Open Source Serverless Architecture 246

Scalegrid

JANUARY 20, 2020

In this blog post, we will cover the top three scenarios to leverage follower clusters for your application. Note: ScaleGrid implements follower clusters using storage snapshots. Engineers are woken from their beds to start a full blown fire fighting operation. How Do Follower Clusters Differ From Replication?

Analytics 185

Analytics Storage Database Systems 185

The Netflix TechBlog

AUGUST 28, 2020

In this blog post, we present benefits of applying the above-mentioned optimizations to standard dynamic range (SDR) 10-bit and 4K streams (some titles are also HFR). For example, two 1080p points with identical VMAF score or four 4K points with identical VMAF score, resulting in wasted bits and increased storage footprint.

Network 330

Network Storage Innovation Mobile 330