Dynatrace

DECEMBER 18, 2023

This challenge has given rise to the discipline of observability engineering, which concentrates on the details of telemetry data to fine-tune observability use cases. But often, we use additional services and solutions within our environment for backups, storage, networking, and more. What is Prometheus?

Metrics 201

Metrics Engineering Energy Tuning 201

The Netflix TechBlog

OCTOBER 19, 2020

Our distributed tracing infrastructure is grouped into three sections: tracer library instrumentation, stream processing, and storage. We earned the trust of our engineers by developing empathy for their operational burden and by focusing on providing efficient tracer library integrations in runtime environments.

Infrastructure 238

Infrastructure Transportation Storage Open Source 238

Percona

DECEMBER 27, 2023

of respondents are currently utilizing databases in Kubernetes (k8s). These indicators suggest that the adoption of databases on k8s is in its early stages and is likely to continue growing in the future. Customers will increasingly prioritize AI efficiency and education to tackle legal and ethical concerns.

Cloud 83

Cloud Open Source Strategy Best Practices 83

Dynatrace

AUGUST 28, 2020

Azure Data Lake Storage Gen1. Manual tasks like shutting down virtual machines in bulk or creating database backups can be error prone. Azure Data Factory is a hybrid data integration service that enables you to quickly and efficiently create automated data pipelines—without writing any code. Azure Logic Apps. Azure Event Grid.

Azure 170

Azure Cloud Tuning Monitoring 170

Percona

AUGUST 17, 2023

PostgreSQL is open source relational database management software. Most recently, in StackOverflow’s 2022 Stack Developer Survey , PostgreSQL took a slight lead over MySQL (46.48% to 45.68%) as the most popular database platform among professional developers. What is PostgreSQL used for? Is PostgreSQL enterprise-ready?

Open Source 83

Open Source Healthcare Social Media Database 83

Percona

JANUARY 19, 2023

Some startups adopted MySQL in its early days such as Facebook, Uber, Pinterest, and many more, which are now big and successful companies that prove that MySQL can run on large databases and on heavily used sites. In this blog post, we will review key topics to consider for managing large datasets more efficiently in MySQL.

Open Source 93

Open Source Storage Database Big Data 93

The Netflix TechBlog

DECEMBER 21, 2020

At this scale, we can gain a significant amount of performance and cost benefits by optimizing the storage layout (records, objects, partitions) as the data lands into our warehouse. We built AutoOptimize to efficiently and transparently optimize the data and metadata storage layout while maximizing their cost and performance benefits.

Storage 212

Storage Latency Efficiency Data Engineering 212

SQL Performance

MARCH 17, 2020

Back in 2014, I wrote an article called Performance Tuning the Whole Query Plan. I will be using the 50GB Stack Overflow 2013 database , but any large table with a low number of distinct values would do. The Stack Overflow 2013 database comes without nonclustered indexes to minimize download time. Test Environment. BountyAmount.

Servers 86

Servers Tuning Database Efficiency 86

Adrian Cockcroft

NOVEMBER 26, 2023

SUS101: Sustainability innovation in AWS Global Infrastructure AWS is determined to make the cloud the cleanest and most energy-efficient way to run customers’ infrastructure and business. This includes providing the efficient, resilient services AWS customers expect, while minimizing their environmental footprint. AWS speaker: T.

AWS 52

AWS Energy IoT Best Practices 52

The Netflix TechBlog

SEPTEMBER 18, 2024

Central to this infrastructure is our use of multiple online distributed databases such as Apache Cassandra , a NoSQL database known for its high availability and scalability. Over time as new key-value databases were introduced and service owners launched new use cases, we encountered numerous challenges with datastore misuse.

Latency 252

Latency Storage Cache Efficiency 252

The Netflix TechBlog

OCTOBER 8, 2024

Building on these foundational abstractions, we developed the TimeSeries Abstraction  — a versatile and scalable solution designed to efficiently store and query large volumes of temporal event data with low millisecond latencies, all in a cost-effective manner across various use cases. Let’s dive into the various aspects of this abstraction.

Latency 240

Latency Storage Traffic Tuning 240

Percona

OCTOBER 18, 2023

Migrating a proprietary database to open source is a major decision that can significantly affect your organization. Today, we’ll be taking a deep dive into the intricacies of database migration, along with specific solutions to help make the process easier.

Open Source 97

Open Source Database Strategy Hardware 97

Percona

JANUARY 4, 2024

Managing storage and performance efficiently in your MySQL database is crucial, and general tablespaces offer flexibility in achieving this. In contrast to the single system tablespace that holds system tables by default, general tablespaces are user-defined storage containers for multiple InnoDB tables.

Storage 92

Storage Engineering Database Open Source 92

Dynatrace

AUGUST 1, 2022

Interestingly, our partner RedHat reported in 2021 that around 80% of deployed workloads are databases or data caches, storing data in persistent volume claims (PVCs). You also decide to run your database for storing user uploads – such as images or videos – directly in Kubernetes. However, you lack insights into your PVCs.

Storage 211

Storage Database Network Cache 211

Dynatrace

APRIL 20, 2023

With more automated approaches to log monitoring and log analysis, however, organizations can gain visibility into their applications and infrastructure efficiently and with greater precision—even as cloud environments grow. ” A data warehouse, on the other hand, is an efficient and fast option for querying data.

Analytics 211

Analytics Infrastructure Storage Architecture 211

Percona

APRIL 24, 2023

Indexes are generally considered to be the panacea when it comes to SQL performance tuning, and PostgreSQL supports different types of indexes catering to different use cases. I keep seeing many articles and talks on “tuning” discussing how creating new indexes speeds up SQL but rarely ones discussing removing them.

Tuning 130

Tuning Cache Storage Database 130

Dynatrace

FEBRUARY 16, 2023

Grail combines the big-data storage of a data warehouse with the analytical flexibility of a data lake. This unified approach enables Grail to vault past the limitations of traditional databases. And without the encumbrances of traditional databases, Grail performs fast. “In

Analytics 210

Analytics Innovation Metrics Database 210

The Netflix TechBlog

OCTOBER 27, 2020

Therefore, we must efficiently move data from the data warehouse to a global, low-latency and highly-reliable key-value store. What is Bulldozer Bulldozer is a self-serve data platform that moves data efficiently from data warehouse tables to key-value stores in batches. Figure 1 shows how we use Bulldozer to move data at Netflix.

Latency 252

Latency Storage Big Data Tuning 252

The Netflix TechBlog

JUNE 13, 2023

They enable us to further fine-tune and configure the system, ensuring the new changes are integrated smoothly and seamlessly. Our system doesn’t require strict consistency guarantees and does not use database transactions. A/B Testing A/B testing is a widely recognized method for verifying hypotheses through a controlled experiment.

Traffic 285

Traffic Metrics Systems Strategy 285

Percona

JULY 2, 2023

The size of the database and your database environment—if it is on colo or cloud—matters. Having a backup strategy in place that takes regular backups and has secure storage is essential to protect the database in an enterprise-grade environment to ensure its availability in the event of failures or disasters.

Strategy 52

Strategy Database Storage Open Source 52

Percona

APRIL 17, 2023

The main objective of this post is to share my experience over the past years tuning MongoDB and centralize the diverse sources that I crossed in this journey in a unique place. The Linux default is usually 60 , which is not ideal for database usage. Spoiler alert: This post focuses on MongoDB 3.6.X

Best Practices 52

Best Practices Design Tuning Database 52

Percona

AUGUST 27, 2018

Database administrators with appropriate privileges can associate virtual CPUs with resource groups and assign threads to these groups. Dangerous , because if you set this to a thread when using connection pooling OR ProxySQL and multiplexing, you may end up assigning a limitation to queries that instead, you wanted to run efficiently.

Tuning 46

Tuning Virtualization Testing Servers 46

Percona

AUGUST 5, 2023

Partitioning is a way in which a database (MySQL in this case) splits its actual data down into separate tables but still gets treated as a single table by the SQL layer. Otherwise, the storage engine does a scatter-gather and queries ALL partitions in a UNION that is not concurrent. So, What is MySQL Partitioning?

Strategy 40

Strategy Storage Database Best Practices 40

The Netflix TechBlog

JULY 22, 2024

It supports a wide range of workflow use cases, including ETL pipelines, ML workflows, AB test pipelines, pipelines to move data between different storages, etc. The output data is transmitted to Maestro via its REST API, ensuring that the step runtime does not have direct access to the Maestro database. We are eager to hear from you.

Strategy 258

Strategy Scalability Code Engineering 258

The Netflix TechBlog

NOVEMBER 2, 2021

For example, VQS relies on the Netflix Media Database (NMDB) to store and index the quality scores, while the Reloaded system uses a mix of non-queryable data models and files. Video quality has matured in Cosmos and we are invested in making VQS more flexible and efficient. Stay tuned for more details on these algorithmic innovations.

Media 179

Media Innovation Metrics Latency 179

All Things Distributed

MARCH 7, 2013

Many of our customers have, with the click of a button, created DynamoDB deployments in a matter of minutes that are able to serve trillions of database requests per year. Indexed Storage costs : We are lowering the price of indexed storage by 75%. This was the genesis of NoSQL databases like Dynamo at Amazon.

Ecommerce 135

Ecommerce Storage Scalability Database 135

All Things Distributed

NOVEMBER 28, 2012

Unlike traditional row-based relational databases, which store data for each row sequentially on disk, Amazon Redshift stores each column sequentially. This means that Redshift performs much less wasted IO than a row-based database because it doesnâ??t t read data from columns it doesnâ??t t need when executing a given query.

Cloud 162

Cloud Storage Architecture Scalability 162

Scalegrid

JANUARY 8, 2019

Simply put, in a MySQL semisynchronous replication configuration, the master commits transactions to the storage engine only after receiving acknowledgement from at least one of the slaves. Another configuration we can use to increase the efficiency of parallel execution on the slaves is to tune binlog_group_commit_sync_delay on the master.

Availability 113

Availability Tuning Speed Network 113

Particular Software

APRIL 9, 2018

Every software system is dependent on a few types of infrastructure (CPU, memory, storage, network), so it's no surprise that there are lots of off-the-shelf tools that provide this capability. To understand if our theme park is running efficiently or not, we'll need monitoring tools that understand how theme parks really work.

Systems 40

Systems Infrastructure Monitoring Tuning 40

SQL Server According to Bob

JANUARY 11, 2020

KB sectors run on smaller sectors   14 System and sample databases   15 Determining the formatted sector size of database   15 What sector sizes does SQL Server support? SQL Server 2005 introduces the snapshot database feature for point-in-time databases ​​ and online DBCC operations.

Servers 40

Servers Cache Database Media 40

SQL Server According to Bob

JANUARY 11, 2020

Microsoft SQL Server I/O Basics Author: ​​ Bob Dorr, Microsoft SQL Server Escalation Published: ​​ December, 2004 SUMMARY: ​​ Learn the I/O requirements for Microsoft SQL Server database file operations. Stable Media Stable media is often confused with physical storage.

Servers 40

Servers Cache Media Hardware 40

Tech News Gather

JUNE 16, 2023

By leveraging the power of mobile technology, UberEats provides a seamless and efficient way for users to order their favorite meals from a wide range of eateries. UberEats also excels in providing a reliable and efficient delivery service. Stay tuned for valuable insights on designing a captivating UberEats clone.

Social Media 52

Social Media Tuning Strategy Media 52

High Scalability

NOVEMBER 25, 2019

It is limited by the disk space; it can’t expand storage elastically; it chokes if you run few I/O intensive processes or try collaborating with 100 other users. Over time, costs for S3 and GCS became reasonable and with Egnyte’s storage plugin architecture, our customers can now bring in any storage backend of their choice.

Architecture 52

Architecture Cache Azure Storage 52

Percona

JANUARY 14, 2019

What if we use ClickHouse (which is a columnar analytical database) as our main datastore? Well, typically, an analytical database is not a replacement for a transactional or key/value datastore. However, ClickHouse is super efficient for timeseries and provides “sharding” out of the box (scalability beyond one node).

Database 40

Database Analytics Blockchain Healthcare 40