DZone

JULY 17, 2023

Apache Spark is a leading platform in the field of big data processing, known for its speed, versatility, and ease of use. However, getting the most out of Spark often involves fine-tuning and optimization. Understanding Apache Spark Apache Spark is a unified computing engine designed for large-scale data processing.

Big Data 130

Big Data Performance Open Source Tuning 130

Percona

MAY 1, 2023

Out of the box, the default PostgreSQL configuration is not tuned for any particular workload. It is primarily the responsibility of the database administrator or developer to tune PostgreSQL according to their system’s workload. What is PostgreSQL performance tuning? Why is PostgreSQL performance tuning important?

Tuning 52

Tuning Efficiency Database Performance 52

Dynatrace

MARCH 29, 2024

As organizations continue to modernize their technology stacks, many turn to Kubernetes , an open source container orchestration system for automating software deployment, scaling, and management. Providing at-a-glance data makes it possible for teams to quickly identify high-level issues and then drill down into the details.

Engineering 218

Engineering DevOps Operating System Open Source 218

Dynatrace

OCTOBER 7, 2020

At the same time, you can now simply flip a switch to begin ingesting observability data from these tools into the Dynatrace platform, all while retaining your investment and increasing the value you get from Dynatrace. Open-source metric sources automatically map to our Smartscape model for AI analytics.

Open Source 250

Open Source Metrics Analytics Tuning 250

Dynatrace

DECEMBER 18, 2023

This challenge has given rise to the discipline of observability engineering, which concentrates on the details of telemetry data to fine-tune observability use cases. To get a more granular look into telemetry data, many analysts rely on custom metrics using Prometheus. What is Prometheus?

Metrics 171

Metrics Engineering Energy Tuning 171

Dynatrace

AUGUST 16, 2021

Microsoft’s GitHub is the largest open-source software community in the world with millions of open-source projects. Kubernetes deployments can be managed using a combination of both the open-source Azure Kubernetes set context Action and Kubernetes deployment GitHub Action. GitHub and GitHub Actions.

Software 246

Software Software Open Source Azure 246

Dynatrace

OCTOBER 6, 2022

OpenTelemetry is an exciting open source observability standard that provides a common collection of tools, APIs, and SDKs to help collect observability signals. You can ingest OpenTelemetry data in two ways: using Dynatrace OneAgent technology or using OpenTelemetry Protocol (OTLP), which relies exclusively on open source technology.

Open Source 167

Open Source Metrics Tuning Technology 167

Percona

DECEMBER 27, 2023

In our conversations with developers and engineers during the conference, we confirmed the CNCF GItOps Microsurvey data – 91% are already using GitOps. We want adopters of our technology to be able to deploy their critical open source databases and applications across any public or private cloud environment.

Cloud 79

Cloud Open Source Strategy Best Practices 79

Percona

NOVEMBER 3, 2023

Sure, database migration is complex, particularly when you’re looking to migrate from a proprietary database to an open source one. From upholding data integrity to ensuring no data is lost during the transition, there’s a lot to consider. Data preparation: How are you going to backup and cleanse your data?

Database 83

Database Open Source Benchmarking Strategy 83

Dynatrace

FEBRUARY 8, 2021

Also, these modern, cloud-native architectures produce an immense volume, velocity, and variety of data. Every service and component exposes observability data (metrics, logs, and traces) that contains crucial information to drive digital businesses. Collecting data requires massive and ongoing configuration efforts.

Cloud 256

Cloud Azure Analytics AWS 256

Dynatrace

NOVEMBER 17, 2020

We at Dynatrace understand the importance of contributing our expertise in enterprise-grade intelligent observability to the open source community. Armin: Dynatrace contributes distributed tracing knowledge, which was gained over years of experience via semantic conventions to ensure the highest quality of OpenTelemetry data.

Open Source 151

Open Source Software Engineering Government Java 151

The Netflix TechBlog

NOVEMBER 9, 2021

The Android launch leveraged the open-source software decoder dav1d built by the VideoLAN, VLC, and FFmpeg communities and sponsored by AOMedia. Netflix has also partnered with YouTube to develop an open-source solution for an AV1 decoder on game consoles that utilizes the additional power of GPUs. Some titles (e.g.,

Media 246

Media Open Source Efficiency Software Engineering 246

Percona

OCTOBER 18, 2023

Migrating a proprietary database to open source is a major decision that can significantly affect your organization. Advantages of migrating to open source For many reasons mentioned earlier, organizations are increasingly shifting towards open source databases for their data management needs.

Open Source 92

Open Source Database Hardware Strategy 92

Percona

MARCH 1, 2023

So you get the Umaluts, Cedils, and C-J-K Language support in your data, plus emojis. EXPLAIN ANALYSE is a big boon to query tuning. Data Masking to hide, mask, or obscure data like credit cards or identification numbers. You’ll always own access to and availability of your data.

Servers 118

Servers Open Source Tuning C++ 118

Percona

AUGUST 31, 2018

Out of the box, the default PostgreSQL configuration is not tuned for any particular workload. It is primarily the responsibility of the database administrator or developer to tune PostgreSQL according to their system’s workload. That means data is stored in memory twice, first in PostgreSQL buffer and then kernel buffer.

Tuning 55

Tuning Database Performance Benchmarking 55

Percona

FEBRUARY 15, 2023

With MyRocks/RocksDB, data is stored in a set of large SST files. Tuning In terms of tuning, two parameters can be tuned, the size of the bitmap and the number of bits set by every value. Bloom filters are an essential component of an LSM-based database engine like MyRocks.

Storage 127

Storage Tuning Cache Engineering 127

Dynatrace

FEBRUARY 17, 2022

Grabner notes that Kubernetes already provides some data of its own. And if you have other tools, like the open-source systems monitoring toolkit, Prometheus , you need a solution to make sense of all the data in context. Infrastructure as code vs infrastructure as data.

Technology 202

Technology Technology Cloud Infrastructure 202

Percona

NOVEMBER 9, 2023

With this data, we’ll be able to better understand and tune our Postgres servers. The first step is to include a new query that will gather data on WAL usage because this is not yet collected by default on the latest version of PMM. PMM already collects the configurations, so we will only miss data from the graphs.

Monitoring 90

Monitoring Tuning Metrics Servers 90

Scalegrid

MARCH 22, 2024

This covers everything from tech specs and what you expect from the service to security, how you’ll handle data, and how services will be managed. Hosting either on-premises or with third-party data center providers and deciding whether management services should be conducted internally or outsourced.

Strategy 130

Strategy Cloud Government Innovation 130

Percona

AUGUST 17, 2023

PostgreSQL is open source relational database management software. It’s used for data management (shocker), application development, and data analytics. It has evolved steadily, however, during 25 years as an open source project. And there’s a global community dedicated to keeping it open source.

Open Source 78

Open Source Healthcare Social Media Database 78

The Netflix TechBlog

JULY 20, 2020

and thanks to the magic of open source, it’s only about a hundred lines of Python code. Our diff solution involves embedding each line into a low dimensional vector and (optionally “fine-tuning” or updating the embedding model at the same time), assigning it to a cluster, and identifying lines in different clusters as “different”.

Development 177

Development Open Source Tuning Network 177

Dynatrace

JULY 16, 2021

The innovative technology powering these recent and future use cases comes from Keptn , our recognized CNCF open-source project. If you want more information on Keptn, then I suggest joining the Keptn Open Source community and help us drive innovation that benefits both the open-source project and Dynatrace Cloud Automation Solution.

DevOps 242

DevOps Open Source Cloud Azure 242

The Netflix TechBlog

OCTOBER 19, 2020

Troubleshooting a session in Edgar When we started building Edgar four years ago, there were very few open-source distributed tracing systems that satisfied our needs. Our tactical approach was to use Netflix-specific libraries for collecting traces from Java-based streaming services until open source tracer libraries matured.

Infrastructure 233

Infrastructure Transportation Storage Open Source 233

Percona

MARCH 6, 2023

MySQL uses Redo Logs internally during crash recovery to correct data written by incomplete transactions. Redo Logs Starting from MySQL 8.0.30, the variable that should be tuned for optimizing the Redo Logs is innodb_redo_log_capacity , and we start with good news here: It’s dynamic! So you won’t have downtime if you need to tune this.

Tuning 89

Tuning Open Source Database Engineering 89

Dynatrace

APRIL 22, 2022

Although many companies adopt solutions such as OpenTelemetry, Prometheus, and Grafana as part of their observability strategy, they often confront a common data analysis problem: data silos. When teams, tools, and data are siloed, it’s harder for organizations to succeed. This leads to multiple tool-specific dashboards.

Open Source 167

Open Source Metrics Cloud Tuning 167

O'Reilly

APRIL 4, 2024

They weren’t buying users with subsidized prices; they were building data centers. OpenAI, for example, has trained not just on publicly available data but reportedly on copyrighted content retrieved from pirate sites. As Mike Loukides points out , “Smaller startups…will be priced out, along with every open-source effort.

Artificial Intelligence 61

Artificial Intelligence Open Source Innovation Transportation 61

Percona

JULY 19, 2023

One way to do this is to remove the instance and add it back, but you’ll lose some stats/historical data. Percona Monitoring and Management is a best-of-breed open source database monitoring solution. What if you want to change the value later on? I hope you find the article useful and somewhat entertaining.

Monitoring 81

Monitoring Entertainment Open Source Tuning 81

Percona

OCTOBER 10, 2023

This is not a general rule, but as databases are responsible for a core layer of any IT system – data storage and processing — they require reliability. Percona Monitoring and Management (PMM) – A fully open source monitoring tool to help you run your databases (not limited to MongoDB).

Availability 99

Availability Servers Database Open Source 99

Scalegrid

OCTOBER 24, 2019

AWS is the #1 cloud provider for open-source database hosting, and the go-to cloud for MySQL deployments. Slave-2 uses an EBS data disk, and you can configure the amount of IOPS required for your cluster. This configuration provides complete safety for your data, even in the event you lose the local SSD disks.

AWS 183

AWS Latency Performance Benchmarking 183

Percona

AUGUST 7, 2023

In InnoDB, the undo log and the redo log are two indispensable components that play a vital role in maintaining data integrity and ensuring transactional consistency. Here’s how it works: Maintaining before-images: Whenever a transaction modifies data in InnoDB, the undo log records the before-image of the modified data.

Database 95

Database Tuning Storage Open Source 95

Percona

OCTOBER 12, 2023

Manual flame graphs collection Although the tool is excellent and automatically provides flame graphs, we don’t have much control over tuning the selected profiler. In some cases, it’s possible to run profilers manually on Kubernetes to have more control over the data collection process.

Code 109

Code Tuning C++ Benchmarking 109

HammerDB

APRIL 19, 2023

sysbench is a widely used open-source benchmarking tool that is designed to evaluate the performance of CPU, memory, disk I/O, and database systems. Please refer to this tuning guide to tune the system for HammerDB: Open Source Database Tuning Guide on 3rd Generation Intel® Xeon® Scalable Processors Based Platform.

Performance 54

Performance Benchmarking Cache Storage 54

Adrian Cockcroft

NOVEMBER 26, 2023

Google Cloud and Microsoft Azure released Scope 3 data in 2021. Access open data from the Amazon Sustainability Data Initiative (ASDI) with Amazon S3 and an API to explore air quality by geography. One of the key challenges organizations face is an ability to extract datasets from different sources.

AWS 52

AWS Energy IoT Best Practices 52

Percona

JULY 27, 2023

This will be handy for auditing data. has EXPLAIN FORMAT=JSON that can output data into a user variable. This could be handy for tracking query tuning and performance auditing. Seems like a lot of work to do while servers are failing, but it should save the data in a better state. has 131 bug fixes and 8.1.0

Servers 82

Servers Open Source Tuning Cache 82

Percona

MARCH 27, 2023

For MySQL and Percona Server for MySQL 8, executing sysbench + I_S queries shows there almost no cache misses, as can be seen in the following snapshot: MySQL 8 improved data dictionary access, which avoids having to open all tables.frm files. Setup The setup consists of creating 10K tables with sysbench and adding 20 FKs to 20 tables.

Cache 101

Cache Hardware Servers Benchmarking 101

Percona

JANUARY 19, 2023

With disks being faster nowadays and CPU and memory resources being cheaper, we could easily say MySQL can handle TBs of data with good performance. For instance, in Percona Managed Services , we have many clients with TBs worth of data that are well performant. There are many compression tools and algorithms for data out there.

Open Source 87

Open Source Storage Database Big Data 87

Abhishek Tiwari

JULY 12, 2018

Building data pipelines can offer strategic advantages to the business. Often companies underestimate the necessary effort and cost involved to build and maintain data pipelines. Data pipeline initiatives are generally unfinished projects. In this post, we will discuss why you should avoid building data pipelines in first place.

Latency 63

Latency Analytics Scalability Engineering 63