DZone

NOVEMBER 7, 2023

In the rapidly evolving digital landscape, the role of data has shifted from being merely a byproduct of business to becoming its lifeblood. With businesses constantly in the race to stay ahead, the process of integrating this data becomes crucial. However, it's no longer enough to assimilate data in isolated, batch-oriented processes.

Systems 281

Systems Analytics Architecture Engineering 281

DZone

MAY 26, 2023

Therefore, I found it important to write a piece based on my understanding of time series data versus relational data as someone with a unique understanding of both. I have coded many applications, both client and web, over my career, and I understand the importance of building a well-developed application from the ground up.

Database 292

Database Code Development 292

DZone

DECEMBER 16, 2023

In my previous article ‘ Data Validation to Improve Data Quality ’, I shared the importance of data quality and a checklist of validation rules to achieve it. Those validation rules alone may not guarantee the best data quality. Most pipelines are automated and run on a fixed schedule.

Best Practices 277

Best Practices 277

The Netflix TechBlog

NOVEMBER 14, 2023

By Abhinaya Shetty , Bharath Mummadisetty At Netflix, our Membership and Finance Data Engineering team harnesses diverse data related to plans, pricing, membership life cycle, and revenue to fuel analytics, power various dashboards, and make data-informed decisions. We expect complete and accurate data at the end of each run.

Data Engineering 264

Data Engineering Engineering Processing Games 264

Dynatrace

OCTOBER 4, 2022

For IT infrastructure managers and site reliability engineers, or SREs , logs provide a treasure trove of data. But on their own, logs present just another data silo as IT professionals attempt to troubleshoot and remediate problems. Data volume explosion in multicloud environments poses log issues.

Analytics 229

Analytics Storage Infrastructure Metrics 229

DZone

OCTOBER 16, 2023

When creating applications that store and analyze large amounts of data, such as time series, log data, or event-storing ones, developing a good and future-proof data model can be a difficult task. Choosing the right data types in PostgreSQL can significantly impact your database's performance and efficiency.

Best Practices 272

Best Practices Speed Database Efficiency 272

DZone

FEBRUARY 5, 2024

In today's cloud computing world, all types of logging data are extremely valuable. Logs can include a wide variety of data, including system events, transaction data, user activities, web browser logs, errors, and performance metrics. This innovative service is transforming the way organizations handle their log data.

Analytics 269

Analytics Innovation Strategy Efficiency 269

Dynatrace

JANUARY 31, 2024

Organizations choose data-driven approaches to maximize the value of their data, achieve better business outcomes, and realize cost savings by improving their products, services, and processes. However, there are many obstacles and limitations along the way to becoming a data-driven organization. Understanding the context.

Analytics 200

Analytics Processing Transportation Storage 200

Dynatrace

JANUARY 4, 2024

While this approach can be effective if the model is trained with a large amount of data, even in the best-case scenarios, it amounts to an informed guess, rather than a certainty. But to be successful, data quality is critical. Teams need to ensure the data is accurate and correctly represents real-world scenarios. Consistency.

Government 208

Government Analytics Benchmarking Storage 208

DZone

MARCH 18, 2024

The data silos problem is like arthritis for online businesses because almost everyone gets it as they grow old. For one reason or another, it is tricky to integrate the data from all these sources. Data stays where it is and cannot be interrelated for further analysis. That's how data silos come to form.

Mobile 136

Mobile Website 136

DZone

NOVEMBER 7, 2023

In the labyrinth of data-driven architectures, the challenge of data integration—fusing data from disparate sources into a coherent, usable form — stands as one of the cornerstones. As businesses amass data at an unprecedented pace, the question of how to integrate this data effectively comes to the fore.

Processing 177

Processing Architecture Technology Technology 177

Dynatrace

JANUARY 31, 2024

I have ingested important custom data into Dynatrace, critical to running my applications and making accurate business decisions… but can I trust the accuracy and reliability?” ” Welcome to the world of data observability. At its core, data observability is about ensuring the availability, reliability, and quality of data.

DevOps 200

DevOps Analytics Airlines Metrics 200

DZone

DECEMBER 29, 2023

In the ever-evolving landscape of technology, the tandem growth of Artificial Intelligence (AI) and Data Science has emerged as a beacon of hope, promising unparalleled advancements that will significantly impact and enhance various aspects of our lives.

Artificial Intelligence 292

Artificial Intelligence Healthcare Technology Technology 292

The Netflix TechBlog

NOVEMBER 14, 2023

By Abhinaya Shetty , Bharath Mummadisetty In the inaugural blog post of this series, we introduced you to the state of our pipelines before Psyberg and the challenges with incremental processing that led us to create the Psyberg framework within Netflix’s Membership and Finance data engineering team.

Processing 226

Processing Data Engineering Efficiency Analytics 226

Dynatrace

APRIL 19, 2023

Creating an ecosystem that facilitates data security and data privacy by design can be difficult, but it’s critical to securing information. When organizations focus on data privacy by design, they build security considerations into cloud systems upfront rather than as a bolt-on consideration.

Design 200

Design Storage Programming Analytics 200

DZone

MAY 12, 2023

I believe that chief among them is d ata residency or data location: Data localization or data residency law requires data about a nation's citizens or residents to be collected, processed, and/or stored inside the country, often before being transferred internationally.

Cloud 242

Cloud Systems Processing 242

DZone

APRIL 11, 2023

Data processing in the cloud has become increasingly popular due to its scalability, flexibility, and cost-effectiveness. This article will explore how these technologies can be used together to create an optimized data pipeline for data processing in the cloud.

Azure 246

Azure Analytics Storage Cloud 246

Dynatrace

SEPTEMBER 14, 2022

As an industry leader, Dynatrace promotes primarily using software and AI to deal with this complexity at scale instead of just putting data on dashboards. Does that mean that reactive and exploratory data analysis, often done manually and with the help of dashboards, are dead? Why today’s data analytics solutions still fail us.

Metrics 237

Metrics Analytics Monitoring Efficiency 237

DZone

FEBRUARY 20, 2023

The data community is striving to incorporate the core concepts of engineering rigor found in software communities but still has further to go. This talk covers ways to leverage software engineering practices for data engineering and demonstrates how measuring key performance metrics could help build more robust and reliable data pipelines.

Software Engineering 264

Software Engineering Engineering Software Software 264

DZone

AUGUST 23, 2023

Monitoring Time-Series IoT Device Data Time-series data is crucial for IoT device monitoring and data visualization in industries such as agriculture, renewable energy, and meteorology. In this tutorial, we will guide you through the process of setting up a monitoring system for IoT device data.

IoT 241

IoT Energy Open Source Analytics 241

DZone

MARCH 21, 2024

This post continues my exploration of concepts and techniques related to both the way so-called “Jewish times” (zmanim) are calculated; as well as the techniques needed to use the PHP Zmanim library – a library of functions that let you easily calculate Jewish times.

Java 274

Java 274

The Netflix TechBlog

MARCH 10, 2023

This platform has evolved from supporting studio applications to data science applications, machine-learning applications to discover the assets metadata, and build various data facts. Hence we built the data pipeline that can be used to extract the existing assets metadata and process it specifically to each new use case.

Media 237

Media Traffic Processing Design 237

DZone

MAY 7, 2023

The following tutorial walks you through how to use Spring Boot apps with ScyllaDB for time series data, taking advantage of shard-aware drivers and prepared statements. ScyllaDB is used to store the stock price (time series data). And you’ll learn how ScyllaDB can be used to store time series data.

Storage 273

Storage Servers Testing 273

DZone

JUNE 6, 2023

Think of your disks as a warehouse: The compaction mechanism is like a team of storekeepers (with genius organizing skills like Marie Kondo) who help put away the incoming data. In particular, the data (which is the inflowing cargo in this metaphor) comes in on a "conveyor belt," which does not allow cutting in line.

Storage 214

Storage Database 214

Dynatrace

MARCH 15, 2024

As organizations strive for observability and data democratization, OpenTelemetry emerges as a key technology to create and transfer observability data. Understanding OpenTelemetry OpenTelemetry is an open, vendor-neutral standard for creating, collecting, and transferring telemetry data, like traces, metrics, and logs.

Open Source 130

Open Source Best Practices Infrastructure Tuning 130

DZone

DECEMBER 2, 2023

When we introduced the concept of data observability four years ago, it resonated with organizations that had unlocked new value…and new problems thanks to the modern data stack. Now, four years later, we are seeing organizations grapple with the tremendous potential…and tremendous challenges posed by generative AI.

177

177

Dynatrace

OCTOBER 4, 2022

While data lakes and data warehousing architectures are commonly used modes for storing and analyzing data, a data lakehouse is an efficient third way to store and analyze data that unifies the two architectures while preserving the benefits of both. What is a data lakehouse? How does a data lakehouse work?

Artificial Intelligence 217

Artificial Intelligence Analytics Storage Government 217

DZone

AUGUST 18, 2023

We all know that data is being generated at an unprecedented rate. You may also know that this has led to an increase in the demand for efficient and secure data storage solutions that won’t break the bank. This article will explore what edge data platforms and real-time services are, why they are important, and how they can be used.

IoT 130

IoT Media Latency Storage 130

DZone

MAY 1, 2023

This article explains why Deep data observability is different from shallow: deep data observability is truly comprehensive in terms of data sources, data formats, data granularity, validator configuration, cadence, and user focus. Nevertheless, the industry is nowhere near fully formed.

Innovation 192

Innovation 192

Dynatrace

OCTOBER 24, 2023

The session focused on how agencies are protecting data within the context of zero trust. Reflecting on their discussion, I identified five emerging “truths” about zero trust and data. Identity is not necessarily as important as data because at the end of the day, we steal identities to get to the data,” said Williamson. “I

Government 227

Government Artificial Intelligence Best Practices Infrastructure 227

DZone

JUNE 29, 2020

Today data is an important factor for business success. In every business, it has been observed that data is playing a game-changing moment to improve business performance. Data is important and necessary in this increasingly competitive world. Data comes in large volumes everywhere and in complex structures.

Games 363

Games Performance Big Data Analytics 363

Dynatrace

JANUARY 23, 2024

AI data analysis can help development teams release software faster and at higher quality. So how can organizations ensure data quality, reliability, and freshness for AI-driven answers and insights? And how can they take advantage of AI without incurring skyrocketing costs to store, manage, and query data?

Performance 194

Performance DevOps Innovation Artificial Intelligence 194

DZone

APRIL 4, 2023

Kafka’s ability to handle large volumes of real-time market data makes it a core infrastructure component for trading, risk management, and fraud detection. Financial institutions use Kafka to stream data from market data feeds, transaction data, and other external sources to drive decisions.

Processing 262

Processing Database Infrastructure Testing 262

Scalegrid

JANUARY 5, 2024

Enhanced data security, better data integrity, and efficient access to information. This article cuts through the complexity to showcase the tangible benefits of DBMS, equipping you with the knowledge to make informed decisions about your data management strategies. What are the key advantages of DBMS?

Efficiency 147

Efficiency Storage Database Scalability 147

The Netflix TechBlog

MARCH 4, 2024

Operational automation–including but not limited to, auto diagnosis, auto remediation, auto configuration, auto tuning, auto scaling, auto debugging, and auto testing–is key to the success of modern data platforms. We have also noted a great potential for further improvement by model tuning (see the section of Rollout in Production).

Tuning 210

Tuning Efficiency Big Data Engineering 210

Scalegrid

JANUARY 19, 2024

When a MongoDB rollback happens, it can cause trouble to your data integrity and system consistency. Understanding how to address a rollback is critical for minimizing potential data loss and maintaining seamless operations. The consequences of rollbacks are significant and should not be underestimated.

Database 130

Database Network Servers Monitoring 130

Dynatrace

NOVEMBER 6, 2023

Recently, the Parliament of India released the Digital Personal Data Protection Act 2023 , which regulates the processing of digital personal data in India and recognizes the right of individuals to protect their data in India. Dynatrace is constantly evaluating the further expansion of its regional presence of Dynatrace SaaS.

AWS 245

AWS Azure Tuning Analytics 245