Dynatrace

JANUARY 23, 2024

AI data analysis can help development teams release software faster and at higher quality. So how can organizations ensure data quality, reliability, and freshness for AI-driven answers and insights? And how can they take advantage of AI without incurring skyrocketing costs to store, manage, and query data?

Performance 176

Performance DevOps Innovation Artificial Intelligence 176

The Netflix TechBlog

JULY 26, 2021

This happens at an unprecedented scale and introduces many interesting challenges; one of the challenges is how to provide visibility of Studio data across multiple phases and systems to facilitate operational excellence and empower decision making.

Big Data 253

Big Data Government Analytics Processing 253

The Netflix TechBlog

JANUARY 13, 2022

By Vikram Srivastava and Marcelo Mayworm Netflix has one of the most complex data platforms in the cloud on which our data scientists and engineers run batch and streaming workloads. As our subscribers grow worldwide and Netflix enters the world of gaming , the number of batch workflows and real-time data pipelines increases rapidly.

Big Data 238

Big Data Infrastructure Metrics Hardware 238

Dynatrace

SEPTEMBER 14, 2022

With digital systems growing exponentially in size and complexity, industry trends like AIOps and DevSecOps are becoming the norm for application performance m onitoring (APM) and observability tools such as Dynatrace. Does that mean that reactive and exploratory data analysis, often done manually and with the help of dashboards, are dead?

Metrics 217

Metrics Analytics Monitoring Efficiency 217

Scalegrid

MAY 13, 2020

Greenplum Database is a massively parallel processing (MPP) SQL database that is built and based on PostgreSQL. It can scale towards a multi-petabyte level data workload without a single issue, and it allows access to a cluster of powerful servers that will work together within a single SQL interface where you can view all of the data.

Big Data 321

Big Data Database Artificial Intelligence Open Source 321

The Netflix TechBlog

OCTOBER 18, 2022

by Jun He , Akash Dwivedi , Natallia Dzenisenka , Snehal Chennuru , Praneeth Yenugutala , Pawan Dixit At Netflix, Data and Machine Learning (ML) pipelines are widely used and have become central for the business, representing diverse use cases that go beyond recommendations, predictions and data transformations.

Java 202

Java Scalability Traffic Architecture 202

Dynatrace

OCTOBER 6, 2023

Grail: Enterprise-ready data lakehouse Grail, the Dynatrace causational data lakehouse, was explicitly designed for observability and security data, with artificial intelligence integrated into its foundation. Tables are a physical data model, essentially the type of observability data that you can store.

Artificial Intelligence 252

Artificial Intelligence Analytics Metrics Storage 252

Dynatrace

SEPTEMBER 8, 2022

Log management is an organization’s rules and policies for managing and enabling the creation, transmission, analysis, storage, and other tasks related to IT systems’ and applications’ log data. In cloud-native environments, there can also be dozens of additional services and functions all generating data from user-driven events.

Systems 178

Systems Cloud Analytics DevOps 178

Dynatrace

OCTOBER 4, 2022

Some time ago, at a restaurant near Boston, three Dynatrace colleagues dined and discussed the growing data challenge for enterprises. At its core, this challenge involves a rapid increase in the amount—and complexity—of data collected within a company. Work with different and independent data types. Thus, Grail was born.

Artificial Intelligence 257

Artificial Intelligence Analytics Storage Architecture 257

Dynatrace

DECEMBER 5, 2023

Streamline privacy requirements with flexible retention periods Data retention is a critical aspect of data handling, and it’s not just about privacy compliance—it’s about having the flexibility to optimize data storage times in Grail for your Dynatrace use cases. Grail buckets function like folders in a file system.

Storage 159

Storage Healthcare Best Practices Speed 159

The Netflix TechBlog

MARCH 4, 2024

Operational automation–including but not limited to, auto diagnosis, auto remediation, auto configuration, auto tuning, auto scaling, auto debugging, and auto testing–is key to the success of modern data platforms. In this way, no human intervention is required in the remediation process. Multi-objective optimizations.

Tuning 210

Tuning Efficiency Big Data Engineering 210

The Netflix TechBlog

DECEMBER 21, 2020

By Anupom Syam Background At Netflix, our current data warehouse contains hundreds of Petabytes of data stored in AWS S3 , and each day we ingest and create additional Petabytes. On the other hand, these optimizations themselves need to be sufficiently inexpensive to justify their own processing cost over the gains they bring.

Storage 203

Storage Latency Efficiency Data Engineering 203

The Netflix TechBlog

OCTOBER 15, 2019

Part I: Overview Andreas Andreakis , Falguni Jhaveri , Ioannis Papapanagiotou , Mark Cho , Poorna Reddy , Tongliang Liu Overview It is a commonly observed pattern for applications to utilize multiple datastores where each is used to serve a specific need such as storing the canonical form of data (MySQL etc.), caching (Memcached etc.),

Transportation 192

Transportation Architecture Processing Storage 192

The Netflix TechBlog

FEBRUARY 9, 2022

JAR) form to be executed as part of the user defined data pipeline. data pipeline ?—?a DAG) for the purpose of transforming data using some business logic. Netflix homegrown CLI tool for data pipeline management. task, an atomic unit of data transformation logic, a non-separable execution block in the workflow chain.

Storage 201

Storage Data Engineering Testing Code 201

The Netflix TechBlog

OCTOBER 28, 2021

Data Engineers of Netflix?—?Interview Interview with Pallavi Phadnis This post is part of our “ Data Engineers of Netflix ” series, where our very own data engineers talk about their journeys to Data Engineering @ Netflix. Pallavi Phadnis is a Senior Software Engineer on the Product Data Science and Engineering team.

Data Engineering 173

Data Engineering Engineering Big Data Software Engineering 173

The Netflix TechBlog

JUNE 13, 2023

This is where large-scale system migrations come into play. Our previous blog post presented replay traffic testing — a crucial instrument in our toolkit that allows us to implement these transformations with precision and reliability. Sticky Canary is an improvement to the traditional canary process that addresses this limitation.

Traffic 279

Traffic Metrics Systems Strategy 279

Dynatrace

FEBRUARY 15, 2023

Grail – the foundation of exploratory analytics Grail can already store and process log and business events. Now we’re adding Smartscape to DQL and two new data sources to Grail: Metrics on Grail and Traces on Grail. Ensuring observability across these environments requires access to data at a massive scale.

Analytics 218

Analytics Social Media Metrics Efficiency 218

ACM Sigarch

DECEMBER 19, 2022

If data movement were faster, more work could be done on the same processors. We look here at a Gedankenexperiment: move 16 bytes per cycle , addressing not just the CPU movement, but also the surrounding system design. The new instructions just need control logic to detect zero data movement, and control logic to zero-extend 0.15

Design 145

Design Systems Cache Network 145

The Netflix TechBlog

MARCH 25, 2019

Building and Scaling Data Lineage at Netflix to Improve Data Infrastructure Reliability, and Efficiency By: Di Lin , Girish Lingappa , Jitender Aswani Imagine yourself in the role of a data-inspired decision maker staring at a metric on a dashboard about to make a critical business decision but pausing to ask a question?—?“Can

Infrastructure 219

Infrastructure Big Data Transportation Architecture 219

Dynatrace

SEPTEMBER 7, 2022

Dynatrace and the Dynatrace Intelligent Observability Platform have added support for the newly introduced Amazon VPC Flow Logs to Amazon Kinesis Data Firehose. This support enables customers to define specific endpoint delivery of real-time streaming data to platforms such as Dynatrace. What is VPC Flow Logs? Why Dynatrace?

Traffic 204

Traffic AWS Network Cloud 204

The Netflix TechBlog

DECEMBER 17, 2019

Andreas Andreakis , Ioannis Papapanagiotou Overview Change-Data-Capture (CDC) allows capturing committed changes from a database in real-time and propagating those changes to downstream consumers [1][2]. This motivated the development of DBLog , which offers log and dump processing under a generic framework.

Database 212

Database Traffic Transportation Open Source 212

The Netflix TechBlog

DECEMBER 17, 2019

Andreas Andreakis , Ioannis Papapanagiotou Overview Change-Data-Capture (CDC) allows capturing committed changes from a database in real-time and propagating those changes to downstream consumers [1][2]. This motivated the development of DBLog , which offers log and dump processing under a generic framework.

Database 197

Database Traffic Transportation Open Source 197

The Netflix TechBlog

MAY 4, 2023

Behind the scenes, a myriad of systems and services are involved in orchestrating the product experience. These backend systems are consistently being evolved and optimized to meet and exceed customer and product expectations. This blog series will examine the tools, techniques, and strategies we have utilized to achieve this goal.

Traffic 339

Traffic Latency Tuning Systems 339

The Netflix TechBlog

JUNE 25, 2021

By Gim Mahasintunan on behalf of Data Platform Engineering. Supporting a rapidly growing base of engineers of varied backgrounds using different data stores can be challenging in any organization. In this blog post, we are thrilled to share that we are open-sourcing one such tool: the Netflix Data Explorer.

Metrics 152

Metrics Best Practices Strategy Design 152

Dynatrace

AUGUST 27, 2019

Welcome back to the blog series in which we share how you can easily solve three common problem scenarios by using Dynatrace and xMatters Flow Designer. In Part 1 we explored how DevOps teams can prevent a process crash from taking down services across an organization in five easy steps. Dynatrace news.

Systems 176

Systems DevOps Latency Azure 176

Dynatrace

MAY 2, 2024

With the ability to generate new content—such as images, text, audio, and other data—based on patterns and examples taken from existing data, organizations are rushing to capitalize on the AI model. As organizations train generative AI systems with critical data, they must be aware of the security and compliance risks.

Artificial Intelligence 218

Artificial Intelligence Cloud Best Practices DevOps 218

Dynatrace

JULY 25, 2019

In this three-part blog series, we’ll share the following three common problem scenarios that you can easily solve by building an automated self-healing system with Dynatrace and xMatters Flow Designer: Process crash. Use case #1: Prevent a process crash from taking down services across an organization.

Systems 169

Systems DevOps Design Google 169

Uber Engineering

SEPTEMBER 23, 2021

With this new ability came new challenges that needed to be solved at Uber, such as systems for ad auctions, bidding, attribution, reporting, and more. This article focuses on how we … The post Real-Time Exactly-Once Ad Event Processing with Apache Flink, Kafka, and Pinot appeared first on Uber Engineering Blog.

Processing 141

Processing Engineering Systems Architecture 141

Dynatrace

JUNE 15, 2021

The fact is, Reliability and Resiliency must be rooted in the architecture of a distributed system. The path to “Architected for Resiliency” is long, but it clearly pays off in the long run, especially when outages occur, as I want to show you in this blog post. Fact #2: No significant impact on Dynatrace Users.

AWS 179

AWS Traffic Architecture Azure 179

Dynatrace

APRIL 8, 2022

Fluentd is an open-source data collector that unifies log collection, processing, and consumption. It collects, processes, and outputs log files to and from a wide variety of technologies. Processing plugins parse (normalize), filter, enrich (tagging), format, and buffer log streams. Dynatrace news. What is Fluentd?

Metrics 170

Metrics Analytics Software Architecture Open Source 170

The Netflix TechBlog

JUNE 1, 2023

In this blog post, we’ll discuss the methods we used to ensure a successful launch, including: How we tested the system Netflix technologies involved Best practices we developed Realistic Test Traffic Netflix traffic ebbs and flows throughout the day in a sinusoidal pattern. Basic with ads was launched worldwide on November 3rd.

Traffic 342

Traffic Best Practices Systems Testing 342

Dynatrace

APRIL 19, 2024

In this blog post, we explore real-life scenarios shared by our customers, partners, and internal users at Dynatrace, offering inspiration to kickstart your own journey with custom solutions powered by AppEngine. Using AppEngine as the foundation, they fed disk utilization data into Davis ® AI to forecast the required resizing operation.

AWS 209

AWS Analytics Cloud Monitoring 209

High Scalability

FEBRUARY 19, 2021

This is a guest post by Paddy Byers , Co-founder and CTO at Ably , a realtime data delivery platform. You can view the original article on Ably's blog. This means a system that is not merely available but is also engineered with extensive redundant measures to continue to work as its users expect.

Engineering 215

Engineering Systems Scalability Availability 215

Official Blog - World Web Technology

DECEMBER 4, 2023

How does it contribute to the software development process ? This article will explain why most companies and their software developers use SDLC to manage their development process. SDLC is an essential framework for the total software development process as it aids in managing software development projects effectively.

Processing 64

Processing Software Software Development 64

Dynatrace

AUGUST 10, 2023

They frequently articulate the need for real-time visibility into business data to support agile business decisions. But existing business intelligence (BI) tools often lack the broad context, ease of data access, and real-time insights needed to understand and improve customer experience and complex business processes.

Database 216

Database Metrics Retail Analytics 216

Scalegrid

JULY 17, 2020

It is an open standard format which organizes data into key/value pairs and arrays detailed in RFC 7159. JSON is the most common format used by web services to exchange data, store documents, unstructured data, etc. You can also check out our Working with JSON Data in PostgreSQL vs. JSONB Patterns & Antipatterns.

Storage 321

Storage Database Efficiency Processing 321

Dynatrace

OCTOBER 28, 2020

As a PSM system administrator, you’ve relied on AppMon as a preconfigured APM tool for detecting, diagnosing, and repairing problems that impact the operational health of your Windchill application suite. The post It’s time to upgrade the PTC System Monitor (PSM)! appeared first on Dynatrace blog. Dynatrace news.

Monitoring 143

Monitoring Systems Infrastructure Cloud 143