article thumbnail

Auto-Diagnosis and Remediation in Netflix Data Platform

The Netflix TechBlog

Pensive infrastructure comprises two separate systems to support batch and streaming workloads. This blog will explore these two systems and how they perform auto-diagnosis and remediation across our Big Data Platform and Real-time infrastructure. Expand Pensive with Machine Learning classifiers.

Big Data 238
article thumbnail

Data Engineers of Netflix?—?Interview with Pallavi Phadnis

The Netflix TechBlog

Netflix’s unique work culture and petabyte-scale data problems are what drew me to Netflix. During earlier years of my career, I primarily worked as a backend software engineer, designing and building the backend systems that enable big data analytics.

Insiders

Sign Up for our Newsletter

This site is protected by reCAPTCHA and the Google Privacy Policy and Terms of Service apply.

Trending Sources

article thumbnail

What is IT automation?

Dynatrace

Vulnerability management is one example of a DevSecOps workflow that teams should automate to ensure vulnerability scans run regularly. With ever-evolving infrastructure, services, and business objectives, IT teams can’t keep up with routine tasks that require human intervention. Big data automation tools.

article thumbnail

RSA Guide 2023: Cloud application security remains core challenge for organizations

Dynatrace

For example, the open source Java library at the heart of the Log4Shell crisis in 2021 was patched within days given the pervasiveness of the code. Observability is critical for monitoring application performance, infrastructure, and user behavior within hybrid, microservices-based environments.

Cloud 187
article thumbnail

Seer: leveraging big data to navigate the complexity of performance debugging in cloud microservices

The Morning Paper

Seer: leveraging big data to navigate the complexity of performance debugging in cloud microservices Gan et al., When a QoS violation is predicted to occur and a culprit microservice located, Seer uses a lower level tracing infrastructure with hardware monitoring primitives to identify the reason behind the QoS violation.

article thumbnail

The Need for Real-Time Device Tracking

ScaleOut Software

We are increasingly surrounded by intelligent IoT devices, which have become an essential part of our lives and an integral component of business and industrial infrastructures. Real-Time Device Tracking with In-Memory Computing Can Fill an Important Gap in Today’s Streaming Analytics Platforms. The list goes on.

IoT 78
article thumbnail

What is Greenplum Database? Intro to the Big Data Database

Scalegrid

When handling large amounts of complex data, or big data, chances are that your main machine might start getting crushed by all of the data it has to process in order to produce your analytics results. Greenplum features a cost-based query optimizer for large-scale, big data workloads. Greenplum Advantages.

Big Data 321