The Netflix TechBlog

OCTOBER 28, 2021

Netflix’s unique work culture and petabyte-scale data problems are what drew me to Netflix. During earlier years of my career, I primarily worked as a backend software engineer, designing and building the backend systems that enable big data analytics. What is your favorite project?

Data Engineering 173

Data Engineering Engineering Big Data Software Engineering 173

Dynatrace

JULY 6, 2022

At its most basic, automating IT processes works by executing scripts or procedures either on a schedule or in response to particular events, such as checking a file into a code repository. When monitoring tools release a stream of alerts, teams can easily identify which ones are false and assess whether an event requires human intervention.

Artificial Intelligence 188

Artificial Intelligence Tuning Strategy Big Data 188

Dynatrace

JULY 5, 2022

AIOps combines big data and machine learning to automate key IT operations processes, including anomaly detection and identification, event correlation, and root-cause analysis. Improved time management and event prioritization. What is AIOps, and how does it work? Seven benefits of AIOps for operational transformation.

Artificial Intelligence 285

Artificial Intelligence Cloud Innovation Strategy 285

Dynatrace

AUGUST 14, 2023

These properties reflect the state of the user and apply to all their associated events and event properties. Dynatrace enables organizations to understand user behavior with big data analytics based on gap-free data, eliminating the guesswork involved in understanding the user experience.

Analytics 229

Analytics Social Media Website IoT 229

The Morning Paper

MAY 14, 2019

Seer: leveraging big data to navigate the complexity of performance debugging in cloud microservices Gan et al., For the services under study, Seer has a sweet spot when trained with around 100GB of data and a 100ms sampling interval for measuring queue depths. ASPLOS’19.

Big Data 45

Big Data Cloud Performance Hardware 45

Dynatrace

OCTOBER 17, 2022

Artificial intelligence for IT operations, or AIOps, combines big data and machine learning to provide actionable insight for IT teams to shape and automate their operational strategy. CloudOps includes processes such as incident management and event management. CloudOps: Applying AIOps to multicloud operations.

DevOps 196

DevOps Artificial Intelligence Healthcare Innovation 196

Dynatrace

APRIL 20, 2023

Further, business leaders must often determine whether the data is relevant for the business and if they can afford it. Logs are automatically produced and time-stamped documentation of events relevant to cloud architectures. Dynatrace Grail unifies data from logs, metrics, traces, and events within a real-time model.

Analytics 187

Analytics Infrastructure Storage Efficiency 187

Dynatrace

MARCH 11, 2021

In the fourth part of the series, I’ll show you how I used Dynatrace’s raw problem and event data to find the best fit for optimized anomaly detection settings. I took a big-data-analysis approach, which started with another problem visualization. Statistically analyzing Dynatrace’s event and problem data.

Tuning 263

Tuning Architecture Monitoring Big Data 263

Dynatrace

MARCH 14, 2022

AIOps (or “AI for IT operations”) uses artificial intelligence so that big data can help IT teams work faster and more effectively. There are two main approaches to AIOps: Traditional AIOps: Machine learning models identify correlations between IT events. Gartner introduced the concept of AIOps in 2016.

Healthcare 227

Healthcare Artificial Intelligence Innovation Strategy 227

Dynatrace

AUGUST 10, 2021

And, they got the chance to do just that at our recent event, DynatraceGo, which I’ll tell you a bit more about in this blog. She dispelled the myth that more big data equals better decisions, higher profits, or more customers. Investing in data is easy but using it is really hard”. DynatraceGo! DynatraceGo!

DevOps 246

DevOps Innovation Big Data Cloud 246

Dynatrace

OCTOBER 14, 2021

Gartner defines AIOps as the combination of “big data and machine learning to automate IT operations processes, including event correlation, anomaly detection, and causality determination.” Typically, only the aggregated events will be accessible to ML and will often exclude additional details. What is AIOps?

Artificial Intelligence 212

Artificial Intelligence DevOps Innovation Metrics 212

Dynatrace

JANUARY 16, 2023

Messaging : RabbitMQ and Kafka are the two main messaging and event streaming systems used. Big data : To store, search, and analyze large datasets, 32% of organizations use Elasticsearch. Accordingly, for classic database use cases, organizations use a variety of relational databases and document stores.

Open Source 343

Open Source Java Operating System Programming 343

The Morning Paper

OCTOBER 26, 2020

As a production system within Microsoft capturing around a quadrillion events and indexing 16 trillion search keys per day it would be interesting in its own right, but there’s a lot more to it than that. it’s getting much more complex and expensive to process, store, and secure potentially sensitive data.

Cloud 104

Cloud Big Data Latency Architecture 104

Dynatrace

DECEMBER 15, 2022

AIOps (artificial intelligence for IT operations) combines big data, AI algorithms, and machine learning for actionable, real-time insights that help ITOps continuously improve operations. ITOps vs. AIOps. The three core components of an AIOps solution are the following: 1. How to modernize ITOps with AIOps.

Artificial Intelligence 202

Artificial Intelligence DevOps Hardware Virtualization 202

ScaleOut Software

JULY 19, 2021

And it can maintain contextual information about every data source (like the medical history of a device wearer or the maintenance history of a refrigeration system) and keep it immediately at hand to enhance the analysis.

IoT 78

IoT Analytics Big Data Architecture 78

Dynatrace

APRIL 25, 2023

Exploratory analytics with collaborative analytics capabilities can be a lifeline for CloudOps, ITOps, site reliability engineering, and other teams struggling to access, analyze, and conquer the never-ending deluge of big data. These analytics can help teams understand the stories hidden within the data and share valuable insights.

Analytics 200

Analytics Big Data Media Operating System 200

Dynatrace

MARCH 24, 2023

Using Marathon, its data center operating system (DC/OS) plugin, Mesos becomes a full container orchestration environment that, like Kubernetes and Docker Swarm, discovers services, balances loads, and manages application containers. Mesos also supports other orchestration engines, including Kubernetes and Docker Swarm.

Infrastructure 186

Infrastructure Open Source Operating System Cloud 186

The Netflix TechBlog

OCTOBER 15, 2019

Beyond data synchronization, some applications also need to enrich their data by calling external services. Delta is an eventual consistent, event driven, data synchronization and enrichment platform. CDC (Change-Data-Capture) events are sent by the Delta-Connector to a Keystone Kafka topic.

Transportation 192

Transportation Architecture Processing Storage 192

Dynatrace

FEBRUARY 16, 2023

As teams try to gain insight into this data deluge, they have to balance the need for speed, data fidelity, and scale with capacity constraints and cost. To solve this problem, Dynatrace launched Grail, its causational data lakehouse , in 2022. But logs are just one pillar of the observability triumvirate.

Analytics 186

Analytics Innovation Metrics Database 186

Dynatrace

JUNE 1, 2020

Artificial intelligence for IT operations (AIOps): AIOps platforms combine big data and machine learning functionality to support IT operations.

Artificial Intelligence 182

Artificial Intelligence Social Media Monitoring IoT 182

The Netflix TechBlog

MAY 4, 2023

In this approach, we record the requests and responses for the service that needs to be updated or replaced to an offline event stream asynchronously. Additionally, for mismatches, we record the normalized and unnormalized responses from both sides to another big data table along with other relevant parameters, such as the diff.

Traffic 339

Traffic Latency Tuning Systems 339

Scalegrid

MARCH 28, 2024

On the other hand, an append-only file ensures data safety by recording every write operation that modifies the dataset, allowing for complete data reconstruction in the event of a restart. Advanced Redis Features Showdown Big data center concept, cloud database, server power station of the future. 3d render.

Cache 130

Cache Storage Scalability Architecture 130

Dynatrace

JUNE 1, 2020

Artificial intelligence for IT operations (AIOps): AIOps platforms combine big data and machine learning functionality to support IT operations.

Monitoring 132

Monitoring Performance Social Media Artificial Intelligence 132

High Scalability

APRIL 14, 2020

Scrapinghub is hiring a Senior Software Engineer (Big Data/AI). You will be designing and implementing distributed systems : large-scale web crawling platform, integrating Deep Learning based web data extraction components, working on queue algorithms, large datasets, creating a development platform for other company departments, etc.

Education 105

Education Software Engineering Scalability Engineering 105

High Scalability

MAY 12, 2020

Scrapinghub is hiring a Senior Software Engineer (Big Data/AI). You will be designing and implementing distributed systems : large-scale web crawling platform, integrating Deep Learning based web data extraction components, working on queue algorithms, large datasets, creating a development platform for other company departments, etc.

Education 102

Education Software Engineering Scalability Engineering 102

High Scalability

MARCH 24, 2020

Scrapinghub is hiring a Senior Software Engineer (Big Data/AI). You will be designing and implementing distributed systems : large-scale web crawling platform, integrating Deep Learning based web data extraction components, working on queue algorithms, large datasets, creating a development platform for other company departments, etc.

Education 105

Education Software Engineering Engineering Big Data 105

The Netflix TechBlog

JUNE 7, 2021

Flow Collector consumes two data streams, the IP address change events from Sonar via Kafka and eBPF flow log data from the Flow Exporter sidecars. It performs real time attribution of flow data with application metadata from Sonar. We use Sonar to attribute an IP address to a specific application at a particular time.

Network 325

Network Transportation AWS Cloud 325

High Scalability

APRIL 28, 2020

Scrapinghub is hiring a Senior Software Engineer (Big Data/AI). You will be designing and implementing distributed systems : large-scale web crawling platform, integrating Deep Learning based web data extraction components, working on queue algorithms, large datasets, creating a development platform for other company departments, etc.

Education 102

Education Software Engineering Scalability Engineering 102

High Scalability

MARCH 30, 2020

Scrapinghub is hiring a Senior Software Engineer (Big Data/AI). You will be designing and implementing distributed systems : large-scale web crawling platform, integrating Deep Learning based web data extraction components, working on queue algorithms, large datasets, creating a development platform for other company departments, etc.

Education 105

Education Software Engineering Engineering Big Data 105

High Scalability

MAY 26, 2020

Scrapinghub is hiring a Senior Software Engineer (Big Data/AI). You will be designing and implementing distributed systems : large-scale web crawling platform, integrating Deep Learning based web data extraction components, working on queue algorithms, large datasets, creating a development platform for other company departments, etc.

Education 100

Education AWS Software Engineering Scalability 100

Dynatrace

APRIL 11, 2023

The focus on bringing various organizational teams together—such as development, business, and security teams — makes sense as observability data, security data, and business event data coalesce in these cloud-native environments.

Cloud 186

Cloud DevOps Open Source Retail 186

Alex Podelko

DECEMBER 19, 2019

Boris has unique expertise in that area – especially in Big Data applications. To facilitate discussions, in addition to Q&A, we have panels, “Meeting of the Minds” sessions, and networking events. How to select appropriate IT Infrastructure to support Digital Transformation by Boris Zibitsker, BEZNext.

Efficiency 100

Efficiency Artificial Intelligence Scalability Performance 100

All Things Distributed

SEPTEMBER 26, 2014

We’re at 925 Market Street and our doors will be open 10AM to 6PM on weekdays, with select events running until 8PM on weeknights. Be sure to check the calendar regularly as new evening events will be added regularly. What’s Happening at the AWS Loft. AWS Technical Bootcamps.

AWS 111

AWS Games Education Innovation 111

All Things Distributed

MAY 27, 2015

Technical Presentations: AWS solution architects, product managers, and evangelists deliver technical presentations covering some of the highest-rated and best-attended sessions from recent AWS events. Both the San Francisco loft and the New York City loft have packed calendars with events.

AWS 95

AWS Education Big Data Games 95

All Things Distributed

JULY 16, 2012

It is shaping up to be a great event with many Amazonians, partners and customers presenting in well over 150 sessions. The first annual AWS user and partner conference will be held November 27-29 at The Venetian in Las Vega.

AWS 83

AWS Big Data Media Storage 83

Abhishek Tiwari

DECEMBER 22, 2017

Cheap storage and on-demand compute in the cloud coupled with the emergence of new big data frameworks and tools are forcing us to rethink the whole ETL and data warehousing architecture. This can be time-consuming and bit challenging if you are dealing with hundreds of data sources and thousands of event types.

Big Data 40

Big Data Retail Storage Google 40

All Things Distributed

JUNE 20, 2017

Beyond running their web properties and applications, Next Digital also uses Amazon RDS (database), Amazon ElastiCache (caching), and Amazon Redshift (data warehousing). I'm excited to see the new and innovative use cases coming from our customers in Hong Kong and across Asia Pacific, all enabled by AWS.

AWS 146

AWS Logistics Cloud Social Media 146