Remove Analytics Remove Database Remove Scalability Remove Video
article thumbnail

Rebuilding Netflix Video Processing Pipeline with Microservices

The Netflix TechBlog

The Netflix video processing pipeline went live with the launch of our streaming service in 2007. To that end, the Video and Image Encoding team in Encoding Technologies (ET) has spent the last few years rebuilding the video processing pipeline on our next-generation microservice-based computing platform Cosmos.

article thumbnail

Scalable Annotation Service?—?Marken

The Netflix TechBlog

Scalable Annotation Service — Marken by Varun Sekhri , Meenakshi Jindal Introduction At Netflix, we have hundreds of micro services each with its own data models or entities. All data should be also available for offline analytics in Hive/Iceberg. All of these services at a later point want to annotate their objects or entities.

Insiders

Sign Up for our Newsletter

This site is protected by reCAPTCHA and the Google Privacy Policy and Terms of Service apply.

Trending Sources

article thumbnail

Designing Instagram

High Scalability

Generating machine learning based personalized recommendations to discover new people, photos, videos, and stories relevant one’s interest. We will use a graph database such as Neo4j to store the information. Additionally, we can use columnar databases like Cassandra to store information like user feeds, activities, and counters.

Design 334
article thumbnail

Accelerating Data: Faster and More Scalable ElastiCache for Redis

All Things Distributed

Fast Data is an emerging industry term for information that is arriving at high volume and incredible rates, faster than traditional databases can manage. While caching continues to be a dominant use of ElastiCache for Redis, we see customers increasingly use it as an in-memory NoSQL database. Building upon Redis.

article thumbnail

Procella: unifying serving and analytical data at YouTube

The Morning Paper

Procella: unifying serving and analytical data at YouTube Chattopadhyay et al., When each of those use cases is powered by a dedicated back-end, investments in better performance, improved scalability and efficiency etc. Procella achieves high scalability and efficiency by segregating storage (in Colossus) from compute (on Borg).

article thumbnail

Data Reprocessing Pipeline in Asset Management Platform @Netflix

The Netflix TechBlog

Production Use Cases Real-Time APIs (backed by the Cassandra database) for asset metadata access don’t fit analytics use cases by data science or machine learning teams. Production assets operations are performed in parallel with older data reprocessing without any service downtime.

Media 237
article thumbnail

Release readiness through AI-based white box resiliency testing with JMeter and Dynatrace

Dynatrace

To do so we have successfully established AI-based White box load and resiliency testing with JMeter and Dynatrace, helping identify and resolve major performance and scalability problems in recent projects before deploying to production. Our customers usually involve us 2-4 weeks before the production release.

Testing 222