Remove Design Remove Event Remove Java Remove Scalability
article thumbnail

Pinterest Open-Sources a Production-Ready PubSub Java Client for Kafka, Flink, and MemQ

InfoQ

The library helped the engineering teams by increasing developer velocity, and the scalability and stability of services using it. Over 90% of Java applications have migrated to PSC with minimal changes. Pinterest open-sourced its generic PubSub client library, PSC, which has been heavily used in production for a year and a half.

article thumbnail

Data Reprocessing Pipeline in Asset Management Platform @Netflix

The Netflix TechBlog

This feature support required a significant update in the data table design (which includes new tables and updating existing table columns). Data Sharding strategy in elasticsearch is updated to provide low search latency (as described in blog post) Design of new Cassandra reverse indices to support different sets of queries.

Media 237
Insiders

Sign Up for our Newsletter

This site is protected by reCAPTCHA and the Google Privacy Policy and Terms of Service apply.

Trending Sources

article thumbnail

Orchestrating Data/ML Workflows at Scale With Netflix Maestro

The Netflix TechBlog

As Big data and ML became more prevalent and impactful, the scalability, reliability, and usability of the orchestrating ecosystem have increasingly become more important for our data scientists and the company. Motivation Scalability and usability are essential to enable large-scale workflows and support a wide range of use cases.

Java 202
article thumbnail

Sponsored Post: ipdata, StackHawk, InterviewCamp.io, Educative, Triplebyte, Stream, Fauna

High Scalability

has hours of system design content. They also do live system design discussions every week. Learn to balance architecture trade-offs and design scalable enterprise-level software. Check out Educative.io 's bestselling new 4-course learning track: Scalability and System Design for Developers. Who's Hiring?

Education 115
article thumbnail

Top 3 Kafka Streams Challenges

VoltDB

Fault tolerance: Kafka Streams is designed to be fault-tolerant and scalable. Java-based: Kafka Streams is primarily a Java library, but it also has support for other JVM languages like Scala. This makes it easy for Java developers to build and deploy stream processing applications using familiar tools and frameworks.

Java 52
article thumbnail

Sponsored Post: G-Core Labs, Close, Wynter, Pinecone, Kinsta, Bridgecrew, IP2Location, StackHawk, InterviewCamp.io, Educative, Stream, Fauna, Triplebyte

High Scalability

has hours of system design content. They also do live system design discussions every week. Learn to balance architecture trade-offs and design scalable enterprise-level software. Check out Educative.io 's 5-part learning path: Scalability and System Design for Developers. Fun and Interesting Events.

Education 100
article thumbnail

Sponsored Post: Close, Wynter, Pinecone, Kinsta, Bridgecrew, IP2Location, StackHawk, InterviewCamp.io, Educative, Stream, Fauna, Triplebyte

High Scalability

has hours of system design content. They also do live system design discussions every week. Learn to balance architecture trade-offs and design scalable enterprise-level software. Check out Educative.io 's 5-part learning path: Scalability and System Design for Developers. Fun and Interesting Events.

Education 100