Skip to main content
Uber logo

Schedule rides in advance

Reserve a rideReserve a ride

Schedule rides in advance

Reserve a rideReserve a ride
Engineering

Cherami: Uber Engineering’s Durable and Scalable Task Queue in Go

December 6, 2016 / Global
Featured image for Cherami: Uber Engineering’s Durable and Scalable Task Queue in Go
Cher Ami was a US Army carrier pigeon in World War I. Despite being shot in the leg, she successfully delivered a message that helped save 194 lives.
was a . Despite being shot in the leg, she successfully delivered a message that .
Producers enqueue tasks into queues A and B. Queue A feeds to two consumers groups that both receive all tasks, distributed across consumers within the respective group. Queue B feeds to only one consumer group.
Producers enqueue tasks into queues A and B. Queue A feeds to two consumers groups that both receive all tasks, distributed across consumers within the respective group. Queue B feeds to only one consumer group.
The append-only property allows a queue to remain available for publishing during hardware failure.
The append-only property allows a queue to remain available for publishing during hardware failure.
Cherami handles a storage host failure.
Cherami handles a storage host failure.
Input host only receives acks for the first three messages from all storage hosts. It acks the first three messages to the producer, as those messages are guaranteed to be fully replicated.
Input host only receives acks for the first three messages from all storage hosts. It acks the first three messages to the producer, as those messages are guaranteed to be fully replicated.
Output host handles out-of-order acks from workers.
Output host handles out-of-order acks from workers.
Interaction of Cherami’s system components.
Interaction of Cherami’s system components.
Xu Ning

Xu Ning

Xu Ning is a Senior Engineering Manager in Uber’s Seattle Engineering office, currently leading multiple development teams in Uber’s Michelangelo Machine Learning Platform. He previously led Uber's Cherami distributed task queue, Hadoop observability, and Data security teams.

Maxim Fateev

Maxim Fateev

Maxim Fateev is a staff software engineer at Uber.

Posted by Xu Ning, Maxim Fateev

Category: