article thumbnail

Migrating Critical Traffic At Scale with No Downtime?—?Part 1

The Netflix TechBlog

This blog series will examine the tools, techniques, and strategies we have utilized to achieve this goal. In this testing strategy, we execute a copy (replay) of production traffic against a system’s existing and new versions to perform relevant validations. This approach has a handful of benefits.

Traffic 339
article thumbnail

Interpreting A/B test results: false negatives and power

The Netflix TechBlog

We then used simple thought exercises based on flipping coins to build intuition around false positives and related concepts such as statistical significance, p-values, and confidence intervals. As a result, if the test treatment results in a small reduction in the latency metric, it’s hard to successfully identify?

Testing 202
Insiders

Sign Up for our Newsletter

This site is protected by reCAPTCHA and the Google Privacy Policy and Terms of Service apply.

Trending Sources

article thumbnail

Seamlessly Swapping the API backend of the Netflix Android app

The Netflix TechBlog

Over the course of this post, we will talk about our approach to this migration, the strategies that we employed, and the tools we built to support this. Functional Testing Functional testing was the most straightforward of them all: a set of tests alongside each path exercised it against the old and new endpoints.

Latency 233
article thumbnail

Real user monitoring vs. synthetic monitoring: Understanding best practices

Dynatrace

connectivity, access, user count, latency) of geographic regions. For example, real-user monitoring metrics might reveal a user performance issue that you can then apply to synthetic testing to replicate the issue by exercising the same transaction across several different variables. Performance testing based on variable metrics (i.e.,

article thumbnail

Bring Your Own Cloud (BYOC) vs. Dedicated Hosting at ScaleGrid

Scalegrid

In this post, we compare ScaleGrid’s Bring Your Own Cloud (BYOC) plan vs. the standard Dedicated Hosting model to help you determine the best strategy for your MySQL, PostgreSQL, Redis™ and MongoDB® database deployment. Deploying your application and database on the same VPC also provides the lowest possible latency path. No problem.

Cloud 242
article thumbnail

Taiji: managing global user traffic for large-scale Internet services at the edge

The Morning Paper

Taiji’s routing table is a materialized representation of how user traffic at various edge nodes ought to be distributed over available data centers to balance data center utilization and minimize latency. For example, balance utilisation across all data centers, or optimise for network latency.

Traffic 42
article thumbnail

Trade-offs under pressure: heuristics and observations of teams resolving internet service outages (Part II)

The Morning Paper

1:18pm a key observation was made that an API call to populate the homepage sidebar saw a huge jump in latency. The process tracing exercise included: Examning IRC transcripts from multiple channels. Members of the team begin diagnosing the issue using the #sysops and #warroom internal IRC channels.