Remove Design Remove Exercise Remove Latency Remove Strategy
article thumbnail

Interpreting A/B test results: false negatives and power

The Netflix TechBlog

We then used simple thought exercises based on flipping coins to build intuition around false positives and related concepts such as statistical significance, p-values, and confidence intervals. As a result, if the test treatment results in a small reduction in the latency metric, it’s hard to successfully identify?

Testing 202
article thumbnail

Seamlessly Swapping the API backend of the Netflix Android app

The Netflix TechBlog

Over the course of this post, we will talk about our approach to this migration, the strategies that we employed, and the tools we built to support this. Functional Testing Functional testing was the most straightforward of them all: a set of tests alongside each path exercised it against the old and new endpoints.

Latency 233
Insiders

Sign Up for our Newsletter

This site is protected by reCAPTCHA and the Google Privacy Policy and Terms of Service apply.

Trending Sources

article thumbnail

Failure Modes and Continuous Resilience

Adrian Cockcroft

There are many possible failure modes, and each exercises a different aspect of resilience. Another problem is that a design control, intended to mitigate a failure mode, may not work as intended. STPA is based on a functional control diagram of the system, and the safety constraints and requirements for each component in the design.

Latency 52
article thumbnail

Failure Modes and Continuous Resilience

Adrian Cockcroft

There are many possible failure modes, and each exercises a different aspect of resilience. Another problem is that a design control, intended to mitigate a failure mode, may not work as intended. STPA is based on a functional control diagram of the system, and the safety constraints and requirements for each component in the design.

Latency 53
article thumbnail

Trade-offs under pressure: heuristics and observations of teams resolving internet service outages (Part II)

The Morning Paper

1:18pm a key observation was made that an API call to populate the homepage sidebar saw a huge jump in latency. The process tracing exercise included: Examning IRC transcripts from multiple channels. Members of the team begin diagnosing the issue using the #sysops and #warroom internal IRC channels.

article thumbnail

Fixing a slow site iteratively

CSS - Tricks

With all of this in mind, I thought improving the speed of my own version of a slow site would be a fun exercise. In that spirit, what we’re looking at in this article is focused more on the incremental wins and less on providing an exhaustive list or checklist of performance strategies. Again, every millisecond counts. Lighthouse.

Cache 92
article thumbnail

Transforming enterprise integration with reactive streams

O'Reilly Software

Service-oriented architecture (SOA) was hyped in the mid-2000s as a modern take on distributed systems architecture, which through modular design would provide productivity through loose coupling between collaborative services—so-called "WebServices"—communicating through externally published APIs. of ( Invoice. alsoTo ( Sink. of ( Order.