article thumbnail

Interpreting A/B test results: false negatives and power

The Netflix TechBlog

We then used simple thought exercises based on flipping coins to build intuition around false positives and related concepts such as statistical significance, p-values, and confidence intervals. In this post, we’ll do the same for false negatives and the related concept of statistical power.

Testing 202
article thumbnail

Interpreting A/B test results: false positives and statistical significance

The Netflix TechBlog

As another way to build intuition, consider the real reason that the internet and machine learning exist: to label if images show cats. Designing experiments so that the rate of false positives is minuscule necessarily increases the false negative rate, and vice versa. To build intuition, let’s run through a thought exercise.

Testing 226
Insiders

Sign Up for our Newsletter

This site is protected by reCAPTCHA and the Google Privacy Policy and Terms of Service apply.

Trending Sources

article thumbnail

Google planning a new ‘Badge of Shame’ for slow websites

MachMetrics

The idea behind these UI decisions is that a badge would let a user know quickly if a site is loading slowly due to a problem on their end (network issue, internet is down, etc) or if the website is known to be built in a way that loads slowly. Most stemmed from Google exercising too much power with this move.

Google 86
article thumbnail

5 signs you need to choose cross browser testing on the cloud

Testsigma

We can observe that Samsung Internet 14.2 Cloud-based tools are designed for teams with agile and DevOps cultures, and these tools offer speed by expediting testing in several ways. When you also want to exercise the advantage of parallel test execution. are still in use.

Cloud 79
article thumbnail

How the Swedish Transport Agency learned distributed systems in six months

Particular Software

We’ve certainly been affected by this shift at Particular with the cancellation of our popular Advanced Distributed Systems Design (ADSD) course this year. We skipped a few sessions (the exercises), but in retrospect we should have watched them all I think. Q: How long did it take? We spent maybe 5 hours each day watching the videos.

article thumbnail

Frustrating Design Patterns: Disabled Buttons

Smashing Magazine

Frustrating Design Patterns: Disabled Buttons. Frustrating Design Patterns: Disabled Buttons. After all, as designers and developers, we want to make it more difficult for our users to make mistakes. Part Of: Design Patterns. Is there an issue with your Internet connection? Vitaly Friedman. Part 5: Perfect Slider.

Design 145
article thumbnail

A Decade of Dynamo: Powering the next wave of high-performance, internet-scale applications

All Things Distributed

With these requirements in mind, and a willingness to question the status quo, a small group of distributed systems experts came together and designed a horizontally scalable distributed database that would scale out for both reads and writes to meet the long-term needs of our business. This was the genesis of the Amazon Dynamo database.

Internet 130