Remove Design Remove Exercise Remove Metrics Remove Testing
article thumbnail

Interpreting A/B test results: false negatives and power

The Netflix TechBlog

Martin Tingley with Wenjing Zheng , Simon Ejdemyr , Stephanie Lane , and Colin McFarland This is the fourth post in a multi-part series on how Netflix uses A/B tests to inform decisions and continuously innovate on our products. Have a look at Part 1 (Decision Making at Netflix), Part 2 (What is an A/B Test?), Need to catch up?

Testing 202
article thumbnail

Interpreting A/B test results: false positives and statistical significance

The Netflix TechBlog

Martin Tingley with Wenjing Zheng , Simon Ejdemyr , Stephanie Lane , and Colin McFarland This is the third post in a multi-part series on how Netflix uses A/B tests to inform decisions and continuously innovate on our products. Have a look at Part 1 (Decision Making at Netflix) and Part 2 (What is an A/B Test?). Need to catch up?

Testing 226
Insiders

Sign Up for our Newsletter

This site is protected by reCAPTCHA and the Google Privacy Policy and Terms of Service apply.

Trending Sources

article thumbnail

Building High-Quality Software

DZone

If you start catching bugs early, it will save you tons of time fixing them later.nn> Design reviewnnIt’s a very powerful tool when used in a good way. I really like what one of the smartest people with whom I worked said: “A good design is a design where you can see the code”. There are so many benefits.

Software 245
article thumbnail

Efficient SLO event integration powers successful AIOps

Dynatrace

However, it’s essential to exercise caution: Limit the quantity of SLOs while ensuring they are well-defined and aligned with business and functional objectives. Error budget burn rate = Error Rate / (1 – Target) Best practices in SLO configuration To detect if an entity is a good candidate for strong SLO, test your SLO.

article thumbnail

Site-Speed Topography

CSS Wizardry

Any time you run a test with WebPageTest, you’ll get this table of different milestones and metrics. Note the bottom row shows me the Standard Deviation of the tests’ results. Higher variance means a less stable metric across pages. With my pen and paper, I’ll make a note of investigating these specifically in my testing.

Speed 292
article thumbnail

Azure Well-Architected Framework: What it is and how to tame it with AI and automation

Dynatrace

Most monitoring tools for migrations, development, and operations focus on collecting and aggregating the three pillars of observability— metrics, traces, and logs. The Azure Well-Architected Framework is a set of guiding tenets organizations can use to evaluate architecture and implement designs that will scale over time. Reliability.

Azure 190
article thumbnail

Seamlessly Swapping the API backend of the Netflix Android app

The Netflix TechBlog

To prepare ourselves for a big change in the tech stack of our endpoint, we decided to track metrics around the time taken to respond to queries. After some consultation with our backend teams, we determined the most effective way to group these metrics were by UI screen. For the migration, testing was a first-class citizen.

Latency 233