Working at Netflix 2017

Brendan Gregg

I've now worked at Netflix for over three years. Time flies! I previously wrote about Netflix in [2015] and [2016], and if you are interested in what it's like to work here, I already covered much in those posts. As before, no one at Netflix has asked me to write this, and this is my personal blog and not a company post. I'll start with some exciting news, describe what my job is really like, the culture and mission, and some work updates. ## 100 Million Subscribers!

Experience SQL Server 2017: Start your journey here

SQL Server According to Bob

Please check out these new resources for SQL Server 2017 and SQL Server on Linux and Docker: Brk2045 upgrade sql server 2017 (on prem, iaa-s and paas) from Bob Ward. With the 2017 PASS Summit approaching next week, I thought it would be helpful to provide for you my list of resources when it comes to SQL Server 2017, which we announced for General Availability on October 2nd. SQL Server 2017 Release Notes. bobsql blog posts on SQL Server 2017.

Azure 40

Sign Up for our Newsletter

This site is protected by reCAPTCHA and the Google Privacy Policy and Terms of Service apply.

SQL Server 2017 generally available on October 2nd…

SQL Server According to Bob

Scott Guthrie and Rohan Kumar officially made the announcement today at Ignite that SQL Server 2017 will be generally available on October 2nd, 2017. Andrew Brust also wrote a very nice article detailing out all the great new features that come with SQL Server 2017. Stay tuned to hear more about SQL Server 2017 over the next several weeks both on our main SQL Server blog [link] and here at the bobsql blog: [link]. SQL Server 2017

Solaris to Linux Migration 2017

Brendan Gregg

Many people have contacted me recently about switching from Solaris (or illumos) to Linux, especially since most of the Solaris kernel team were let go this year (including my former colleagues, I'm sorry to hear). This includes many great engineers who I'm sure will excel in whatever they choose to work on next. They have been asking me about Linux because I've worked for years on each platform: Solaris, illumos, and Linux, in all cases full time and as a subject matter expert.

Experience SQL Server 2017: The Fast and the Furious

SQL Server According to Bob

Please check out a new blog post I created detailing how SQL Server 2017 is Fast, Built-In. Look for more details on my blog post and the SQL Server Blog about some of these concepts and new features for performance for SQL Server 2017. I just presented on this topic at Ignite and will repeat this same session at the upcoming SQL 2017 PASS Summit. SQL Server 2017 Fast and the Furious

AWS EC2 Virtualization 2017: Introducing Nitro

Brendan Gregg

Xen AWS 2017 In 2015, AWS launched c4, which used hardware virtualization for EBS volumes. This finally came to storage-optimized instance types in 2017 with the [i3 instance type], which used SR-IOV and the nvme storage driver. AWS Nitro 2017. --> As was announced at AWS re:Invent last night, and covered in Anthony Liguori's talk today ([CMP332]: video ), and the bare metal talk ([CMP330]: video ), the [c5 instance type] uses a new hypervisor called Nitro.

How to Optimize Website Speed for Black Friday 2017


The post How to Optimize Website Speed for Black Friday 2017 appeared first on Abstracta Software Testing Services. Is your website prepared for record breaking traffic and sales? Halloween is almost here and before you know it, it will be Black Friday and then, Cyber Monday. These consumer “holidays” bring about the race for shoppers to buy all of the coveted items on. Performance Engineering Software Testing Black Friday load testing Page Speed Performance Performance Testing

Speed 43

Total Solar Eclipse 2017

Brendan Gregg

While all eyes and instruments were aimed at the total eclipse of August 21st, I had a video camera pointed in the opposite direction. I was hoping to catch changes in the landscape – perhaps the Moon's shadow as it raced by. This was filmed from on a road north east of Madras, Oregon, almost right on the center line. It was just an experiment that I wasn't going to share, but when I saw it afterwards I thought it might interest some.

2017 Wheel of Fortune

J. Paul Reed

I mentioned last week I’d have 2017 predictions for you… well, last week. So, what do I think 2017 will have in store for us? Prediction #1: DevOps Will Be Declared 1.0-STABLE This prediction was originally published in a larger T echBeacon piece on 2017 DevOps predictions. But we’ll see this mechanism happen in 2017. So ultimately, in 2017, Release Engineering will need to (continue?)

IoT 40

Serverless at re:Invent 2017

The Symphonia

Serverless at re:Invent 2017 was originally published in The Symphonium on Medium, where people are continuing the conversation by highlighting and responding to this story. The light show at the re:Play party Another re:Invent has come and gone, and us mere AWS-using mortals are now rapidly trying to sort the wheat from the chaff of a heady harvest of announcements. It’s funny to think that AWS Lambda was announced at re:Invent only 3 years ago?—?the

Excited by the Upcoming CMG imPACt Performance and Capacity Conference

Alex Podelko

This year it would be held on November 6-9, 2017 in New Orleans, LA. A lot of events are scheduled: Performance and Capacity Hackathon (which is $10 / free for the conference attendees), Performance Testing Workshop, CMG Training / Badge Program, and the CMG imPACt 2017 conference itself. I am very excited by the upcoming CMG imPACt performance and capacity conference.

IoT 109

Follow up on the Test Automation Discussion – Bringing in the Performance View

Alex Podelko

There was a rather heated discussion around A Context-Driven Approach to Automation in Testing by James Bach and Michael Bolton (referred below as the article and the authors – other references would be explicit). Here are a couple of places where it got fiercely attacked Reviewing “Context Driven Approach to Automation in Testing” by Chris McMahon. and Open letter to “CDT Test Automation” reviewers as well as in many other different places.

Symphonia’s Serverless Insights — December 2017

The Symphonia

Hello everyone, and welcome to the last Serverless Insights of 2017! Symphonia’s Serverless Insights — December 2017 was originally published in The Symphonium on Medium, where people are continuing the conversation by highlighting and responding to this story. This is the latest edition of our newsletter. To receive the email version please subscribe here.

AWS re:Invent 2017: How Netflix Tunes EC2

Brendan Gregg

My last talk for 2017 was at AWS re:Invent, on "How Netflix Tunes EC2 Instances for Performance," an updated version of my [2014] talk. WARNING: These tunables were developed in late 2017, for Ubuntu Xenial instances on EC2.** There was so much demand for it this year that I had three overflow rooms streaming it, and people still couldn't get in. (I I shouldn't let this go to my head, as there were 42,000 attendees at re:Invent looking for something to see!)

PerformanceObserver and Paint Timing API


Image taken from the Chrome 60 blog post , which first appeared in “Web Performance: Leveraging the Metrics that Most Affect User Experience” at Google I/O 2017 Up until now we have been measuring performance through other metrics, mostly using the Navigation Timing API , which is also what Google Analytics uses for their Site Speed report.

Google 100

The Role of Recording in Load Testing Tools

Alex Podelko

An interesting discussion started around a very good post Open Source Load Testing Tool Review by Ragnar Lönn. Somehow the latest comment, which I got in my e-mail, didn’t get published. So I decided to copy the discussion as is and publish the comment (and my answer) here. Check the original post (as well as many other great posts) following the link above – but the discussion below is somewhat independent.

SQL Server Management Studio Provides–“XE Profiler”

SQL Server According to Bob

Here is an example using the SQL Server SSMS 2017 against my SQL 2016 server. SQL 2016 SQL Server 2017 XEventBob Ward and I worked with our SQL Server Tool developers (thanks David) to enable ‘Q uick XE Trace ’ capabilities. The feature is available in the latest SQL Server Management Studio (SSMS) release.

How to design a RESTful API architecture from a human-language spec

O'Reilly Software

A process to build RESTful APIs that solve users’ needs with simplicity, reliability, and performance. Every piece of software exists to solve a real-world problem. Directly or indirectly. Most web APIs are consumed by client applications running on PCs, mobile devices, etc., which in turn are used by humans. Despite being consumed directly by machines, APIs are made to satisfy the needs of human beings, so designing them should follow a user-centered process, but often it doesn’t.

SQL Server Linux: Distributed transactions requiring the Microsoft Distributed Transaction Coordinator service are not supported on SQL Server running on Linux. SQL Server to SQL Server distributed transactions are supported. – CLARIFIED!!!

SQL Server According to Bob

Current limitation: Engaging in a distributed transaction with SQL Server 2017 on Linux. SQL Server 2017 on Linux does not have distributed transaction support. However, SQL Server 2017 on Windows provides distributed transaction support allowing interactions with SQL Server 2019 on Linux. SQL Server 2017 ( Historical Information ). Linux SQL Server 2017 SQL Server on LinuxSQL Server 2019 CTP2.

As-Salaam-Alaikum: The cloud arrives in the Middle East!

All Things Distributed

2017 continues a busy year for AWS in the Middle East. They are also migrating 700 government websites, with more than 50 TB of data, onto AWS, helping them to meet their goal of decommissioning their hosting platform by the end of 2017. Today, I am excited to announce plans for Amazon Web Services (AWS) to bring an infrastructure Region to the Middle East!

AWS 100

Redis vs Memcached

Software Architecture

Memcached is in-memory key value store whereas Redis is in-memory data structures store. Memcached supports only string data type which is ideal for storing read-only data. Redis supports almost all types of data. Redis can also be used as a messaging system such as pubsub. Memcached cannot be used as a pubsub system. Memcached is more memory efficient than Redis. Redis is more memory efficient only after you use hashes. Memcached is volatile store and Redis is non-volatile.

Data Consistency in Apache Cassandra — Part 1

Software Architecture

For a quick introduction on what Apache Cassandra is, take a look here. Consistency is a significantly large topic to cover in one part. So I’ll be completing it in 3 parts. This first part defines consistency in general, write consistency, read consistency, consistency levels (CL), immediate, eventual and tunable consistency. Consistency The topic and concept of consistency is very important when you work with a distributed database like Cassandra.

Data Consistency in Apache Cassandra — Part 2

Software Architecture

In part 1 , I introduced the basics of consistency in general, write consistency, read consistency, consistency levels (CL), immediate, eventual and tunable consistency. In this part 2, I will talk about how to achieve immediate and eventual consistency using different write and read consistency levels.

Tuning 100

Data Consistency in Apache Cassandra — Part 3

Software Architecture

In part 2 , I explained how to achieve immediate and eventual consistency using different write and read consistency levels. In this part, I’ll go a bit deeper into understanding different configuration settings and consistency levels. Immediate Consistency with Write CL = ALL, Read CL = ONE The write request is sent to all replicas. Complete write operation is considered successful when all replicas respond a write success.

A Decade of Dynamo: Powering the next wave of high-performance, internet-scale applications

All Things Distributed

In support of Amazon Prime Day 2017, the biggest day in Amazon retail history, DynamoDB served over 12.9 Today marks the 10 year anniversary of Amazon's Dynamo whitepaper , a milestone that made me reflect on how much innovation has occurred in the area of databases over the last decade and a good reminder on why taking a customer obsessed approach to solving hard problems can have lasting impact beyond your original expectations.

How a RESTful API server reacts to requests

O'Reilly Software

Learn how to properly design RESTful APIs communication with clients, accounting for request structure, authentication, and caching. This series of articles shows you how to derive an easy-to-use, robust, efficient API to serve users on the web or on mobile devices. We are using the principles of RESTful architecture over HTTP. In the first piece, we started from a list of specs for a simple bike rental service, defining URLs and the HTTP methods to serve the app.

Cache 67

Improving Customer Service with Amazon Connect and Amazon Lex

All Things Distributed

So much so, that in March 2017, we announced Amazon Connect, which is the result of nearly ten years of work to build cloud-based contact centers at scale to power customer service for more than 50 Amazon teams and subsidiaries, including, Zappos, and Audible. Customer service is central to the overall customer experience that all consumers are familiar with when communicating with companies.

Becoming an accidental architect

O'Reilly Software

How software architects can balance technical proficiencies with an appropriate mastery of communication. One of the demographics Brian and I noticed in the several O'Reilly Software Architecture Conferences we've hosted is the Accidental Architect : someone who makes architecture-level decisions on projects without a formal Architect title. Over time, we're building more material into the conference program to accommodate this common role.

AI for everyone - How companies can benefit from the advance of machine learning

All Things Distributed

This article titled " Wie Unternehmen vom Vormarsch des maschinellen Lernens profitieren können " appeared in German last week in the "Digitaliserung" column of Wirtschaftwoche. When a technology has its breakthrough, can often only be determined in hindsight. In the case of artificial intelligence (AI) and machine learning (ML), this is different. ML is that part of AI that describes rules and recognizes patterns from large amounts of data in order to predict future data.

Brilliant Jerks in Engineering

Brendan Gregg

My colleague Justin Becker is also giving a talk at QConSF 2017 on the topic: [Am I a Brilliant Jerk?]. Houghton Mifflin Harcourt, 2017. - [link] - [link] - [link] - [link] - [link] - [link] - [link] - [link] - [link] - [link] - The blood bag series: part 1 , part 2 , part 3. - (Updated) discussion on hackernews. [Am Notice board at Ericsson, Stockholm (pic by DeirdreS ).

SQL Server 2017 Improved Resource Usage On Smaller Machines

SQL Server According to Bob

The changes made in SQL Server 2017 CU4 improve resource usage on smaller machines, such as a VM or Container used for testing and development. The change I made in SQL Server 2017 CU4 was to consider RAM as another factor in the default calculation. The change I made in SQL Server 2017 CU4 is to use Client garbage collection mode on smaller machines, removing the additional overhead of Server mode garbage collection. Linux SQL Server 2017

Scaling Amazon ElastiCache for Redis with Online Cluster Resizing

All Things Distributed

Amazon ElastiCache embodies much of what makes fast data a reality for customers looking to process high volume data at incredible rates, faster than traditional databases can manage. Developers love the performance, simplicity, and in-memory capabilities of Redis, making it among the most popular NoSQL key-value stores. Redis's microsecond latency has made it a de facto choice for caching.

Games 94

Expanding the Cloud – An AWS Region is coming to Hong Kong

All Things Distributed

Today, I am very excited to announce our plans to open a new AWS Region in Hong Kong! The new region will give Hong Kong-based businesses, government organizations, non-profits, and global companies with customers in Hong Kong, the ability to leverage AWS technologies from data centers in Hong Kong. The new AWS Asia Pacific (Hong Kong) Region will have three Availability Zones and be ready for customers for use in 2018. Over the past decade, we have seen tremendous growth at AWS.

Software Developers Should Have Sysadmin Experience

Professor Beekums

Being? ?a? software? ?developer? developer? ?and? and? ?being? being? ?a? system? ?administrator? administrator? ?are? are? ?very? very? ?different? different? ?things.? things.? ?Many Many folks? ?lump? lump? ?the? the? ?two? two? ?professions? professions? ?together,? together,? ?but? but? ?the? the? ?skillsets? skillsets? ?do? not? ?overlap? overlap? ?much.? much.? ?Software Software developers? ?write? write? ?code.? code.? ?System? System? ?administrators? administrators? ?maintain?

This spring: High-Performance and Low-Latency C++ (Stockholm) and ACCU (Bristol)

Sutter's Mill

Next, I’ll be heading to Bristol to catch the end of the ACCU 2017 conference, and give the closing talk on “Something(s) New in C++.” Here is the current abstract: By the time the ACCU 2017 conference begins, C++17 is expected to be technically complete and in its final approval ballot.

Shift-Left Testing in the Enterprise and the Case for Open Source


Why continuous testing and open source are a perfect match I recently visited the offices of CA Technologies (one of Abstracta’s partners) in Santa Clara, where I had the chance to discuss shift-left testing, continuous testing, and why and how to turn to open source. The post Shift-Left Testing in the Enterprise and the Case for Open Source appeared first on Abstracta Software Testing Services.

Weekend Reading: Amazon Aurora: Design Considerations for High Throughput Cloud-Native Relational Databases.

All Things Distributed

Amazon Aurora: Design Considerations for HighThroughput Cloud-Native Relational Databases ", Alexandre Verbitski, Anurag Gupta, Debanjan Saha, Murali Brahmadesam, Kamal Gupta, Raman Mittal, Sailesh Krishnamurthy, Sandor Maurice, Tengiz Kharatishvili, Xiaofeng Bao, in SIGMOD '17 Proceedings of the 2017 ACM International Conference on Management of Data, Pages 1041-1052 May 14 – 19, 2017, Chicago, IL, USA

Can You Afford It?: Real-world Web Performance Budgets

Alex Russell

— Alex Russell (@slightlylate) October 4, 2017. We now have everything we need to create a ballpark perf budget for a product in 2017. TL;DR: performance budgets are an essential but under-appreciated part of product success and team health. Most partners we work with are not aware of the real-world operating environment and make inappropriate technology choices as a result.

Coming to STATION F: The first Mentor's Office powered by AWS!

All Things Distributed

I am excited to announce that AWS is opening its first Mentor's Office at STATION F in Paris! The Mentor's Office is a workplace exclusively dedicated to meetings between AWS experts and the startups. STATION F is the world's biggest startup campus. With this special offer starting at the end of June, at the campus opening, AWS increases the support already available to startup customers in France.

Linux Load Averages: Solving the Mystery

Brendan Gregg

Load averages are an industry-critical metric – my company spends millions auto-scaling cloud instances based on them and other metrics – but on Linux there's some mystery around them. Linux load averages track not just runnable tasks, but also tasks in the uninterruptible sleep state. I've never seen an explanation. In this post I'll solve this mystery, and summarize load averages as a reference for everyone trying to interpret them.

Välkommen till Stockholm – An AWS Region is coming to the Nordics

All Things Distributed

Today, I am very excited to announce our plans to open a new AWS Region in the Nordics! The new region will give Nordic-based businesses, government organisations, non-profits, and global companies with customers in the Nordics, the ability to leverage the AWS technology infrastructure from data centers in Sweden. The new AWS EU (Stockholm) Region will have three Availability Zones and will be ready for customers to use in 2018. Over the past decade, we have seen tremendous growth at AWS.

Amazon DynamoDB Accelerator (DAX): Speed Up DynamoDB Response Times from Milliseconds to Microseconds without Application Rewrite.

All Things Distributed

Today, I'm excited to announce the general availability of Amazon DynamoDB Accelerator (DAX) , a fully managed, highly available, in-memory cache that can speed up DynamoDB response times from milliseconds to microseconds, even at millions of requests per second. You can add DAX to your existing DynamoDB applications with just a few clicks in the AWS Management Console – no application rewrites required.

Cache 92

Stop waiting for perfection and learn from your mistakes

All Things Distributed

This article titled " Wartet nicht auf Perfektion – lernt aus euren Fehlern! appeared in German last week in the "Digitaliserung" column of Wirtschaftwoche. "Man errs as long as he doth strive." " Goethe, the German prince of poets, knew that already more than 200 years ago. His words still ring true today, but with a crucial difference: Striving alone is not enough. You have to strive faster than the rest.