Brendan Gregg

MAY 16, 2017

I've now worked at Netflix for over three years. Time flies! I previously wrote about Netflix in [2015] and [2016], and if you are interested in what it's like to work here, I already covered much in those posts. As before, no one at Netflix has asked me to write this, and this is my personal blog and not a company post. I'll start with some exciting news, describe what my job is really like, the culture and mission, and some work updates. ## 100 Million Subscribers!

Entertainment 61

Entertainment Java Engineering Scalability 61

SQL Server According to Bob

OCTOBER 26, 2017

Please check out these new resources for SQL Server 2017 and SQL Server on Linux and Docker: Brk2045 upgrade sql server 2017 (on prem, iaa-s and paas) from Bob Ward. With the 2017 PASS Summit approaching next week, I thought it would be helpful to provide for you my list of resources when it comes to SQL Server 2017, which we announced for General Availability on October 2nd. SQL Server 2017 Release Notes. bobsql blog posts on SQL Server 2017.

Azure 40

Azure Servers Database Open Source 40

SQL Server According to Bob

SEPTEMBER 25, 2017

Scott Guthrie and Rohan Kumar officially made the announcement today at Ignite that SQL Server 2017 will be generally available on October 2nd, 2017. Andrew Brust also wrote a very nice article detailing out all the great new features that come with SQL Server 2017. Stay tuned to hear more about SQL Server 2017 over the next several weeks both on our main SQL Server blog [link] and here at the bobsql blog: [link]. SQL Server 2017

Tuning 40

Tuning Availability Servers Database 40

Brendan Gregg

SEPTEMBER 5, 2017

Many people have contacted me recently about switching from Solaris (or illumos) to Linux, especially since most of the Solaris kernel team were let go this year (including my former colleagues, I'm sorry to hear). This includes many great engineers who I'm sure will excel in whatever they choose to work on next. They have been asking me about Linux because I've worked for years on each platform: Solaris, illumos, and Linux, in all cases full time and as a subject matter expert.

Virtualization 63

Virtualization AWS Network Entertainment 63

SQL Server According to Bob

SEPTEMBER 27, 2017

Please check out a new blog post I created detailing how SQL Server 2017 is Fast, Built-In. Look for more details on my blog post and the SQL Server Blog about some of these concepts and new features for performance for SQL Server 2017. I just presented on this topic at Ignite and will repeat this same session at the upcoming SQL 2017 PASS Summit. SQL Server 2017 Fast and the Furious

Tuning 40

Tuning Servers Database Cloud 40

Brendan Gregg

NOVEMBER 29, 2017

Xen AWS 2017 In 2015, AWS launched c4, which used hardware virtualization for EBS volumes. This finally came to storage-optimized instance types in 2017 with the [i3 instance type], which used SR-IOV and the nvme storage driver. AWS Nitro 2017. --> As was announced at AWS re:Invent last night, and covered in Anthony Liguori's talk today ([CMP332]: video ), and the bare metal talk ([CMP330]: video ), the [c5 instance type] uses a new hypervisor called Nitro.

Virtualization 63

Virtualization Hardware AWS Storage 63

Abstracta

OCTOBER 24, 2017

The post How to Optimize Website Speed for Black Friday 2017 appeared first on Abstracta Software Testing Services. Is your website prepared for record breaking traffic and sales? Halloween is almost here and before you know it, it will be Black Friday and then, Cyber Monday. These consumer “holidays” bring about the race for shoppers to buy all of the coveted items on. Performance Engineering Software Testing Black Friday load testing Page Speed Performance Performance Testing

Speed 43

Speed Website Traffic Engineering 43

Brendan Gregg

AUGUST 24, 2017

While all eyes and instruments were aimed at the total eclipse of August 21st, I had a video camera pointed in the opposite direction. I was hoping to catch changes in the landscape – perhaps the Moon's shadow as it raced by. This was filmed from on a road north east of Madras, Oregon, almost right on the center line. It was just an experiment that I wasn't going to share, but when I saw it afterwards I thought it might interest some.

Engineering 40

Engineering 40

J. Paul Reed

JANUARY 24, 2017

I mentioned last week I’d have 2017 predictions for you… well, last week. So, what do I think 2017 will have in store for us? Prediction #1: DevOps Will Be Declared 1.0-STABLE This prediction was originally published in a larger T echBeacon piece on 2017 DevOps predictions. But we’ll see this mechanism happen in 2017. So ultimately, in 2017, Release Engineering will need to (continue?)

IoT 40

IoT C++ Infrastructure Internet 40

The Symphonia

DECEMBER 6, 2017

Serverless at re:Invent 2017 was originally published in The Symphonium on Medium, where people are continuing the conversation by highlighting and responding to this story. The light show at the re:Play party Another re:Invent has come and gone, and us mere AWS-using mortals are now rapidly trying to sort the wheat from the chaff of a heady harvest of announcements. It’s funny to think that AWS Lambda was announced at re:Invent only 3 years ago?—?the

Serverless 40

Serverless Traffic Azure Virtualization 40

Alex Podelko

OCTOBER 3, 2017

This year it would be held on November 6-9, 2017 in New Orleans, LA. A lot of events are scheduled: Performance and Capacity Hackathon (which is $10 / free for the conference attendees), Performance Testing Workshop, CMG Training / Badge Program, and the CMG imPACt 2017 conference itself. I am very excited by the upcoming CMG imPACt performance and capacity conference.

IoT 109

IoT Programming Performance Performance Testing 109

Alex Podelko

AUGUST 24, 2017

There was a rather heated discussion around A Context-Driven Approach to Automation in Testing by James Bach and Michael Bolton (referred below as the article and the authors – other references would be explicit). Here are a couple of places where it got fiercely attacked Reviewing “Context Driven Approach to Automation in Testing” by Chris McMahon. and Open letter to “CDT Test Automation” reviewers as well as in many other different places.

Testing 127

Testing Performance Testing Tools Systems 127

The Symphonia

DECEMBER 19, 2017

Hello everyone, and welcome to the last Serverless Insights of 2017! Symphonia’s Serverless Insights — December 2017 was originally published in The Symphonium on Medium, where people are continuing the conversation by highlighting and responding to this story. This is the latest edition of our newsletter. To receive the email version please subscribe here.

Serverless 40

Serverless Lambda DevOps Availability 40

Brendan Gregg

DECEMBER 31, 2017

My last talk for 2017 was at AWS re:Invent, on "How Netflix Tunes EC2 Instances for Performance," an updated version of my [2014] talk. WARNING: These tunables were developed in late 2017, for Ubuntu Xenial instances on EC2.** There was so much demand for it this year that I had three overflow rooms streaming it, and people still couldn't get in. (I I shouldn't let this go to my head, as there were 42,000 attendees at re:Invent looking for something to see!)

Tuning 44

Tuning Best Practices AWS Games 44

Jos

JUNE 17, 2017

Image taken from the Chrome 60 blog post , which first appeared in “Web Performance: Leveraging the Metrics that Most Affect User Experience” at Google I/O 2017 Up until now we have been measuring performance through other metrics, mostly using the Navigation Timing API , which is also what Google Analytics uses for their Site Speed report.

Google 100

Google Metrics Analytics Monitoring 100

Alex Podelko

AUGUST 15, 2017

An interesting discussion started around a very good post Open Source Load Testing Tool Review by Ragnar Lönn. Somehow the latest comment, which I got in my e-mail, didn’t get published. So I decided to copy the discussion as is and publish the comment (and my answer) here. Check the original post (as well as many other great posts) following the link above – but the discussion below is somewhat independent.

Testing Tools 100

Testing Tools Testing Website Open Source 100

SQL Server According to Bob

DECEMBER 26, 2017

Here is an example using the SQL Server SSMS 2017 against my SQL 2016 server. SQL 2016 SQL Server 2017 XEventBob Ward and I worked with our SQL Server Tool developers (thanks David) to enable ‘Q uick XE Trace ’ capabilities. The feature is available in the latest SQL Server Management Studio (SSMS) release.

Servers 40

Servers Software Engineering Engineering Development 40

O'Reilly Software

DECEMBER 21, 2017

A process to build RESTful APIs that solve users’ needs with simplicity, reliability, and performance. Every piece of software exists to solve a real-world problem. Directly or indirectly. Most web APIs are consumed by client applications running on PCs, mobile devices, etc., which in turn are used by humans. Despite being consumed directly by machines, APIs are made to satisfy the needs of human beings, so designing them should follow a user-centered process, but often it doesn’t.

Architecture 72

Architecture Design Database Mobile 72

SQL Server According to Bob

DECEMBER 11, 2017

Current limitation: Engaging in a distributed transaction with SQL Server 2017 on Linux. SQL Server 2017 on Linux does not have distributed transaction support. However, SQL Server 2017 on Windows provides distributed transaction support allowing interactions with SQL Server 2019 on Linux. SQL Server 2017 ( Historical Information ). Linux SQL Server 2017 SQL Server on LinuxSQL Server 2019 CTP2.

Servers 40

Servers Database Software Engineering Engineering 40

All Things Distributed

SEPTEMBER 25, 2017

2017 continues a busy year for AWS in the Middle East. They are also migrating 700 government websites, with more than 50 TB of data, onto AWS, helping them to meet their goal of decommissioning their hosting platform by the end of 2017. Today, I am excited to announce plans for Amazon Web Services (AWS) to bring an infrastructure Region to the Middle East!

AWS 100

AWS Education Government Energy 100

Software Architecture

AUGUST 24, 2017

Memcached is in-memory key value store whereas Redis is in-memory data structures store. Memcached supports only string data type which is ideal for storing read-only data. Redis supports almost all types of data. Redis can also be used as a messaging system such as pubsub. Memcached cannot be used as a pubsub system. Memcached is more memory efficient than Redis. Redis is more memory efficient only after you use hashes. Memcached is volatile store and Redis is non-volatile.

Traffic 100

Traffic Software Architecture Efficiency Website 100

Software Architecture

AUGUST 24, 2017

For a quick introduction on what Apache Cassandra is, take a look here. Consistency is a significantly large topic to cover in one part. So I’ll be completing it in 3 parts. This first part defines consistency in general, write consistency, read consistency, consistency levels (CL), immediate, eventual and tunable consistency. Consistency The topic and concept of consistency is very important when you work with a distributed database like Cassandra.

Software Architecture 100

Software Architecture Tuning Servers Speed 100

Software Architecture

AUGUST 24, 2017

In part 1 , I introduced the basics of consistency in general, write consistency, read consistency, consistency levels (CL), immediate, eventual and tunable consistency. In this part 2, I will talk about how to achieve immediate and eventual consistency using different write and read consistency levels.

Tuning 100

Tuning Software Architecture C++ Architecture 100

Software Architecture

AUGUST 24, 2017

In part 2 , I explained how to achieve immediate and eventual consistency using different write and read consistency levels. In this part, I’ll go a bit deeper into understanding different configuration settings and consistency levels. Immediate Consistency with Write CL = ALL, Read CL = ONE The write request is sent to all replicas. Complete write operation is considered successful when all replicas respond a write success.

Software Architecture 100

Software Architecture Architecture Availability Software 100

All Things Distributed

OCTOBER 2, 2017

In support of Amazon Prime Day 2017, the biggest day in Amazon retail history, DynamoDB served over 12.9 Today marks the 10 year anniversary of Amazon's Dynamo whitepaper , a milestone that made me reflect on how much innovation has occurred in the area of databases over the last decade and a good reminder on why taking a customer obsessed approach to solving hard problems can have lasting impact beyond your original expectations.

Internet 97

Internet Internet AWS Database 97

O'Reilly Software

DECEMBER 28, 2017

Learn how to properly design RESTful APIs communication with clients, accounting for request structure, authentication, and caching. This series of articles shows you how to derive an easy-to-use, robust, efficient API to serve users on the web or on mobile devices. We are using the principles of RESTful architecture over HTTP. In the first piece, we started from a list of specs for a simple bike rental service, defining URLs and the HTTP methods to serve the app.

Cache 67

Cache Servers Mobile Efficiency 67

All Things Distributed

JUNE 30, 2017

So much so, that in March 2017, we announced Amazon Connect, which is the result of nearly ten years of work to build cloud-based contact centers at scale to power customer service for more than 50 Amazon teams and subsidiaries, including Amazon.com, Zappos, and Audible. Customer service is central to the overall customer experience that all consumers are familiar with when communicating with companies.

Artificial Intelligence 97

Artificial Intelligence Lambda AWS Virtualization 97

O'Reilly Software

NOVEMBER 2, 2017

How software architects can balance technical proficiencies with an appropriate mastery of communication. One of the demographics Brian and I noticed in the several O'Reilly Software Architecture Conferences we've hosted is the Accidental Architect : someone who makes architecture-level decisions on projects without a formal Architect title. Over time, we're building more material into the conference program to accommodate this common role.

Software Architecture 67

Software Architecture Architecture Programming Software 67

All Things Distributed

SEPTEMBER 4, 2017

This article titled " Wie Unternehmen vom Vormarsch des maschinellen Lernens profitieren können " appeared in German last week in the "Digitaliserung" column of Wirtschaftwoche. When a technology has its breakthrough, can often only be determined in hindsight. In the case of artificial intelligence (AI) and machine learning (ML), this is different. ML is that part of AI that describes rules and recognizes patterns from large amounts of data in order to predict future data.

Artificial Intelligence 98

Artificial Intelligence Transportation Education Airlines 98

Brendan Gregg

NOVEMBER 13, 2017

My colleague Justin Becker is also giving a talk at QConSF 2017 on the topic: [Am I a Brilliant Jerk?]. Houghton Mifflin Harcourt, 2017. - [link] - [link] - [link] - [link] - [link] - [link] - [link] - [link] - [link] - [link] - The blood bag series: part 1 , part 2 , part 3. - (Updated) discussion on hackernews. [Am Notice board at Ericsson, Stockholm (pic by DeirdreS ).

Engineering 87

Engineering Energy Open Source Technology 87

SQL Server According to Bob

MARCH 18, 2018

The changes made in SQL Server 2017 CU4 improve resource usage on smaller machines, such as a VM or Container used for testing and development. The change I made in SQL Server 2017 CU4 was to consider RAM as another factor in the default calculation. The change I made in SQL Server 2017 CU4 is to use Client garbage collection mode on smaller machines, removing the additional overhead of Server mode garbage collection. Linux SQL Server 2017

Servers 40

Servers Best Practices Systems Software Engineering 40

All Things Distributed

NOVEMBER 21, 2017

Amazon ElastiCache embodies much of what makes fast data a reality for customers looking to process high volume data at incredible rates, faster than traditional databases can manage. Developers love the performance, simplicity, and in-memory capabilities of Redis, making it among the most popular NoSQL key-value stores. Redis's microsecond latency has made it a de facto choice for caching.

Games 94

Games Education Retail Latency 94

All Things Distributed

JUNE 20, 2017

Today, I am very excited to announce our plans to open a new AWS Region in Hong Kong! The new region will give Hong Kong-based businesses, government organizations, non-profits, and global companies with customers in Hong Kong, the ability to leverage AWS technologies from data centers in Hong Kong. The new AWS Asia Pacific (Hong Kong) Region will have three Availability Zones and be ready for customers for use in 2018. Over the past decade, we have seen tremendous growth at AWS.

Logistics 100

Logistics Social Media AWS Entertainment 100

Professor Beekums

JANUARY 8, 2017

Being? ?a? software? ?developer? developer? ?and? and? ?being? being? ?a? system? ?administrator? administrator? ?are? are? ?very? very? ?different? different? ?things.? things.? ?Many Many folks? ?lump? lump? ?the? the? ?two? two? ?professions? professions? ?together,? together,? ?but? but? ?the? the? ?skillsets? skillsets? ?do? not? ?overlap? overlap? ?much.? much.? ?Software Software developers? ?write? write? ?code.? code.? ?System? System? ?administrators? administrators? ?maintain?

Development 72

Development Software Software Systems 72

Sutter's Mill

FEBRUARY 13, 2017

Next, I’ll be heading to Bristol to catch the end of the ACCU 2017 conference, and give the closing talk on “Something(s) New in C++.” Here is the current abstract: By the time the ACCU 2017 conference begins, C++17 is expected to be technically complete and in its final approval ballot.

Latency 62

Latency C++ Hardware Cache 62

Abstracta

NOVEMBER 19, 2017

Why continuous testing and open source are a perfect match I recently visited the offices of CA Technologies (one of Abstracta’s partners) in Santa Clara, where I had the chance to discuss shift-left testing, continuous testing, and why and how to turn to open source. The post Shift-Left Testing in the Enterprise and the Case for Open Source appeared first on Abstracta Software Testing Services.

Open Source 60

Open Source Testing Technology Technology 60

All Things Distributed

MAY 19, 2017

Amazon Aurora: Design Considerations for HighThroughput Cloud-Native Relational Databases ", Alexandre Verbitski, Anurag Gupta, Debanjan Saha, Murali Brahmadesam, Kamal Gupta, Raman Mittal, Sailesh Krishnamurthy, Sandor Maurice, Tengiz Kharatishvili, Xiaofeng Bao, in SIGMOD '17 Proceedings of the 2017 ACM International Conference on Management of Data, Pages 1041-1052 May 14 – 19, 2017, Chicago, IL, USA

Design 85

Design Storage Database AWS 85

Alex Russell

OCTOBER 22, 2017

— Alex Russell (@slightlylate) October 4, 2017. We now have everything we need to create a ballpark perf budget for a product in 2017. TL;DR: performance budgets are an essential but under-appreciated part of product success and team health. Most partners we work with are not aware of the real-world operating environment and make inappropriate technology choices as a result.

Benchmarking 81

Benchmarking Network Cache Transportation 81

All Things Distributed

APRIL 12, 2017

I am excited to announce that AWS is opening its first Mentor's Office at STATION F in Paris! The Mentor's Office is a workplace exclusively dedicated to meetings between AWS experts and the startups. STATION F is the world's biggest startup campus. With this special offer starting at the end of June, at the campus opening, AWS increases the support already available to startup customers in France.

Innovation 96

Innovation AWS Programming Cloud 96

Brendan Gregg

AUGUST 8, 2017

Load averages are an industry-critical metric – my company spends millions auto-scaling cloud instances based on them and other metrics – but on Linux there's some mystery around them. Linux load averages track not just runnable tasks, but also tasks in the uninterruptible sleep state. I've never seen an explanation. In this post I'll solve this mystery, and summarize load averages as a reference for everyone trying to interpret them.

Metrics 87

Metrics Latency Systems C++ 87

All Things Distributed

APRIL 4, 2017

Today, I am very excited to announce our plans to open a new AWS Region in the Nordics! The new region will give Nordic-based businesses, government organisations, non-profits, and global companies with customers in the Nordics, the ability to leverage the AWS technology infrastructure from data centers in Sweden. The new AWS EU (Stockholm) Region will have three Availability Zones and will be ready for customers to use in 2018. Over the past decade, we have seen tremendous growth at AWS.

Airlines 96

Airlines Games Latency IoT 96

All Things Distributed

JUNE 21, 2017

Today, I'm excited to announce the general availability of Amazon DynamoDB Accelerator (DAX) , a fully managed, highly available, in-memory cache that can speed up DynamoDB response times from milliseconds to microseconds, even at millions of requests per second. You can add DAX to your existing DynamoDB applications with just a few clicks in the AWS Management Console – no application rewrites required.

Cache 92

Cache Speed AWS Latency 92

All Things Distributed

JUNE 28, 2017

This article titled " Wartet nicht auf Perfektion – lernt aus euren Fehlern! appeared in German last week in the "Digitaliserung" column of Wirtschaftwoche. "Man errs as long as he doth strive." " Goethe, the German prince of poets, knew that already more than 200 years ago. His words still ring true today, but with a crucial difference: Striving alone is not enough. You have to strive faster than the rest.

Innovation 92

Innovation Website Servers Systems 92