High Scalability

DECEMBER 14, 2018

Bill English : What we did was lease two video circuits from the phone company. That was our video link. davidfrum : In order to lose 80% of its value, the bitcoin network expended more electricity than 150 of the world's 195 countries. Cables of course going down into the room at both ends.

Internet 141

Internet Internet Scalability Serverless 141

The Netflix TechBlog

MAY 21, 2019

Netflix engineers run a series of tests and benchmarks to validate the device across multiple dimensions including compatibility of the device with the Netflix SDK, device performance, audio-video playback quality, license handling, encryption and security. The neural network (NN)-based agent uses a deep net with fully connected layers.

Testing 163

Testing AWS Lambda Network 163

High Scalability

OCTOBER 19, 2018

four petabytes : added to Internet Archive per year; 60,000 : patents donated by Microsoft to the Open Invention Network; 30 million : DuckDuckGo daily searches; 5 seconds : Google+ session length; 1 trillion : ARM device goal; $40B : Softbank investment in 5G; 30 : Happy Birthday IRC!; They'll love it and you'll be their hero forever.

Internet 123

Internet Internet Scalability Lambda 123

The Netflix TechBlog

OCTOBER 19, 2020

Investigating a video streaming failure consists of inspecting all aspects of a member account. Reconstructing a streaming session was a tedious and time consuming process that involved tracing all interactions (requests) between the Netflix app, our Content Delivery Network (CDN), and backend microservices.

Infrastructure 233

Infrastructure Transportation Storage Open Source 233

Dynatrace

OCTOBER 28, 2021

In 2009, the purveyor of online videos migrated to AWS cloud infrastructure to deliver its entertainment to a growing audience. A disruption spanning hardware, networks, and cloud infrastructure can require input and participation from network and infrastructure architects, risk experts, security teams, and even procurement officers.

Engineering 235

Engineering Entertainment Cloud Infrastructure 235

Brendan Gregg

JULY 4, 2021

AWS Graviton2); for memory with the arrival of DDR5 and High Bandwidth Memory (HBM) on-processor; for storage including new uses for 3D Xpoint as a 3D NAND accelerator; for networking with the rise of QUIC and eXpress Data Path (XDP); and so on.

Performance 140

Performance Latency Hardware Storage 140

High Scalability

SEPTEMBER 21, 2018

From this data, the team found that internet service providers are “giving a fixed amount of bandwidth—typically something in the range of one and a half megabits per second to four megabits per second—to video traffic, but they don’t impose these limits on other network traffic.”

Internet 126

Internet Internet Scalability Traffic 126

Brendan Gregg

DECEMBER 22, 2019

For my AWS re:Invent talk on BPF Performance Analysis at Netflix, I began with a demo of "BPF superpowers" (aka eBPF). The video is on [youtube] and the demo starts at [0:50] (the sildes are on [slideshare]): I'm demonstrating a tool I developed to turn my laptop's wifi signal strength into audio. Attaching 1 probe.

C++ 102

C++ Education Java AWS 102

All Things Distributed

JANUARY 19, 2011

AWS Elastic Beanstalk: A Quick and Simple Way into the Cloud. Additionally customers are not restricted to AWS services; they can mix-and-match services from other providers to best meet their needs. There are some excellent platforms running on AWS that do precisely this; Ruby on Rails developers have Heroku. Comments ().

AWS 115

AWS Cloud Java Operating System 115

All Things Distributed

FEBRUARY 4, 2014

AWS has built a reputation over the years for the breadth and depth of our services and the pace of our innovation with 280 features released in 2013. These are the people working behind the scenes helping customers fully leverage all of AWS’s capabilities when running their infrastructure on AWS.

AWS 132

AWS Energy Innovation Best Practices 132

Brendan Gregg

FEBRUARY 28, 2023

The video is now on [YouTube]: The slides are [online] and as a [PDF]: first prev next last / permalink/zoom In Q&A I was asked about CXL (compute express link) which was fortunate as I had planned to cover it and then forgot, so the question let me talk about it (although Q&A is missing from the video).

Performance 110

Performance Latency Cache Virtualization 110

O'Reilly

DECEMBER 2, 2019

The most obvious change 5G might bring about isn’t to cell phones but to local networks, whether at home or in the office. High-speed networks through 5G may represent the next generation of cord cutting. Those waits can be significant, even if you’re on a corporate network. Let’s get back to home networking.

Wireless 53

Wireless Serverless IoT Artificial Intelligence 53

Brendan Gregg

DECEMBER 21, 2019

For my AWS re:Invent talk on BPF Performance Analysis at Netflix, I began with a demo of "BPF superpowers" (aka eBPF). In the video I explained how I arrived at tracing __iwl_dbg() in this way, and how you can follow a similar approach for tracing unfamiliar code. strength: -46 strength: -46 strength: -45 strength: -45 [.]

C++ 52

C++ Education Java AWS 52

Brendan Gregg

DECEMBER 21, 2019

For my AWS re:Invent talk on BPF Performance Analysis at Netflix, I began with a demo of "BPF superpowers" (aka eBPF). The video is on [youtube] and the demo starts at [0:50] (the sildes are on [slideshare]): I'm demonstrating a tool I developed to turn my laptop's wifi signal strength into audio (someone described this as a BPF [theremin].)

C++ 52

C++ Education Java AWS 52

All Things Distributed

AUGUST 13, 2013

This year as I hosted AWS Summits in 12 different cities around the world, I met thousands of developers who are building powerful new applications for smartphones, tablets and other connected devices, all running mobile cloud backends on AWS. per million push notifications ($0.50 per million publishes and $0.50

Mobile 108

Mobile Development AWS Games 108

All Things Distributed

AUGUST 20, 2012

One factor in this growth is the presence of image, audio and video archives, which are often archived in their original un-edited form. In Amazon Glacier data is stored as archives, which are uploaded to Glacier and organized in vaults, which customers can control access to using the AWS Identity and Access Management (IAM) service.

Storage 122

Storage Cloud AWS Media 122

Brendan Gregg

JULY 4, 2021

AWS Graviton2); for memory with the arrival of DDR5 and High Bandwidth Memory (HBM) on-processor; for storage including new uses for 3D Xpoint as a 3D NAND accelerator; for networking with the rise of QUIC and eXpress Data Path (XDP); and so on.

Performance 52

Performance Latency Hardware Storage 52

Brendan Gregg

FEBRUARY 28, 2023

The video is now on YouTube : The slides are online and as a PDF : first prev next last / permalink/zoom In Q&A I was asked about CXL (compute express link) which was fortunate as I had planned to cover it and then forgot, so the question let me talk about it (although Q&A is missing from the video).

Performance 52

Performance Latency Cache Virtualization 52

The Netflix TechBlog

JUNE 14, 2023

We could also swap out the implementation of a field from GraphQL Shim to Video API with federation directives. The next phase in the migration was to reimplement our existing Falcor API in a GraphQL-first server (Video API Service). Watch our Chaos Engineering talk from AWS Reinvent to learn more about Sticky Canaries.

Traffic 353

Traffic Latency Cache Metrics 353

Brendan Gregg

NOVEMBER 29, 2017

At Netflix, we've been using these technologies as they've been made available for instance types in the AWS EC2 cloud. The latest AWS hypervisor, Nitro, uses everything to provide a new hardware-assisted hypervisor that is easy to use and has near bare-metal performance. AWS called this [enhanced networking]. Impressive!

Virtualization 111

Virtualization AWS Hardware Storage 111

Dynatrace

DECEMBER 22, 2019

AWS offers a broad set of global, cloud-based services including computing, storage, networking, Internet of Things (IoT), and many others. At Dynatrace, we’re constantly improving our AWS monitoring capabilities. Monitor and understand additional AWS services. Get up to 300 new AWS metrics out of the box.

AWS 137

AWS Metrics IoT Storage 137

Dynatrace

DECEMBER 22, 2019

AWS offers a broad set of global, cloud-based services including computing, storage, networking, Internet of Things (IoT), and many others. At Dynatrace, we’re constantly improving our AWS monitoring capabilities. Monitor and understand additional AWS services. Get up to 300 new AWS metrics out of the box.

AWS 134

AWS Metrics IoT Storage 134

The Netflix TechBlog

APRIL 29, 2019

We use Python through the full content lifecycle, from deciding which content to fund all the way to operating the CDN that serves the final video to 148 million members. Open Connect Open Connect is Netflix’s content delivery network (CDN). takes place in Amazon Web Services (AWS), whereas everything that happens afterwards (i.e.,

Open Source 279

Open Source Network Infrastructure Big Data 279

VoltDB

JUNE 1, 2017

It was – like the hypothetical movie I describe above – more than a little bit odd, as you could leave a session discussing ever more abstract layers of virtualization and walk into one where they emphasized the critical importance of pinning a network interface to a specific VM for optimal performance. Watch the Video.

Hardware 56

Hardware Latency Virtualization Cloud 56

The Netflix TechBlog

SEPTEMBER 24, 2021

After content ingestion, inspection and encoding, the packaging step encapsulates encoded video and audio in codec agnostic container formats and provides features such as audio video synchronization, random access and DRM protection. It is worth pointing out that cloud processing is always subject to variable network conditions.

Cloud 237

Cloud Media Storage Cache 237

Smashing Magazine

SEPTEMBER 3, 2021

It can be hard to visualize the huge network of hardware that allows you to send a request for a page to a server and then receive a response back. The good news is that, as developers, we can do an awful lot about it. Recommended reading : How Improving Website Performance Can Help Save The Planet. More after jump!

Website 135

Website Energy Internet Internet 135

Brendan Gregg

NOVEMBER 28, 2017

At Netflix, we've been using these technologies as they've been made available for instance types in the AWS EC2 cloud. The latest AWS hypervisor, Nitro, uses everything to provide a new hardware-assisted hypervisor that is easy to use and has near bare-metal performance. AWS called this [enhanced networking]. Impressive!

Virtualization 40

Virtualization AWS Hardware Storage 40

High Scalability

JUNE 29, 2018

kellabyte : LOL at racking up an AWS bill of $140,000 in 4 hours of compute time. kellabyte : Recently I got to work on a project that really stressed Amazon AWS scalability. We spun up a cluster of 100,000 AWS instances multiple times. Google applies bulk discounts after the fact, AWS makes you ask for them ahead of time.

Scalability 131

Scalability Internet Internet Wireless 131

The Netflix TechBlog

APRIL 26, 2022

Figure 1: Netflix ML Architecture Fact: A fact is data about our members or videos. An example of data about members is the video they had watched or added to their My List. An example of video data is video metadata, like the length of a video. Pay the storage, network, and compute cost.

Storage 187

Storage Design Scalability Latency 187

All Things Distributed

MARCH 11, 2016

The epoch of AWS is the launch of Amazon S3 on March 14, 2006, now almost 10 years ago. Given that AWS is a pioneer in building and operating these services world-wide, these lessons have been of crucial importance to our business. AWS helps its customers do this too. Build security in from the ground up.

AWS 144

AWS Hardware Retail Virtualization 144

Smashing Magazine

AUGUST 3, 2023

This could be an image, a block of text, or even an embedded video. You need to beware that slow server response times can significantly increase TTFB, often due to server overload, network issues, or un-optimized logic on the server side. Download Time of HTML This is the time it takes to download the page’s HTML file.

Performance 74

Performance Cache Traffic Metrics 74

The Morning Paper

MARCH 1, 2020

Firecracker is the virtual machine monitor (VMM) that powers AWS Lambda and AWS Fargate, and has been used in production at AWS since 2018. The first version of AWS Lambda was built using Linux containers. All of the existing approaches we just examined involve trade-offs that AWS didn’t want to make.

Serverless 101

Serverless Virtualization Lambda AWS 101

O'Reilly

MAY 19, 2020

Live online training, videos, books, certification prep, and more, from O’Reilly and our partner publishers. AWS is far and away the cloud leader, followed by Azure (at more than half of share) and Google Cloud. But most Azure and GCP users also use AWS; the reverse isn’t necessarily true. Amazon and AWS Ascendant.

Cloud 141

Cloud Serverless AWS Architecture 141

O'Reilly

JANUARY 17, 2023

There are various implementations–from the industrial-grade AWS SageMaker Autopilot and Google Cloud Vertex AI , to offerings from smaller players–but, in a nutshell, some developers spotted that same for() loop and built a slick UI on top. “What if we build a model to detect motion in the video feed? Your Job Has Changed.

Tuning 130

Tuning Open Source AWS Software 130

All Things Distributed

OCTOBER 2, 2017

As we began growing the AWS business, we realized that external customers might find our Dynamo database just as useful as we found it within Amazon.com. So, we set out to build a fully hosted AWS database service based upon the original Dynamo design.

Internet 128

Internet Internet AWS Performance 128

Smashing Magazine

JUNE 25, 2020

Also called “static assets,” these files include CSS and JavaScript, both written by the application developer and third-party libraries, as well as PDFs, software installers, images, music, videos, and icons. Static files range from a few kilobytes of CSS to gigabytes of video. These files are only used client-side.

Media 69

Media Cache Storage AWS 69

O'Reilly

NOVEMBER 28, 2023

How will AI adopters react when the cost of renting infrastructure from AWS, Microsoft, or Google rises? Several respondents also mentioned working with video: analyzing video data streams, video analytics, and generating or editing videos. But they may back off on AI development.

Open Source 108

Open Source Artificial Intelligence Google Infrastructure 108