High Scalability

JANUARY 11, 2022

Generating machine learning based personalized recommendations to discover new people, photos, videos, and stories relevant one’s interest. After that, the post gets added to the feed of all the followers in the columnar data storage. Users should be able to like and comment the posts. Out of Scope. High Level Design.

Design 334

Design Media Storage Logistics 334

The Netflix TechBlog

AUGUST 28, 2020

As the number of 4K titles in our catalog continues to grow and more devices support the premium features, we expect these video streams to have an increasing impact on our members and the network. We also show the corresponding full frame which helps to get a sense of how the cutout fits in the corresponding video frame.

Network 330

Network Storage Innovation Mobile 330

The Netflix TechBlog

APRIL 26, 2022

Figure 1: Netflix ML Architecture Fact: A fact is data about our members or videos. An example of data about members is the video they had watched or added to their My List. An example of video data is video metadata, like the length of a video. Time is a critical component of Axion?—?When

Storage 187

Storage Design Scalability Latency 187

All Things Distributed

AUGUST 20, 2012

Expanding the Cloud â?? Managing Cold Storage with Amazon Glacier. With the introduction of Amazon Glacier , IT organizations now have a solution that removes the headaches of digital archiving and provides extremely low cost storage. All Things Distributed. By Werner Vogels on 21 August 2012 12:01 AM. Comments ().

Storage 122

Storage Cloud AWS Media 122

O'Reilly

MAY 19, 2020

Live online training, videos, books, certification prep, and more, from O’Reilly and our partner publishers. O’Reilly Learning > We wanted to discover what our readers were doing with cloud, microservices, and other critical infrastructure and operations technologies. Role of survey respondents.

Cloud 141

Cloud Serverless AWS Architecture 141

The Netflix TechBlog

OCTOBER 19, 2020

Investigating a video streaming failure consists of inspecting all aspects of a member account. Reconstructing a streaming session was a tedious and time consuming process that involved tracing all interactions (requests) between the Netflix app, our Content Delivery Network (CDN), and backend microservices.

Infrastructure 233

Infrastructure Transportation Storage Open Source 233

All Things Distributed

DECEMBER 8, 2016

The AWS Cloud now operates in 40 Availability Zones within 15 geographic regions around the world, with seven more Availability Zones and three more regions coming online in China, France, and the U.K. To increase online readership, it worked with AWS Partner Network (APN) Partner ClearScale to develop a personal recommendation capability.

AWS 155

AWS Cloud Lambda Innovation 155

Dynatrace

DECEMBER 22, 2019

AWS offers a broad set of global, cloud-based services including computing, storage, networking, Internet of Things (IoT), and many others. Therefore, the ability to view a real-time map of your applications, services, and cloud resources is key to your success. Amazon Kinesis Video Streams. Dynatrace news.

AWS 137

AWS Metrics IoT Storage 137

Dynatrace

DECEMBER 22, 2019

AWS offers a broad set of global, cloud-based services including computing, storage, networking, Internet of Things (IoT), and many others. Therefore, the ability to view a real-time map of your applications, services, and cloud resources is key to your success. Amazon Kinesis Video Streams. Dynatrace news.

AWS 133

AWS Metrics IoT Storage 133

The Morning Paper

NOVEMBER 19, 2019

Local-first software: you own your data, in spite of the cloud Kleppmann et al., On the one-hand we have ‘cloud apps’ which make it easy to access our work from multiple devices and to collaborate online with others (e.g. The primary issue with cloud apps (the SaaS model) is ownership of the data. I’m all in :).

Cloud 83

Cloud Software Software Internet 83

Brendan Gregg

JULY 4, 2021

It's an exciting time for developments in computer performance, not just for the BPF technology (which I often [write about]) but also for processors with 3D stacking and cloud vendor CPUs (e.g., This was a chance to talk about other things I've been working on, such as the present and future of hardware performance. Ford, et al., “TCP

Performance 140

Performance Latency Hardware Storage 140

The Morning Paper

OCTOBER 4, 2020

The first 5G networks are now deployed and operational. The study is based on one of the world’s first commercial 5G network deployments (launched in April 2019), a 0.5 The 5G network is operating at 3.5GHz). Does 5G improve the end-user experience for applications (web browsing, and 4K+ video streaming)?

Energy 130

Energy Latency Performance Network 130

Brendan Gregg

FEBRUARY 28, 2023

The video is now on [YouTube]: The slides are [online] and as a [PDF]: first prev next last / permalink/zoom In Q&A I was asked about CXL (compute express link) which was fortunate as I had planned to cover it and then forgot, so the question let me talk about it (although Q&A is missing from the video). Ford, et al., “TCP

Performance 110

Performance Latency Cache Virtualization 110

Brendan Gregg

NOVEMBER 29, 2017

Hardware virtualization for cloud computing has come a long way, improving performance using technologies such as VT-x, SR-IOV, VT-d, NVMe, and APICv. At Netflix, we've been using these technologies as they've been made available for instance types in the AWS EC2 cloud. AWS called this [enhanced networking]. The first was c3.

Virtualization 111

Virtualization AWS Hardware Storage 111

O'Reilly

SEPTEMBER 19, 2023

Cloud computing? It progressed from “raw compute and storage” to “reimplementing key services in push-button fashion” to “becoming the backbone of AI work”—all under the umbrella of “renting time and storage on someone else’s computers.” So we can ride that wave of publicity.)

Hardware 97

Hardware Storage Big Data Blockchain 97

Smashing Magazine

NOVEMBER 15, 2021

Mosh is a remote terminal app (or mobile shell) for interactive SSH usage that includes several useful features for those who need to do terminal-based tasks over weak WiFi, cellular networks, or other less-reliable connections. Storage is in plain text, includes Git-based versioning, wiki-style linking, color themes, and lots more.

Development 134

Development Google Storage Open Source 134

Tech News Gather

JULY 19, 2023

Hubs The hub would be the primary data collector and manager for a network of remote monitoring smart devices. This action may prevent hackers from gaining access to the network and taking it over. However, there is a disadvantage to using wifi: it requires a network connection. It’s what makes your smart home smart.

Wireless 52

Wireless IoT Development Internet 52

All Things Distributed

AUGUST 20, 2015

Today, we are releasing a plugin that allows customers to use the Titan graph engine with Amazon DynamoDB as the backend storage layer. It opens up the possibility to enjoy the value that graph databases bring to relationship-centric use cases, without worrying about managing the underlying storage. The importance of relationships.

Database 134

Database Logistics Availability Social Media 134

Brendan Gregg

FEBRUARY 28, 2023

The video is now on YouTube : The slides are online and as a PDF : first prev next last / permalink/zoom In Q&A I was asked about CXL (compute express link) which was fortunate as I had planned to cover it and then forgot, so the question let me talk about it (although Q&A is missing from the video). Ford, et al., “TCP

Performance 52

Performance Latency Cache Virtualization 52

ACM Sigarch

DECEMBER 6, 2018

As we move beyond individual image and frame-based analysis, it is clear that video data is much richer and including temporal information (across frames) in addition to the spatial aspects (within a frame) adds additional attributes that should be considered in workload/architecture research.

Wireless 54

Wireless IoT Analytics Architecture 54

O'Reilly

DECEMBER 2, 2019

The most obvious change 5G might bring about isn’t to cell phones but to local networks, whether at home or in the office. High-speed networks through 5G may represent the next generation of cord cutting. Those waits can be significant, even if you’re on a corporate network. Let’s get back to home networking.

Wireless 53

Wireless Serverless IoT Artificial Intelligence 53

Brendan Gregg

JULY 4, 2021

It's an exciting time for developments in computer performance, not just for the BPF technology (which I often [write about]) but also for processors with 3D stacking and cloud vendor CPUs (e.g., Ford, et al., “TCP on Upcoming Sapphire Rapids CPUs,” [link] Oct 2020 - [Liu 20] Linda Liu, “Samsung QVO vs EVO vs PRO: What’s the Difference?

Performance 52

Performance Latency Hardware Storage 52

Smashing Magazine

JANUARY 11, 2021

Assets Optimizations Brotli, AVIF, WebP, responsive images, AV1, adaptive media loding, video compression, web fonts, Google fonts. Networking, HTTP/2, HTTP/3 OCSP stapling, EV/DV certificates, packaging, IPv6, QUIC, HTTP/3. You can then mimic the 90th percentile’s experience for testing.

Performance 145

Performance Cache Media Metrics 145

All Things Distributed

OCTOBER 2, 2017

" Of course, no technology change happens in isolation, and at the same time NoSQL was evolving, so was cloud computing. The requirements for a fully hosted cloud database service needed to be at an even higher bar than what we had set for our Amazon internal system.

Internet 128

Internet Internet AWS Performance 128

Smashing Magazine

JUNE 25, 2020

We’ll also discuss the costs and benefits of CDNs and dedicated file storage solutions. Also called “static assets,” these files include CSS and JavaScript, both written by the application developer and third-party libraries, as well as PDFs, software installers, images, music, videos, and icons. Media Files. Media Files.

Media 69

Media Cache Storage AWS 69

All Things Distributed

JANUARY 19, 2011

AWS Elastic Beanstalk: A Quick and Simple Way into the Cloud. Elastic Beanstalk makes it easy for developers to deploy and manage scalable and fault-tolerant applications on the AWS cloud. This is exactly where Elastic Beanstalk will help: to make it even simpler to get started and to run applications in the AWS cloud.

AWS 115

AWS Cloud Java Scalability 115

Brendan Gregg

DECEMBER 31, 2017

Fortunately, it was videoed for those who missed it. A video of the talk is on youtube : The slides are on slideshare : I love this talk as I get to share more about what the Performance and Operating Systems team at Netflix does, rather than just my work. Storage I/O. Networking. File System. tcp_tw_reuse = 1 net.ipv4.ip_local_port_range

Tuning 61

Tuning AWS Best Practices Network 61

Brendan Gregg

DECEMBER 30, 2017

Fortunately, it was videoed for those who missed it. A video of the talk is on youtube : The slides are on slideshare : I love this talk as I get to share more about what the Performance and Operating Systems team at Netflix does, rather than just my work. Storage I/O. Networking. File System. tcp_tw_reuse = 1 net.ipv4.ip_local_port_range

Tuning 52

Tuning AWS Best Practices Network 52

All Things Distributed

MARCH 11, 2016

This becomes an even more important lesson at scale: for example, as S3 processes trillions and trillions of storage transactions, anything that has even the slightest probability of error will become realistic. If customers have many tiny files, then storage and bandwidth don’t amount to much even if they are making millions of requests.

AWS 144

AWS Hardware Retail Virtualization 144

High Scalability

NOVEMBER 14, 2018

New topics range from additional workloads like video streaming, machine learning, and public cloud to specialized silicon accelerators, storage and network building blocks, and a revised discussion of data center power and cooling, and uptime.

Design 54

Design Education Google Internet 54

Brendan Gregg

NOVEMBER 28, 2017

Hardware virtualization for cloud computing has come a long way, improving performance using technologies such as VT-x, SR-IOV, VT-d, NVMe, and APICv. At Netflix, we've been using these technologies as they've been made available for instance types in the AWS EC2 cloud. AWS called this [enhanced networking]. The first was c3.

Virtualization 40

Virtualization AWS Hardware Storage 40

High Scalability

NOVEMBER 25, 2019

It is limited by the disk space; it can’t expand storage elastically; it chokes if you run few I/O intensive processes or try collaborating with 100 other users. Our only option was to roll up our sleeves and build basic cloud file system components such as object store ourselves. Video Transcoding. Migration to cloud.

Architecture 52

Architecture Cache Azure Storage 52

All Things Distributed

OCTOBER 12, 2016

There are some great examples and use cases for Redis, which you can see at companies like Hudl, which offers mobile and desktop video analytics solutions to sports teams and athletes. Fast data can have a transformational impact on data-driven development that is underlying a lot of the interesting things happening in the Cloud today.

Scalability 130

Scalability Analytics Cache AWS 130

Dotcom-Montior

MAY 12, 2020

Websites are now more than just the storage and retrieval of information to present content to users. Network or connection error. Network latency. Network Latency. Network latency can be affected due to. Network equipment like cable or router. The list goes on and on. DNS resolving error. HTTP error.

Monitoring 63

Monitoring Entertainment Hardware Latency 63

Smashing Magazine

JANUARY 6, 2020

To get a good first impression of how your competitors perform, you can use Chrome UX Report ( CrUX , a ready-made RUM data set, video introduction by Ilya Grigorik and detailed guide by Rick Viscomi) or Treo Sites , a RUM monitoring tool that is powered by Chrome UX Report. You can then mimic the 90th percentile’s experience for testing.

Performance 145

Performance Cache Network Metrics 145

Dotcom-Montior

MARCH 6, 2022

Users can also watch the actual URL loading video in the browser window. It’s important to mention that within the Dotcom Monitor platform there is a feature that a user can watch the actual URL loading video in the browser window (shown below). Waiting time. The time captured by the server to produce a response.

Performance 52

Performance Cache Website Speed 52

The Morning Paper

SEPTEMBER 10, 2019

to understand YouTube video performance) drive tens of thousands of canned (known in advance) queries per second, that need to be served with latency in the tens of milliseconds. Procella achieves high scalability and efficiency by segregating storage (in Colossus) from compute (on Borg). Reporting and dashboarding use cases (e.g.

Analytics 75

Analytics Latency Cache Google 75