All Things Distributed

AUGUST 20, 2012

Expanding the Cloud â?? Managing Cold Storage with Amazon Glacier. With the introduction of Amazon Glacier , IT organizations now have a solution that removes the headaches of digital archiving and provides extremely low cost storage. All Things Distributed. By Werner Vogels on 21 August 2012 12:01 AM. Comments ().

Storage 122

Storage Cloud AWS Media 122

All Things Distributed

MARCH 11, 2015

Cloud computing is enabling amazing new innovations both in consumer and enterprise products, as it became the new normal for organizations of all sizes. So many exciting new areas are being empowered by cloud that it is fascinating to watch. Cloud analytics are everywhere. Cloud enables self-service analytics.

Analytics 135

Analytics Cloud Healthcare Retail 135

The Netflix TechBlog

OCTOBER 19, 2020

Investigating a video streaming failure consists of inspecting all aspects of a member account. Our distributed tracing infrastructure is grouped into three sections: tracer library instrumentation, stream processing, and storage. Storage: don’t break the bank! which is difficult when troubleshooting distributed systems.

Infrastructure 233

Infrastructure Transportation Storage Open Source 233

Brendan Gregg

JULY 4, 2021

It's an exciting time for developments in computer performance, not just for the BPF technology (which I often [write about]) but also for processors with 3D stacking and cloud vendor CPUs (e.g., This was a chance to talk about other things I've been working on, such as the present and future of hardware performance. Ford, et al., “TCP

Performance 140

Performance Latency Hardware Storage 140

All Things Distributed

JANUARY 19, 2011

AWS Elastic Beanstalk: A Quick and Simple Way into the Cloud. Additionally customers are not restricted to AWS services; they can mix-and-match services from other providers to best meet their needs. There are some excellent platforms running on AWS that do precisely this; Ruby on Rails developers have Heroku.

AWS 115

AWS Cloud Java Operating System 115

Brendan Gregg

DECEMBER 31, 2017

My last talk for 2017 was at AWS re:Invent, on "How Netflix Tunes EC2 Instances for Performance," an updated version of my [2014] talk. Fortunately, it was videoed for those who missed it. Storage I/O. What I was most excited for was the launch of a new EC2 hypervisor, which I referred to in the video as the "c5 hypervisor".

Tuning 61

Tuning AWS Best Practices Network 61

Brendan Gregg

DECEMBER 30, 2017

My last talk for 2017 was at AWS re:Invent, on "How Netflix Tunes EC2 Instances for Performance," an updated version of my [2014] talk. Fortunately, it was videoed for those who missed it. Storage I/O. What I was most excited for was the launch of a new EC2 hypervisor, which I referred to in the video as the "c5 hypervisor".

Tuning 52

Tuning AWS Best Practices Network 52

Brendan Gregg

NOVEMBER 28, 2017

Hardware virtualization for cloud computing has come a long way, improving performance using technologies such as VT-x, SR-IOV, VT-d, NVMe, and APICv. At Netflix, we've been using these technologies as they've been made available for instance types in the AWS EC2 cloud. AWS called this [enhanced networking].

Virtualization 40

Virtualization AWS Hardware Storage 40

Brendan Gregg

FEBRUARY 28, 2023

The video is now on [YouTube]: The slides are [online] and as a [PDF]: first prev next last / permalink/zoom In Q&A I was asked about CXL (compute express link) which was fortunate as I had planned to cover it and then forgot, so the question let me talk about it (although Q&A is missing from the video). Ford, et al., “TCP

Performance 110

Performance Latency Cache Virtualization 110

All Things Distributed

FEBRUARY 11, 2013

Expanding the Cloud: The Amazon Elastic Transcoder. While I was returning from an exciting time in New Orleans watching the Super Bowl, AWS launched a very cool, brand new service: Amazon Elastic Transcoder. Amazon Elastic Transcoder is designed to be very easy to use, scalable and cost-effective video transcoding in the cloud.

AWS 102

AWS Media Scalability Storage 102

Brendan Gregg

JULY 4, 2021

It's an exciting time for developments in computer performance, not just for the BPF technology (which I often [write about]) but also for processors with 3D stacking and cloud vendor CPUs (e.g., Ford, et al., “TCP on Upcoming Sapphire Rapids CPUs,” [link] Oct 2020 - [Liu 20] Linda Liu, “Samsung QVO vs EVO vs PRO: What’s the Difference?

Performance 52

Performance Latency Hardware Storage 52

Brendan Gregg

FEBRUARY 28, 2023

The video is now on YouTube : The slides are online and as a PDF : first prev next last / permalink/zoom In Q&A I was asked about CXL (compute express link) which was fortunate as I had planned to cover it and then forgot, so the question let me talk about it (although Q&A is missing from the video). Ford, et al., “TCP

Performance 52

Performance Latency Cache Virtualization 52

Smashing Magazine

JUNE 25, 2020

We’ll also discuss the costs and benefits of CDNs and dedicated file storage solutions. Also called “static assets,” these files include CSS and JavaScript, both written by the application developer and third-party libraries, as well as PDFs, software installers, images, music, videos, and icons. Media Files.

Media 69

Media Cache Storage AWS 69

All Things Distributed

AUGUST 18, 2011

The Amazon Elastic MapReduce (EMR) team announced today the ability to seamlessly use Amazon EC2 Spot Instances with their service, significantly driving down the cost of data analytics in the cloud. Many of our Big-Data customers already saw a big drop in their AWS bill last month when the cost of incoming bandwidth was dropped to $0.00.

Big Data 111

Big Data Analytics AWS Cloud 111

The Netflix TechBlog

SEPTEMBER 24, 2021

After content ingestion, inspection and encoding, the packaging step encapsulates encoded video and audio in codec agnostic container formats and provides features such as audio video synchronization, random access and DRM protection. Packaging has always been an important step in media processing. is 220 Mbps.

Cloud 237

Cloud Media Storage Cache 237

O'Reilly

DECEMBER 2, 2019

I don’t need more bandwidth for video conferences or movies, but I would like to be able to download operating system updates and other large items in seconds rather than minutes. What would the world look like if all of our storage was in the cloud, and access to that storage was so fast we didn’t care?

Wireless 53

Wireless Serverless IoT Artificial Intelligence 53

Tech News Gather

JUNE 16, 2023

Database management: Choose a database management system that can handle large amounts of data and ensure efficient retrieval and storage. Cloud infrastructure and hosting: Consider using cloud platforms like Amazon Web Services (AWS) or Google Cloud Platform (GCP) for hosting and scaling your UberEats clone.

Social Media 52

Social Media Tuning Strategy Media 52

Smashing Magazine

JANUARY 11, 2021

Assets Optimizations Brotli, AVIF, WebP, responsive images, AV1, adaptive media loding, video compression, web fonts, Google fonts. Paddy Ganti’s script constructs two URLs (one normal and one blocking the ads), prompts the generation of a video comparison via WebPageTest and reports a delta.

Performance 145

Performance Cache Media Metrics 145

O'Reilly

MAY 19, 2020

Live online training, videos, books, certification prep, and more, from O’Reilly and our partner publishers. O’Reilly Learning > We wanted to discover what our readers were doing with cloud, microservices, and other critical infrastructure and operations technologies. Respondent Demographics.

Cloud 141

Cloud Serverless AWS Architecture 141

Brendan Gregg

SEPTEMBER 5, 2017

. ## Observability Here's the big picture of performance observability tools on Linux, from my [Linux performance] page, where I also have diagrams for other tool types, as well as videos and slides of prior Linux performance talks: I also have a USE Method: Linux Performance Checklist , as a different way to navigate and apply the tools.

Virtualization 76

Virtualization AWS Engineering Hardware 76

Smashing Magazine

JANUARY 6, 2020

To get a good first impression of how your competitors perform, you can use Chrome UX Report ( CrUX , a ready-made RUM data set, video introduction by Ilya Grigorik and detailed guide by Rick Viscomi) or Treo Sites , a RUM monitoring tool that is powered by Chrome UX Report. Are videos properly optimized? Large preview ).

Performance 145

Performance Cache Network Metrics 145

The Netflix TechBlog

DECEMBER 3, 2019

Netflix applies data science to hundreds of use cases across the company, including optimizing content delivery and video encoding. Metaflow is a cloud-native framework. It leverages elasticity of the cloud by design?—?both both for compute and storage. Get started at metaflow.org.

Open Source 232

Open Source AWS Infrastructure Energy 232

The Netflix TechBlog

APRIL 26, 2022

Figure 1: Netflix ML Architecture Fact: A fact is data about our members or videos. An example of data about members is the video they had watched or added to their My List. An example of video data is video metadata, like the length of a video. Time is a critical component of Axion?—?When

Storage 187

Storage Design Scalability Latency 187

Smashing Magazine

JANUARY 7, 2019

To get a good first impression of how your competitors perform, you can use Chrome UX Report ( CrUX , a ready-made RUM data set, video introduction by Ilya Grigorik), Speed Scorecard (also provides a revenue impact estimator), Real User Experience Test Comparison or SiteSpeed CI (based on synthetic testing). Are videos properly optimized?

Performance 104

Performance Cache Metrics Network 104

The Netflix TechBlog

DECEMBER 3, 2019

Netflix applies data science to hundreds of use cases across the company, including optimizing content delivery and video encoding. Metaflow is a cloud-native framework. It leverages elasticity of the cloud by design?—?both both for compute and storage. Get started at metaflow.org.

Open Source 134

Open Source AWS Infrastructure Energy 134

All Things Distributed

MARCH 11, 2016

The epoch of AWS is the launch of Amazon S3 on March 14, 2006, now almost 10 years ago. Given that AWS is a pioneer in building and operating these services world-wide, these lessons have been of crucial importance to our business. AWS helps its customers do this too.

AWS 144

AWS Hardware Retail Virtualization 144

All Things Distributed

OCTOBER 2, 2017

" Of course, no technology change happens in isolation, and at the same time NoSQL was evolving, so was cloud computing. As we began growing the AWS business, we realized that external customers might find our Dynamo database just as useful as we found it within Amazon.com.

Internet 128

Internet Internet AWS Performance 128

All Things Distributed

OCTOBER 12, 2016

Three years ago, as part of our AWS Fast Data journey we introduced Amazon ElastiCache for Redis , a fully managed in-memory data store that operates at sub-millisecond latency. Fast data can have a transformational impact on data-driven development that is underlying a lot of the interesting things happening in the Cloud today.

Scalability 130

Scalability Analytics Cache AWS 130

All Things Distributed

AUGUST 20, 2015

Today, we are releasing a plugin that allows customers to use the Titan graph engine with Amazon DynamoDB as the backend storage layer. It opens up the possibility to enjoy the value that graph databases bring to relationship-centric use cases, without worrying about managing the underlying storage. The importance of relationships.

Database 134

Database Logistics Availability Social Media 134

High Scalability

MARCH 15, 2019

Know anyone who needs cloud? I wrote Explain the Cloud Like I'm 10 just for them. 300% : AWS IoT growth per year; 74% : mobile games user spending in the App store; 31.4 trillion : new record for calculating digits of pi (121 days); 112Gbps : Intel's SerDes; 100M : image and video dataset; 1.5

Internet 34

Internet Internet Scalability IoT 34

All Things Distributed

DECEMBER 9, 2010

However, as use of the cloud as grown, so have the file sizes customers want to store in Amazon S3 as objects. This is especially true for customers managing HD video or data-intensive instruments such as genomic sequencers. Driving Storage Costs Down for AWS Customers. Expanding the Cloud - The AWS Storage Gateway.

AWS 80

AWS Big Data Scalability Storage 80

The Netflix TechBlog

MARCH 6, 2019

Mounting object storage in Netflix’s media processing platform By Barak Alon (on behalf of Netflix’s Media Cloud Engineering team) MezzFS (short for “Mezzanine File System”) is a tool we’ve developed at Netflix that mounts cloud objects as local files via FUSE. What problem are we solving? Encoding is not a one-time process?—?large

Media 214

Media Storage Processing Cache 214

All Things Distributed

APRIL 19, 2010

As Dan Rayburn wrote on his Business of Video blog, "Amazon will be in the drivers seat to own the market for small and medium sized content owners who need simple delivery at a great price.". Driving Storage Costs Down for AWS Customers. Expanding the Cloud - The AWS Storage Gateway. Expanding the Cloud â??

Media 60

Media AWS Storage Big Data 60

Smashing Magazine

NOVEMBER 15, 2021

Fig adds VSCode-style autocomplete to your existing terminal and includes support for existing CLI tools like Git, npm, Kubernetes, Docker, AWS, Google Cloud, and more. Storage is in plain text, includes Git-based versioning, wiki-style linking, color themes, and lots more. Large preview ). Large preview ). Large preview ).

Development 134

Development Google Storage Open Source 134

High Scalability

NOVEMBER 25, 2019

It is limited by the disk space; it can’t expand storage elastically; it chokes if you run few I/O intensive processes or try collaborating with 100 other users. Egnyte is a secure Content Collaboration and Data Governance platform, founded in 2007 when Google drive wasn't born and AWS S3 was cost-prohibitive. Video Transcoding.

Architecture 52

Architecture Cache Azure Storage 52