Dynatrace

DECEMBER 8, 2022

As cloud-native, distributed architectures proliferate, the need for DevOps technologies and DevOps platform engineers has increased as well. It coincides with the advent of complex, distributed cloud technologies such as microservices and containers. Amazon Web Services (AWS). ” What does a DevOps platform engineer do?

DevOps 200

DevOps Engineering Testing Open Source 200

O'Reilly

MAY 19, 2020

Live online training, videos, books, certification prep, and more, from O’Reilly and our partner publishers. O’Reilly Learning > We wanted to discover what our readers were doing with cloud, microservices, and other critical infrastructure and operations technologies. Respondent Demographics.

Cloud 141

Cloud Serverless AWS Architecture 141

Brendan Gregg

NOVEMBER 29, 2017

Hardware virtualization for cloud computing has come a long way, improving performance using technologies such as VT-x, SR-IOV, VT-d, NVMe, and APICv. At Netflix, we've been using these technologies as they've been made available for instance types in the AWS EC2 cloud. AWS called this [enhanced networking].

Virtualization 111

Virtualization AWS Hardware Storage 111

Adrian Cockcroft

NOVEMBER 30, 2020

I’m helping manage AWS contributions to the project, as we build an open source data lake and analysis service that can be used to model climate related asset risks for investors. AWS is supporting customers who are working to change the way they operate and build more sustainable products. Orange mid-day sky at San Gregorio Beach?—?smoky

AWS 52

AWS IoT Open Source Government 52

All Things Distributed

MARCH 11, 2015

Cloud computing is enabling amazing new innovations both in consumer and enterprise products, as it became the new normal for organizations of all sizes. So many exciting new areas are being empowered by cloud that it is fascinating to watch. Cloud analytics are everywhere. Cloud enables self-service analytics.

Analytics 135

Analytics Cloud Healthcare Retail 135

The Netflix TechBlog

MARCH 6, 2019

Mounting object storage in Netflix’s media processing platform By Barak Alon (on behalf of Netflix’s Media Cloud Engineering team) MezzFS (short for “Mezzanine File System”) is a tool we’ve developed at Netflix that mounts cloud objects as local files via FUSE. Encoding is not a one-time process?—?large We have one file?

Media 214

Media Storage Processing Cache 214

Dynatrace

OCTOBER 28, 2021

But with the complexity that comes with digital transformation and cloud-native architecture, teams need a way to make sure applications can withstand the “chaos” of production. Chaos engineering answers this need so organizations can deliver robust, resilient cloud-native applications that can stand up under any conditions.

Engineering 234

Engineering Entertainment Cloud Infrastructure 234

The Netflix TechBlog

APRIL 26, 2022

Figure 1: Netflix ML Architecture Fact: A fact is data about our members or videos. An example of data about members is the video they had watched or added to their My List. An example of video data is video metadata, like the length of a video. Time is a critical component of Axion?—?When

Storage 187

Storage Design Scalability Latency 187

The Netflix TechBlog

APRIL 29, 2019

We use Python through the full content lifecycle, from deciding which content to fund all the way to operating the CDN that serves the final video to 148 million members. takes place in Amazon Web Services (AWS), whereas everything that happens afterwards (i.e., video streaming) takes place in the Open Connect network.

Open Source 279

Open Source Network Infrastructure Big Data 279

Brendan Gregg

DECEMBER 31, 2017

My last talk for 2017 was at AWS re:Invent, on "How Netflix Tunes EC2 Instances for Performance," an updated version of my [2014] talk. Fortunately, it was videoed for those who missed it. What I was most excited for was the launch of a new EC2 hypervisor, which I referred to in the video as the "c5 hypervisor".

Tuning 61

Tuning AWS Best Practices Network 61

Brendan Gregg

DECEMBER 30, 2017

My last talk for 2017 was at AWS re:Invent, on "How Netflix Tunes EC2 Instances for Performance," an updated version of my [2014] talk. Fortunately, it was videoed for those who missed it. What I was most excited for was the launch of a new EC2 hypervisor, which I referred to in the video as the "c5 hypervisor".

Tuning 52

Tuning AWS Best Practices Network 52

High Scalability

JUNE 29, 2018

And if you know anyone looking for a simple book that uses lots of pictures and lots of examples to explain the cloud, then please recommend my new book: Explain the Cloud Like I'm 10. kellabyte : LOL at racking up an AWS bill of $140,000 in 4 hours of compute time. We spun up a cluster of 100,000 AWS instances multiple times.

Scalability 131

Scalability Internet Internet Wireless 131

Brendan Gregg

NOVEMBER 28, 2017

Hardware virtualization for cloud computing has come a long way, improving performance using technologies such as VT-x, SR-IOV, VT-d, NVMe, and APICv. At Netflix, we've been using these technologies as they've been made available for instance types in the AWS EC2 cloud. AWS called this [enhanced networking].

Virtualization 40

Virtualization AWS Hardware Storage 40

Brendan Gregg

FEBRUARY 28, 2023

The video is now on [YouTube]: The slides are [online] and as a [PDF]: first prev next last / permalink/zoom In Q&A I was asked about CXL (compute express link) which was fortunate as I had planned to cover it and then forgot, so the question let me talk about it (although Q&A is missing from the video). Ford, et al., “TCP

Performance 110

Performance Latency Cache Virtualization 110

The Morning Paper

MARCH 1, 2020

Firecracker is the virtual machine monitor (VMM) that powers AWS Lambda and AWS Fargate, and has been used in production at AWS since 2018. The first version of AWS Lambda was built using Linux containers. All of the existing approaches we just examined involve trade-offs that AWS didn’t want to make.

Serverless 101

Serverless Virtualization Lambda AWS 101

The Netflix TechBlog

OCTOBER 19, 2020

Investigating a video streaming failure consists of inspecting all aspects of a member account. We earned the trust of our engineers by developing empathy for their operational burden and by focusing on providing efficient tracer library integrations in runtime environments. which is difficult when troubleshooting distributed systems.

Infrastructure 233

Infrastructure Transportation Storage Open Source 233

The Netflix TechBlog

OCTOBER 18, 2019

In this talk, Kinjal used the example of the LinkedIn Feed, to demonstrate how they use bandit algorithms to solve for the optimal parameter selection problem efficiently. He concluded by stressing the efficiency their teams had achieved by doing online parameter exploration instead of the much slower human-in-the-loop manual explorations.

Infrastructure 126

Infrastructure Metrics Architecture Efficiency 126

All Things Distributed

OCTOBER 12, 2016

Three years ago, as part of our AWS Fast Data journey we introduced Amazon ElastiCache for Redis , a fully managed in-memory data store that operates at sub-millisecond latency. ElastiCache for Redis Multi-AZ capability is built to handle any failover case for Redis Cluster with robustness and efficiency. Building upon Redis.

Scalability 130

Scalability Analytics Cache AWS 130

Smashing Magazine

SEPTEMBER 3, 2021

When you can’t see a process in action, the whole thing can feel a little bit like magic — something that isn’t helped by the insistence of certain companies on adding words like “cloud” and “serverless” to their product names. The good news is that, as developers, we can do an awful lot about it. More after jump! KG per hour.

Website 135

Website Energy Internet Internet 135

All Things Distributed

AUGUST 18, 2011

The Amazon Elastic MapReduce (EMR) team announced today the ability to seamlessly use Amazon EC2 Spot Instances with their service, significantly driving down the cost of data analytics in the cloud. Many of our Big-Data customers already saw a big drop in their AWS bill last month when the cost of incoming bandwidth was dropped to $0.00.

Big Data 111

Big Data Analytics AWS Cloud 111

Brendan Gregg

FEBRUARY 28, 2023

The video is now on YouTube : The slides are online and as a PDF : first prev next last / permalink/zoom In Q&A I was asked about CXL (compute express link) which was fortunate as I had planned to cover it and then forgot, so the question let me talk about it (although Q&A is missing from the video). Ford, et al., “TCP

Performance 52

Performance Latency Cache Virtualization 52

The Netflix TechBlog

OCTOBER 18, 2019

In this talk, Kinjal used the example of the LinkedIn Feed, to demonstrate how they use bandit algorithms to solve for the optimal parameter selection problem efficiently. He concluded by stressing the efficiency their teams had achieved by doing online parameter exploration instead of the much slower human-in-the-loop manual explorations.

Infrastructure 100

Infrastructure Metrics Architecture Efficiency 100

All Things Distributed

MARCH 11, 2016

The epoch of AWS is the launch of Amazon S3 on March 14, 2006, now almost 10 years ago. Given that AWS is a pioneer in building and operating these services world-wide, these lessons have been of crucial importance to our business. AWS helps its customers do this too. Build security in from the ground up.

AWS 144

AWS Hardware Retail Virtualization 144

All Things Distributed

AUGUST 20, 2015

Developers need efficient methods to store, traverse, and query these relationships. Social media apps navigate relationships between friends, photos, videos, pages, and followers. Like many AWS innovations, the desire to build a solution for a scalable graph database came from Amazon’s retail business.

Database 134

Database Logistics Availability Social Media 134

Tech News Gather

JUNE 16, 2023

By leveraging the power of mobile technology, UberEats provides a seamless and efficient way for users to order their favorite meals from a wide range of eateries. UberEats also excels in providing a reliable and efficient delivery service. Once an order is placed, it is seamlessly transmitted to the restaurant, where it is prepared.

Social Media 52

Social Media Tuning Strategy Media 52

O'Reilly

NOVEMBER 28, 2023

Even with cloud-based foundation models like GPT-4, which eliminate the need to develop your own model or provide your own infrastructure, fine-tuning a model for any particular use case is still a major undertaking. This is an area where cloud providers already bear much of the burden, and will continue to bear it in the future.

Open Source 106

Open Source Artificial Intelligence Google Infrastructure 106

VoltDB

JUNE 1, 2017

Private Clouds made of commodity hardware are perceived as the logical solution to this problem. In this case, what’s needed from a private cloud is lots of lightweight copies of what a production environment looks like, with heavy sharing of physical hardware. The “Public Private Cloud” folks. Where VoltDB fits. Very Well!”

Hardware 56

Hardware Latency Virtualization Cloud 56

All Things Distributed

DECEMBER 12, 2013

We launched DynamoDB last year to address the need for a cloud database that provides seamless scalability, irrespective of whether you are doing ten transactions or ten million transactions, while providing rock solid durability and availability. Efficient Queries. What are Global Secondary Indexes?

Games 130

Games Scalability Database Retail 130

Smashing Magazine

JANUARY 11, 2021

Assets Optimizations Brotli, AVIF, WebP, responsive images, AV1, adaptive media loding, video compression, web fonts, Google fonts. Paddy Ganti’s script constructs two URLs (one normal and one blocking the ads), prompts the generation of a video comparison via WebPageTest and reports a delta.

Performance 145

Performance Cache Media Metrics 145

High Scalability

NOVEMBER 25, 2019

Egnyte is a secure Content Collaboration and Data Governance platform, founded in 2007 when Google drive wasn't born and AWS S3 was cost-prohibitive. Our only option was to roll up our sleeves and build basic cloud file system components such as object store ourselves. Video Transcoding. Migration to cloud. Google cloud.

Architecture 52

Architecture Cache Azure Storage 52

Smashing Magazine

AUGUST 20, 2021

Fast-forward 30 years, and website technology has changed significantly — we have images, stylesheets, JavaScript, streaming video, AJAX, animation, WebSockets, WebGL, rounded corners in CSS — the list goes on. Twitch developer documentation hosted on AWS, edited on CloudCannon. The site will load in a snap. Large preview ).

Ecommerce 139

Ecommerce Website Government Internet 139

Brendan Gregg

SEPTEMBER 5, 2017

. ## Observability Here's the big picture of performance observability tools on Linux, from my [Linux performance] page, where I also have diagrams for other tool types, as well as videos and slides of prior Linux performance talks: I also have a USE Method: Linux Performance Checklist , as a different way to navigate and apply the tools.

Virtualization 76

Virtualization AWS Engineering Hardware 76

Smashing Magazine

JANUARY 6, 2020

To get a good first impression of how your competitors perform, you can use Chrome UX Report ( CrUX , a ready-made RUM data set, video introduction by Ilya Grigorik and detailed guide by Rick Viscomi) or Treo Sites , a RUM monitoring tool that is powered by Chrome UX Report. You can then mimic the 90th percentile’s experience for testing.

Performance 145

Performance Cache Network Metrics 145

Smashing Magazine

JANUARY 7, 2019

To get a good first impression of how your competitors perform, you can use Chrome UX Report ( CrUX , a ready-made RUM data set, video introduction by Ilya Grigorik), Speed Scorecard (also provides a revenue impact estimator), Real User Experience Test Comparison or SiteSpeed CI (based on synthetic testing).

Performance 104

Performance Cache Network Metrics 104