All Things Distributed

MAY 10, 2016

Building your applications with only managed components has become very popular, and AWS Lambda plays a crucial role in that. If you are looking for more examples there are the Lambda Serverless Reference Architectures that can serve as the blueprint for building your own serverless applications. about teletext.io

Lambda 105

Lambda Serverless AWS Architecture 105

Adrian Cockcroft

NOVEMBER 26, 2023

The Pantheon in Rome — Extremely sustainable architecture — photo by Adrian I wrote a medium post after AWS re:Invent 2022 summarizing the (lack of) news and all the talks related to Sustainability. This includes providing the efficient, resilient services AWS customers expect, while minimizing their environmental footprint.

AWS 52

AWS Energy IoT Best Practices 52

Scalegrid

MARCH 14, 2024

The architecture usually integrates several private, public, and on-premises infrastructures. In practice, a hybrid cloud operates by melding resources and services from multiple computing environments, which necessitates effective coordination, orchestration, and integration to work efficiently.

Strategy 130

Strategy Cloud Artificial Intelligence Infrastructure 130

Scalegrid

MAY 13, 2020

In this blog post, we explain what Greenplum is, and break down the Greenplum architecture, advantages, major use cases, and how to get started. It’s architecture was specially designed to manage large-scale data warehouses and business intelligence workloads by giving you the ability to spread your data out across a multitude of servers.

Big Data 321

Big Data Database Artificial Intelligence Open Source 321

The Netflix TechBlog

SEPTEMBER 24, 2021

Table 1: Movie and File Size Examples Initial Architecture A simplified view of our initial cloud video processing pipeline is illustrated in the following diagram. Figure 1: A Simplified Video Processing Pipeline With this architecture, chunk encoding is very efficient and processed in distributed cloud computing instances.

Cloud 237

Cloud Media Storage Cache 237

The Morning Paper

MARCH 8, 2020

Building an elastic query engine on disaggregated storage , Vuppalapati, NSDI’20. For such workloads, shared-nothing architectures beget high cost, inflexibility, poor performance, and inefficiency, which hurts production applications and cluster deployments. joins) during query processing. Disaggregation (or not).

Storage 112

Storage Engineering Cache Serverless 112

Scalegrid

JANUARY 8, 2024

Real-world examples like Spotify’s multi-cloud strategy for cost reduction and performance, and Netflix’s hybrid cloud setup for efficient content streaming and creation, illustrate the practical applications of each model. Thus making it an ideal choice for businesses seeking a successful implementation of their multi-cloud strategy.

Strategy 147

Strategy Cloud Scalability Artificial Intelligence 147

Percona

JULY 1, 2023

Now that Database-as-a-service (DBaaS) is in high demand, there are multiple questions regarding AWS services that cannot always be answered easily: When should I use Aurora and when should I use RDS MySQL ? Amazon Aurora is a proprietary, cloud-native, fully managed relational database service developed by Amazon Web Services (AWS).

AWS 52

AWS Database Serverless Storage 52

Dynatrace

MAY 26, 2021

The second focused on the OTel community, with more technical talks by representatives from companies like AWS, Elastic, and more. Trace-based sampling can help you save storage costs. This can help you save money in storage costs in the long run. AWS aims to support the streamlining of observability.

AWS 218

AWS Strategy Storage Metrics 218

Percona

MARCH 7, 2023

One approach is to separate compute and storage to allow for independent scaling. It’s an open source alternative to AWS Aurora Postgres that utilizes a serverless architecture. It’s an open source alternative to AWS Aurora Postgres that utilizes a serverless architecture. So, what is Neon?

Serverless 107

Serverless Storage Open Source Architecture 107

Dynatrace

DECEMBER 15, 2022

In this blog, we share three log ingestion strategies from the field that demonstrate how building up efficient log collection can be environment-agnostic by using our generic log ingestion application programming interface (API). 3: Ingest AWS Fargate logs with Fluent Bit. Let’s take this example of AWS Fargate.

Strategy 185

Strategy AWS Open Source Transportation 185

Dynatrace

OCTOBER 6, 2023

Buckets are similar to folders, a physical storage location. Debug-level logs, which also generate high volumes and have a shorter lifespan or value period than other logs, could similarly benefit from dedicated storage. Suppose a single Grail environment is central storage for pre-production and production systems.

Artificial Intelligence 266

Artificial Intelligence Analytics Metrics Storage 266

High Scalability

SEPTEMBER 8, 2018

RISELabs , those wonderfully innovative folks over at Berkeley, have uplifted their Anna datatabase —a shared-nothing, thread-per-core architecture to achieve lightning-fast speeds by avoiding all coordination mechanisms—to become cloud-aware. No, I don’t think that is because AWS is earning a 355x margin on DynamoDB!

Storage 199

Storage Performance AWS Efficiency 199

Scalegrid

JANUARY 8, 2024

This article delves into the specifics of how AI optimizes cloud efficiency, ensures scalability, and reinforces security, providing a glimpse at its transformative role without giving away extensive details. Using AI for Enhanced Cloud Operations The integration of AI in cloud computing is enhancing operational efficiency in several ways.

Artificial Intelligence 147

Artificial Intelligence Cloud Scalability Analytics 147

Brendan Gregg

NOVEMBER 29, 2017

At Netflix, we've been using these technologies as they've been made available for instance types in the AWS EC2 cloud. The latest AWS hypervisor, Nitro, uses everything to provide a new hardware-assisted hypervisor that is easy to use and has near bare-metal performance. AWS called this [enhanced networking]. The first was c3.

Virtualization 111

Virtualization AWS Hardware Storage 111

Dynatrace

MAY 26, 2021

The second focused on the OTel community, with more technical talks by representatives from companies like AWS, Elastic, and more. Trace-based sampling can help you save storage costs. This can help you save money in storage costs in the long run. AWS aims to support the streamlining of observability.

AWS 147

AWS Strategy Storage Metrics 147

Dynatrace

NOVEMBER 15, 2021

Log monitoring, log analysis, and log analytics are more important than ever as organizations adopt more cloud-native technologies, containers, and microservices-based architectures. Driving this growth is the increasing adoption of hyperscale cloud providers (AWS, Azure, and GCP) and containerized microservices running on Kubernetes.

Analytics 214

Analytics Monitoring Artificial Intelligence DevOps 214

The Netflix TechBlog

OCTOBER 27, 2020

The processed data is typically stored as data warehouse tables in AWS S3. At Netflix, we also heavily embrace a microservice architecture that emphasizes separation of concerns. Therefore, we must efficiently move data from the data warehouse to a global, low-latency and highly-reliable key-value store.

Latency 243

Latency Storage Big Data Tuning 243

The Netflix TechBlog

APRIL 26, 2022

Figure 1: Netflix ML Architecture Fact: A fact is data about our members or videos. The first version of our logger library optimized for storage by deduplicating facts and optimized for network i/o using different compression methods for each fact. An example of video data is video metadata, like the length of a video.

Storage 187

Storage Design Scalability Latency 187

All Things Distributed

JUNE 26, 2016

In June 2015, Amazon Web Services announced that it would launch a new AWS infrastructure region in India. Examples of continuous sensing are found in the managed cloud platform built by Rachio on AWS IoT to enable the secure interaction of its connected devices with cloud applications/other devices. The opportunity to revolutionize.

AWS 90

AWS Cloud Healthcare Blockchain 90

All Things Distributed

MAY 19, 2017

In many, high-throughput, OLTP style applications the database plays a crucial role to achieve scale, reliability, high-performance and cost efficiency. Amazon Aurora has become the fastest-growing service in the history of AWS and frequently is the target of migration from on-premise proprietary databases.

Database 117

Database Design Cloud Storage 117

The Netflix TechBlog

MARCH 10, 2023

are stored in secure storage layers. Amsterdam is built on top of three storage layers. These applications are built on a microservices architecture, and the Asset Management Platform provides asset management to those dozens of services for various asset types. The first layer, Cassandra , is the source of truth for us.

Strategy 258

Strategy Cache Analytics Storage 258

KeyCDN

FEBRUARY 22, 2023

Chatbots and virtual assistants Chatbots and virtual assistants are becoming more common on websites and web applications as they provide an efficient and convenient way for users to interact with a business. There are several popular cloud-based platforms for web development and deployment, such as AWS , Azure , and Google Cloud Platform.

Artificial Intelligence 111

Artificial Intelligence Development Serverless Website 111

High Scalability

OCTOBER 24, 2022

Each cloud-native evolution is about using the hardware more efficiently. You can even go old school and use non cloud-native architectures. There's a huge short-term and long-term efficiency of services that depends on the successful coordination of cloud services and infrastructure. For example, AWS created Nitro.

Cloud 187

Cloud Hardware Infrastructure Innovation 187

Brendan Gregg

NOVEMBER 28, 2017

At Netflix, we've been using these technologies as they've been made available for instance types in the AWS EC2 cloud. The latest AWS hypervisor, Nitro, uses everything to provide a new hardware-assisted hypervisor that is easy to use and has near bare-metal performance. AWS called this [enhanced networking]. The first was c3.

Virtualization 40

Virtualization AWS Hardware Storage 40

All Things Distributed

AUGUST 20, 2015

Today, we are releasing a plugin that allows customers to use the Titan graph engine with Amazon DynamoDB as the backend storage layer. It opens up the possibility to enjoy the value that graph databases bring to relationship-centric use cases, without worrying about managing the underlying storage. The importance of relationships.

Database 134

Database Logistics Availability Social Media 134

High Scalability

NOVEMBER 25, 2019

It is limited by the disk space; it can’t expand storage elastically; it chokes if you run few I/O intensive processes or try collaborating with 100 other users. Egnyte is a secure Content Collaboration and Data Governance platform, founded in 2007 when Google drive wasn't born and AWS S3 was cost-prohibitive. Offline access.

Architecture 52

Architecture Cache Azure Storage 52

All Things Distributed

MARCH 11, 2016

The epoch of AWS is the launch of Amazon S3 on March 14, 2006, now almost 10 years ago. Given that AWS is a pioneer in building and operating these services world-wide, these lessons have been of crucial importance to our business. AWS helps its customers do this too.

AWS 144

AWS Hardware Retail Virtualization 144

The Morning Paper

FEBRUARY 6, 2020

Given the simplicity and economic appeal of FaaS, it is interesting to explore designs that preserve the autoscaling and operational benefits of current offerings, while adding performant, cost-efficient and consistent shared state and communication. High level architecture. Evaluation. A closing thought.

Lambda 98

Lambda Serverless Cache Latency 98

O'Reilly

MAY 19, 2020

AWS is far and away the cloud leader, followed by Azure (at more than half of share) and Google Cloud. But most Azure and GCP users also use AWS; the reverse isn’t necessarily true. Amazon and AWS Ascendant. If Microsoft and Google really are coming on strong, they aren’t dislodging Amazon and AWS.

Cloud 141

Cloud Serverless AWS Architecture 141

IO River

NOVEMBER 2, 2023

It’s not just limited to cloud resources like AWS and Azure; Terraform is versatile, extending its capabilities to key performance areas like Content Delivery Network (CDN) management, ensuring efficient content delivery and optimal user experience.‍Started

Open Source 52

Open Source DevOps AWS Azure 52

The Netflix TechBlog

JULY 30, 2019

Many of the Netflix Content and Studio Engineering services rely on Conductor for efficient processing of their business flows. Our cluster size ranges from 12–18 instances of AWS EC2 m4.4xlarge instances, typically running at ~30% capacity. *?—?Cassandra The Netflix Media Database (NMDB) is one such example.

Lambda 189

Lambda Media Open Source Metrics 189

The Netflix TechBlog

MAY 21, 2019

There are several problems efficient test scheduling could help us solve: Quickly detect a regression in the integration of the Netflix SDK on a consumer electronic or MVPD (multichannel video programming distributor) device. Lerner, as a web-service, relies on Amazon Web Services (AWS) and Netflix’s Open Source Software (OSS) tools.

Testing 163

Testing AWS Lambda Network 163

Brendan Gregg

FEBRUARY 28, 2023

Titus, the Netflix container management platform, is now open source,” [link] Apr 2018 - [Cutress 19] Dr. [link] 2021 - [Trader 21] Tiffany Trader, “Cerebras Second-Gen 7nm Wafer Scale Engine Doubles AI Performance Over First-Gen Chip ,” [link] Apr 2021 - [Ghigoff 21] Yoann Ghigoff, et al., "BMC:

Performance 110

Performance Latency Cache Virtualization 110

All Things Distributed

NOVEMBER 28, 2012

Also, because similar data are stored sequentially, Amazon Redshift can compress data efficiently, which further reduces the amount of IO it needs to perform to return results. s architecture and underlying platform are also optimized to deliver high performance for data warehousing workloads. Amazon Redshiftâ??s Amazon Redshiftâ??s

Cloud 164

Cloud Storage Architecture Scalability 164

Adrian Cockcroft

APRIL 5, 2023

At the instance level, many CPU and system architectures have direct access to power consumption metrics. Clock rates also change, some ARM architectures include a mixture of high performance and low power cores in the same CPU, and Intel architecture CPUs have hyperthreading, so the performance of an execution thread will vary over time.

Energy 52

Energy Metrics Cloud Operating System 52

IO River

NOVEMBER 2, 2023

In fact, Amazon Web Services (AWS) paved the way with its service, CloudFormation, which introduced the concept of Infrastructure as Code (IaC). Taking inspiration from AWS's CloudFormation, HashiCorp introduced Terraform with a broader vision. But here's the catch: if the provider lacks certain features, you're handcuffed.

Open Source 40

Open Source DevOps AWS Azure 40