Big Data, Design, Storage and Technology - Technology Performance Pulse

In-Stream Big Data Processing

Highly Scalable

AUGUST 20, 2013

The shortcomings and drawbacks of batch-oriented data processing were widely recognized by the Big Data community quite a long time ago. This system has been designed to supplement and succeed the existing Hadoop-based system that had too high latency of data processing and too high maintenance costs.

Big Data

Big Data Processing Lambda Database

Data Engineers of Netflix?—?Interview with Pallavi Phadnis

The Netflix TechBlog

OCTOBER 28, 2021

Netflix’s unique work culture and petabyte-scale data problems are what drew me to Netflix. During earlier years of my career, I primarily worked as a backend software engineer, designing and building the backend systems that enable big data analytics.

Data Engineering

Data Engineering Engineering Big Data Software Engineering

What is ITOps? Why IT operations is more crucial than ever in a multicloud world

Dynatrace

DECEMBER 15, 2022

As more organizations adopt cloud-native technologies, traditional approaches to IT operations have been evolving. Complex cloud computing environments are increasingly replacing traditional data centers. In fact, Gartner estimates that 80% of enterprises will shut down their on-premises data centers by 2025. So, what is ITOps?

Artificial Intelligence

Artificial Intelligence DevOps Hardware Virtualization

What is a Distributed Storage System

Scalegrid

FEBRUARY 8, 2024

A distributed storage system is foundational in today’s data-driven landscape, ensuring data spread over multiple servers is reliable, accessible, and manageable. This guide delves into how these systems work, the challenges they solve, and their essential role in businesses and technology.

Storage

Storage Systems Big Data Azure

Driving down the cost of Big-Data analytics - All Things Distributed

All Things Distributed

AUGUST 18, 2011

Driving down the cost of Big-Data analytics. The Amazon Elastic MapReduce (EMR) team announced today the ability to seamlessly use Amazon EC2 Spot Instances with their service, significantly driving down the cost of data analytics in the cloud. Driving Storage Costs Down for AWS Customers. Comments (). or rss feed.

Big Data

Big Data Analytics AWS Cloud

Expanding the Cloud ? Managing Cold Storage with Amazon Glacier

All Things Distributed

AUGUST 20, 2012

Managing Cold Storage with Amazon Glacier. With the introduction of Amazon Glacier , IT organizations now have a solution that removes the headaches of digital archiving and provides extremely low cost storage. With Amazon Glacier any organization now has access to the same data archiving capabilities as the worldâ??s

Storage

Storage Cloud AWS Media

Why MySQL Could Be Slow With Large Tables

Percona

JANUARY 19, 2023

While the technologies have evolved and matured enough, there are still some people thinking that MySQL is only for small projects or that it can’t perform well with large tables. Make sure you design the data types correctly while planning for the future growth of the table. MyRocks is shipped in Percona Server for MySQL.

Open Source

Open Source Storage Database Big Data

Expanding the Cloud - AWS Import/Export Support for Amazon EBS.

All Things Distributed

JULY 7, 2011

AWS Import/Export transfers data off of storage devices using Amazons high-speed internal network and bypassing the Internet. With this new functionality AWS Import/Export now supports importing data directly into Amazon EBS snapshots. Driving Storage Costs Down for AWS Customers. Subscribe to this weblogs. or rss feed.

AWS

AWS Cloud Storage Internet

No Server Required - Jekyll & Amazon S3 - All Things Distributed

All Things Distributed

AUGUST 17, 2011

As some of you may remember I was pretty excited when Amazon Simple Storage Service (S3) released its website feature such that I could serve this weblog completely from S3. It is simple and elegant, as you would expect from someone who has won several design awards. Driving Storage Costs Down for AWS Customers. or rss feed.

Servers

Servers Social Media AWS Website

Hacking with AWS at The Next Web Hackaton - All Things Distributed

All Things Distributed

MARCH 24, 2011

Over the past years The Next Web Conference has become a premier conference on internet life and its technologies. Up to 200 developers and designers will get together to hack up interesting applications using the Internets APIs and SDKs. If you want to be that team of 5-7 developer/designers you can sign up using this form.

AWS

AWS Internet Internet Storage

Job Openings in AWS - Senior Leader in Database Services - All.

All Things Distributed

AUGUST 19, 2011

AWS Database Services is responsible for setting the database strategy and delivering distributed structured storage services to our AWS customers. Driving Storage Costs Down for AWS Customers. Expanding the Cloud - The AWS Storage Gateway. a Fast and Scalable NoSQL Database Service Designed for Internet Scale Applications.

AWS

AWS Database Storage Scalability

The AWS GovCloud (US) Region - All Things Distributed

All Things Distributed

AUGUST 16, 2011

The Cloud First strategy is most visible with new Federal IT programs, which are all designed to be â??Cloud Government and Big Data. One particular early use case for AWS GovCloud (US) will be massive data processing and analytics. Driving Storage Costs Down for AWS Customers. Cloud Readyâ??; More information.

AWS

AWS Government Big Data Cloud

Introducing the AWS South America - All Things Distributed

All Things Distributed

DECEMBER 14, 2011

Driving Storage Costs Down for AWS Customers. Expanding the Cloud - The AWS Storage Gateway. a Fast and Scalable NoSQL Database Service Designed for Internet Scale Applications. Driving down the cost of Big-Data analytics. The Amazon.com 2010 Shareholder Letter Focusses on Technology. or rss feed.

AWS

AWS Latency Storage Big Data

Expanding the Cloud - Introducing Amazon ElastiCache - All Things.

All Things Distributed

AUGUST 22, 2011

Driving Storage Costs Down for AWS Customers. Expanding the Cloud - The AWS Storage Gateway. a Fast and Scalable NoSQL Database Service Designed for Internet Scale Applications. Driving down the cost of Big-Data analytics. The Amazon.com 2010 Shareholder Letter Focusses on Technology. or rss feed.

Cloud

Cloud Cache AWS Storage

Expanding the Cloud - Introducing the AWS Asia Pacific (Tokyo.

All Things Distributed

MARCH 2, 2011

Driving Storage Costs Down for AWS Customers. Expanding the Cloud - The AWS Storage Gateway. a Fast and Scalable NoSQL Database Service Designed for Internet Scale Applications. Driving down the cost of Big-Data analytics. The Amazon.com 2010 Shareholder Letter Focusses on Technology. or rss feed.

AWS

AWS Cloud Games Latency

Driving Bandwidth Cost Down for AWS Customers. - All Things.

All Things Distributed

JUNE 29, 2011

Driving Storage Costs Down for AWS Customers. Expanding the Cloud - The AWS Storage Gateway. a Fast and Scalable NoSQL Database Service Designed for Internet Scale Applications. Driving down the cost of Big-Data analytics. The Amazon.com 2010 Shareholder Letter Focusses on Technology. or rss feed.

AWS

AWS Retail Innovation Strategy

Free at Last - A Fully Self-Sustained Blog Running in Amazon S3.

All Things Distributed

FEBRUARY 23, 2011

Driving Storage Costs Down for AWS Customers. Expanding the Cloud - The AWS Storage Gateway. a Fast and Scalable NoSQL Database Service Designed for Internet Scale Applications. Driving down the cost of Big-Data analytics. The Amazon.com 2010 Shareholder Letter Focusses on Technology. or rss feed.

AWS

AWS Storage Big Data Servers

Redis vs Memcached in 2024

Scalegrid

MARCH 28, 2024

With these goals in mind, two in-memory data stores, Redis and Memcached, have emerged as the top contenders. This article will explore how they handle data storage and scalability, perform in different scenarios, and, most importantly, how these factors influence your choice. Data transfer technology.

Cache

Cache Storage Scalability Architecture

From the Archives - Gapingvoid's Nobody Cares - All Things.

All Things Distributed

FEBRUARY 27, 2011

Driving Storage Costs Down for AWS Customers. Expanding the Cloud - The AWS Storage Gateway. a Fast and Scalable NoSQL Database Service Designed for Internet Scale Applications. Driving down the cost of Big-Data analytics. The Amazon.com 2010 Shareholder Letter Focusses on Technology. or rss feed.

AWS

AWS Storage Big Data Scalability

DROAM - Dreaming about Cheap Data Roaming - All Things.

All Things Distributed

JANUARY 11, 2011

Driving Storage Costs Down for AWS Customers. Expanding the Cloud - The AWS Storage Gateway. a Fast and Scalable NoSQL Database Service Designed for Internet Scale Applications. Driving down the cost of Big-Data analytics. The Amazon.com 2010 Shareholder Letter Focusses on Technology. or rss feed.

Wireless

Wireless AWS Internet Internet

Expanding the Cloud - Cluster Compute Instances for Amazon EC2.

All Things Distributed

JULY 13, 2010

Cluster Computer Instances for Amazon EC2 are a new instance type specifically designed for High Performance Computing applications. During my academic career, I spent many years working on HPC technologies such as user-level networking interfaces, large scale high-speed interconnects, HPC software stacks, etc. or rss feed.

Cloud

Cloud AWS Automotive Latency

Reboot - All Things Distributed

All Things Distributed

SEPTEMBER 29, 2010

Driving Storage Costs Down for AWS Customers. Expanding the Cloud - The AWS Storage Gateway. a Fast and Scalable NoSQL Database Service Designed for Internet Scale Applications. Driving down the cost of Big-Data analytics. The Amazon.com 2010 Shareholder Letter Focusses on Technology. or rss feed.

AWS

AWS Storage Big Data Scalability

Spot Instances - Increased Control - All Things Distributed

All Things Distributed

JULY 11, 2011

Driving Storage Costs Down for AWS Customers. Expanding the Cloud - The AWS Storage Gateway. a Fast and Scalable NoSQL Database Service Designed for Internet Scale Applications. Driving down the cost of Big-Data analytics. The Amazon.com 2010 Shareholder Letter Focusses on Technology. or rss feed.

AWS

AWS Storage Cloud Big Data

New Route 53 and ELB features: IPv6, Zone Apex, WRR and more.

All Things Distributed

MAY 24, 2011

Driving Storage Costs Down for AWS Customers. Expanding the Cloud - The AWS Storage Gateway. a Fast and Scalable NoSQL Database Service Designed for Internet Scale Applications. Driving down the cost of Big-Data analytics. The Amazon.com 2010 Shareholder Letter Focusses on Technology. or rss feed.

Internet

Internet Internet AWS Scalability

APAC Summer Tour - All Things Distributed

All Things Distributed

JULY 3, 2011

Driving Storage Costs Down for AWS Customers. Expanding the Cloud - The AWS Storage Gateway. a Fast and Scalable NoSQL Database Service Designed for Internet Scale Applications. Driving down the cost of Big-Data analytics. The Amazon.com 2010 Shareholder Letter Focusses on Technology. or rss feed.

AWS

AWS Storage Cloud Big Data

What is a data lakehouse? Combining data lakes and warehouses for the best of both worlds

Dynatrace

OCTOBER 4, 2022

A data lakehouse features the flexibility and cost-efficiency of a data lake with the contextual and high-speed querying capabilities of a data warehouse. Data warehouses offer a single storage repository for structured data and provide a source of truth for organizations. How does a data lakehouse work?

Artificial Intelligence

Artificial Intelligence Storage Analytics Government

Around the World in 28 Days - All Things Distributed

All Things Distributed

SEPTEMBER 30, 2010

Mumbai on October 20 with Patni , Inxs Technologies and India Games. Driving Storage Costs Down for AWS Customers. Expanding the Cloud - The AWS Storage Gateway. a Fast and Scalable NoSQL Database Service Designed for Internet Scale Applications. Driving down the cost of Big-Data analytics. or rss feed.

AWS

AWS Storage Cloud Best Practices

AWS Elastic Beanstalk: A Quick and Simple Way into the Cloud - All.

All Things Distributed

JANUARY 19, 2011

It targets both the application developer by providing a set of simple tools to get started quickly and the platform developer by giving control over the underlying technology stack. Driving Storage Costs Down for AWS Customers. Expanding the Cloud - The AWS Storage Gateway. Driving down the cost of Big-Data analytics.

AWS

AWS Cloud Java Scalability

I am looking for new application and platform services - All Things.

All Things Distributed

APRIL 23, 2010

Driving Storage Costs Down for AWS Customers. Expanding the Cloud - The AWS Storage Gateway. a Fast and Scalable NoSQL Database Service Designed for Internet Scale Applications. Driving down the cost of Big-Data analytics. The Amazon.com 2010 Shareholder Letter Focusses on Technology. or rss feed.

AWS

AWS Storage Cloud Big Data

Expanding the Cloud with DNS - Introducing Amazon Route 53 - All.

All Things Distributed

DECEMBER 5, 2010

The Domain Name System is a wonderful practical piece of technology; it is a fundamental building block of our modern internet. We have designed Route 53 to propagate updates very quickly and give the customer the tools to find out when all changes have been propagated. Driving Storage Costs Down for AWS Customers. or rss feed.

Cloud

Cloud Internet Internet AWS

Expanding the Cloud - New AWS Region: US-West (Northern.

All Things Distributed

DECEMBER 3, 2009

Driving Storage Costs Down for AWS Customers. Expanding the Cloud - The AWS Storage Gateway. a Fast and Scalable NoSQL Database Service Designed for Internet Scale Applications. Driving down the cost of Big-Data analytics. The Amazon.com 2010 Shareholder Letter Focusses on Technology. or rss feed.

AWS

AWS Cloud Latency Storage

This week in review: GPUs, Zombies, Biomimicry and Tom Waits.

All Things Distributed

NOVEMBER 19, 2010

Driving Storage Costs Down for AWS Customers. Expanding the Cloud - The AWS Storage Gateway. a Fast and Scalable NoSQL Database Service Designed for Internet Scale Applications. Driving down the cost of Big-Data analytics. The Amazon.com 2010 Shareholder Letter Focusses on Technology. or rss feed.

AWS

AWS Cloud Benchmarking Storage

Powerful New Amazon EC2 Boot Features - All Things Distributed

All Things Distributed

DECEMBER 3, 2009

with new security patches installed), or add new user data. While the instance is stopped it does not accrue any usage hours and customers are only charged for the storage associated with the Amazon EBS volume. Driving Storage Costs Down for AWS Customers. Expanding the Cloud - The AWS Storage Gateway. or rss feed.

AWS

AWS Storage Operating System Cloud

The Amazon.com 2010 Shareholder Letter Focusses on Technology.

All Things Distributed

APRIL 27, 2011

The Amazon.com 2010 Shareholder Letter Focusses on Technology. In the 2010 Shareholder Letter Jeff Bezos writes about the unique technologies developed at Amazon.com over the years. Given that I have frequently written about many of these technologies on this blog I asked investor relations to be allowed to reprint it here.

Technology

Technology Technology AWS Storage

Mastering Hybrid Cloud Strategy

Scalegrid

MARCH 14, 2024

Integrating technology from private and public clouds and on-premises resources within one hybrid cloud platform creates an integrated IT infrastructure that leverages the strengths of each component. We will examine each of these elements in more detail.

Strategy

Strategy Cloud Artificial Intelligence Infrastructure

Expanding the Cloud - Opening the AWS Asia Pacific (Singapore.

All Things Distributed

APRIL 28, 2010

You can begin using Amazon Elastic Compute Cloud (Amazon EC2), Amazon Simple Storage Service (Amazon S3), Amazon SimpleDB, Amazon Relational Database Service (Amazon RDS), Amazon Simple Queue Service (Amazon SQS), Amazon Simple Notification Service (Amazon SNS), Amazon CloudWatch, and Amazon CloudFront from the Singapore Region beginning today.

AWS

AWS Cloud Latency Storage

Expanding the Cloud - Amazon EC2 Spot Instances - All Things.

All Things Distributed

DECEMBER 13, 2009

Consistently we have lowered compute, storage and bandwidth prices based on such cost savings. There is one technological requirement for the applications that run as Spot Instances: they need to be able to handle that their computation can be stopped and restarted based on the Spot Price in relation to the customers pricing limit.

Cloud

Cloud AWS Storage Innovation

Bulldozer: Batch Data Moving from Data Warehouse to Online Key-Value Stores

The Netflix TechBlog

OCTOBER 27, 2020

Data scientists and engineers collect this data from our subscribers and videos, and implement data analytics models to discover customer behaviour with the goal of maximizing user joy. The processed data is typically stored as data warehouse tables in AWS S3.

Latency

Latency Storage Big Data Tuning

Structural Evolutions in Data

O'Reilly

SEPTEMBER 19, 2023

.” That came to mind when a friend raised a point about emerging technology’s fractal nature. Each time, the underlying implementation changed a bit while still staying true to the larger phenomenon of “Analyzing Data for Fun and Profit.” ” (It will be easier to fit in the overhead storage.)

Hardware

Hardware Storage Big Data Blockchain

Expanding the AWS Cloud: Introducing the AWS Europe (London) Region

All Things Distributed

DECEMBER 13, 2016

Today, I'm happy to announce that the AWS Europe (London) Region, our 16th technology infrastructure region globally, is now generally available for use by customers worldwide. The British Government is also helping to drive innovation and has embraced a cloud-first policy for technology adoption.

AWS

AWS Cloud Artificial Intelligence IoT

Expanding the Cloud - Amazon S3 Reduced Redundancy Storage.

All Things Distributed

MAY 18, 2010

Expanding the Cloud - Amazon S3 Reduced Redundancy Storage. Today a new storage option for Amazon S3 has been launched: Amazon S3 Reduced Redundancy Storage (RRS). This new storage option enables customers to reduce their costs by storing non-critical, reproducible data at lower levels of redundancy. Comments ().

Storage

Storage Cloud AWS Scalability

Music to my Ears - All Things Distributed

All Things Distributed

MARCH 28, 2011

The scalability, reliability and durability requirements for Cloud Drive are very high which is why they decided to make use of the Amazon Simple Storage Service (S3) as the core component of their service. Driving Storage Costs Down for AWS Customers. Expanding the Cloud - The AWS Storage Gateway. More details at [link].

AWS

AWS Cloud Storage Internet

New AWS feature: Run your website from Amazon S3 - All Things.

All Things Distributed

FEBRUARY 17, 2011

Since a few days ago this weblog serves 100% of its content directly out of the Amazon Simple Storage Service (S3) without the need for a web server to be involved. Driving Storage Costs Down for AWS Customers. Expanding the Cloud - The AWS Storage Gateway. Driving down the cost of Big-Data analytics. Comments ().

AWS

AWS Website Storage Servers

Simplifying IT - Create Your Application with AWS CloudFormation.

All Things Distributed

FEBRUARY 25, 2011

Driving Storage Costs Down for AWS Customers. Expanding the Cloud - The AWS Storage Gateway. a Fast and Scalable NoSQL Database Service Designed for Internet Scale Applications. Driving down the cost of Big-Data analytics. The Amazon.com 2010 Shareholder Letter Focusses on Technology. or rss feed.

AWS

AWS Cloud Scalability Storage

In-Stream Big Data Processing

Data Engineers of Netflix?—?Interview with Pallavi Phadnis

Trending Sources

What is ITOps? Why IT operations is more crucial than ever in a multicloud world

What is a Distributed Storage System

Driving down the cost of Big-Data analytics - All Things Distributed

Expanding the Cloud ? Managing Cold Storage with Amazon Glacier

Why MySQL Could Be Slow With Large Tables

Expanding the Cloud - AWS Import/Export Support for Amazon EBS.

No Server Required - Jekyll & Amazon S3 - All Things Distributed

Hacking with AWS at The Next Web Hackaton - All Things Distributed

Job Openings in AWS - Senior Leader in Database Services - All.

The AWS GovCloud (US) Region - All Things Distributed

Introducing the AWS South America - All Things Distributed

Expanding the Cloud - Introducing Amazon ElastiCache - All Things.

Expanding the Cloud - Introducing the AWS Asia Pacific (Tokyo.

Driving Bandwidth Cost Down for AWS Customers. - All Things.

Free at Last - A Fully Self-Sustained Blog Running in Amazon S3.

Redis vs Memcached in 2024

From the Archives - Gapingvoid's Nobody Cares - All Things.

DROAM - Dreaming about Cheap Data Roaming - All Things.

Expanding the Cloud - Cluster Compute Instances for Amazon EC2.

Reboot - All Things Distributed

Spot Instances - Increased Control - All Things Distributed

New Route 53 and ELB features: IPv6, Zone Apex, WRR and more.

APAC Summer Tour - All Things Distributed

What is a data lakehouse? Combining data lakes and warehouses for the best of both worlds

Around the World in 28 Days - All Things Distributed

AWS Elastic Beanstalk: A Quick and Simple Way into the Cloud - All.

I am looking for new application and platform services - All Things.

Expanding the Cloud with DNS - Introducing Amazon Route 53 - All.

Expanding the Cloud - New AWS Region: US-West (Northern.

This week in review: GPUs, Zombies, Biomimicry and Tom Waits.

Powerful New Amazon EC2 Boot Features - All Things Distributed

The Amazon.com 2010 Shareholder Letter Focusses on Technology.

Mastering Hybrid Cloud Strategy

Expanding the Cloud - Opening the AWS Asia Pacific (Singapore.

Expanding the Cloud - Amazon EC2 Spot Instances - All Things.

Bulldozer: Batch Data Moving from Data Warehouse to Online Key-Value Stores

Structural Evolutions in Data

Expanding the AWS Cloud: Introducing the AWS Europe (London) Region

Expanding the Cloud - Amazon S3 Reduced Redundancy Storage.

Music to my Ears - All Things Distributed

New AWS feature: Run your website from Amazon S3 - All Things.

Simplifying IT - Create Your Application with AWS CloudFormation.

Stay Connected