Big Data, Database, Storage and Technology - Technology Performance Pulse

In-Stream Big Data Processing

Highly Scalable

AUGUST 20, 2013

The shortcomings and drawbacks of batch-oriented data processing were widely recognized by the Big Data community quite a long time ago. The pipelines can be stateful and the engine’s middleware should provide a persistent storage to enable state checkpointing. Interoperability with Hadoop.

Big Data

Big Data Processing Lambda Database

Kubernetes in the wild report 2023

Dynatrace

JANUARY 16, 2023

While Kubernetes is still a relatively young technology, a large majority of global enterprises use it to run business-critical applications in production. Findings provide insights into Kubernetes practitioners’ infrastructure preferences and how they use advanced Kubernetes platform technologies. Java, Go, and Node.js

Open Source

Open Source Java Operating System Programming

What is a Distributed Storage System

Scalegrid

FEBRUARY 8, 2024

A distributed storage system is foundational in today’s data-driven landscape, ensuring data spread over multiple servers is reliable, accessible, and manageable. This guide delves into how these systems work, the challenges they solve, and their essential role in businesses and technology.

Storage

Storage Systems Big Data Azure

Driving down the cost of Big-Data analytics - All Things Distributed

All Things Distributed

AUGUST 18, 2011

Driving down the cost of Big-Data analytics. The Amazon Elastic MapReduce (EMR) team announced today the ability to seamlessly use Amazon EC2 Spot Instances with their service, significantly driving down the cost of data analytics in the cloud. Driving Storage Costs Down for AWS Customers. Comments (). Syndication.

Big Data

Big Data Analytics AWS Scalability

The Need for Real-Time Device Tracking

ScaleOut Software

JULY 19, 2021

Incoming data is saved into data storage (historian database or log store) for query by operational managers who must attempt to find the highest priority issues that require their attention. The technology that can do this is called in-memory computing. The heavy lifting is deferred to the back office.

IoT

IoT Analytics Big Data Architecture

Expanding the Cloud ? Managing Cold Storage with Amazon Glacier

All Things Distributed

AUGUST 20, 2012

Managing Cold Storage with Amazon Glacier. With the introduction of Amazon Glacier , IT organizations now have a solution that removes the headaches of digital archiving and provides extremely low cost storage. Data is retrieved by scheduling a job, which typically completes within 3 to 5 hours. All Things Distributed.

Storage

Storage Cloud AWS Media

Why MySQL Could Be Slow With Large Tables

Percona

JANUARY 19, 2023

While the technologies have evolved and matured enough, there are still some people thinking that MySQL is only for small projects or that it can’t perform well with large tables. With disks being faster nowadays and CPU and memory resources being cheaper, we could easily say MySQL can handle TBs of data with good performance.

Open Source

Open Source Storage Database Big Data

Job Openings in AWS - Senior Leader in Database Services - All.

All Things Distributed

AUGUST 19, 2011

Job Openings in AWS - Senior Leader in Database Services. This week it is an opening for senior leaders with AWS Database Services. AWS Database Services is responsible for setting the database strategy and delivering distributed structured storage services to our AWS customers. Comments (). Contact Info.

AWS

AWS Database Storage Scalability

Delta: A Data Synchronization and Enrichment Platform

The Netflix TechBlog

OCTOBER 15, 2019

Thus, ensuring the atomicity of writes across different storage technologies remains a challenging problem for applications [3]. Delta Delta has been developed to address the limitations of existing solutions for data synchronization, and also allows to enrich data on the fly. clock drift on the partition leader).

Transportation

Transportation Architecture Processing Storage

Expanding the Cloud - AWS Import/Export Support for Amazon EBS.

All Things Distributed

JULY 7, 2011

AWS Import/Export transfers data off of storage devices using Amazons high-speed internal network and bypassing the Internet. With this new functionality AWS Import/Export now supports importing data directly into Amazon EBS snapshots. Driving Storage Costs Down for AWS Customers. Syndication. Subscribe to this weblogs.

AWS

AWS Cloud Storage Internet

Track Thousands of Assets in a Time of Crisis Using Real-Time Digital Twins

ScaleOut Software

APRIL 3, 2020

For example, the parameters for a ventilator could include its identifier, make and model, current location, status (in use, in storage, broken), time in use, technical issues and repairs, and contact information. These questions can be answered using the latest data as it streams in from the field.

Logistics

Logistics Analytics Scalability Cloud

Track Thousands of Assets in a Time of Crisis Using Real-Time Digital Twins

ScaleOut Software

APRIL 3, 2020

For example, the parameters for a ventilator could include its identifier, make and model, current location, status (in use, in storage, broken), time in use, technical issues and repairs, and contact information. These questions can be answered using the latest data as it streams in from the field.

Logistics

Logistics Analytics Scalability Cloud

No Server Required - Jekyll & Amazon S3 - All Things Distributed

All Things Distributed

AUGUST 17, 2011

As some of you may remember I was pretty excited when Amazon Simple Storage Service (S3) released its website feature such that I could serve this weblog completely from S3. Driving Storage Costs Down for AWS Customers. Expanding the Cloud - The AWS Storage Gateway. Job Openings in AWS - Senior Leader in Database Services.

Servers

Servers Social Media AWS Website

Hacking with AWS at The Next Web Hackaton - All Things Distributed

All Things Distributed

MARCH 24, 2011

Over the past years The Next Web Conference has become a premier conference on internet life and its technologies. It is likely that the Amazon Web Services will be used by many of the participants for their compute, storage, database and other cloud resource needs. Driving Storage Costs Down for AWS Customers.

AWS

AWS Internet Internet Storage

Redis vs Memcached in 2024

Scalegrid

MARCH 28, 2024

With these goals in mind, two in-memory data stores, Redis and Memcached, have emerged as the top contenders. This article will explore how they handle data storage and scalability, perform in different scenarios, and, most importantly, how these factors influence your choice. Data transfer technology.

Cache

Cache Storage Scalability Architecture

Expanding the Cloud - Introducing Amazon ElastiCache - All Things.

All Things Distributed

AUGUST 22, 2011

There are many success stories about the effectiveness of caching in many different scenarios; next to helping applications achieving fast and predictable performance, it often protects databases from requests bursts and brownouts under overload conditions. Driving Storage Costs Down for AWS Customers. Syndication. or rss feed.

Cloud

Cloud Cache AWS Storage

Introducing the AWS South America - All Things Distributed

All Things Distributed

DECEMBER 14, 2011

Driving Storage Costs Down for AWS Customers. Expanding the Cloud - The AWS Storage Gateway. a Fast and Scalable NoSQL Database Service Designed for Internet Scale Applications. Job Openings in AWS - Senior Leader in Database Services. Driving down the cost of Big-Data analytics. Syndication. or rss feed.

AWS

AWS Latency Storage Big Data

The AWS GovCloud (US) Region - All Things Distributed

All Things Distributed

AUGUST 16, 2011

Government and Big Data. One particular early use case for AWS GovCloud (US) will be massive data processing and analytics. Several agencies of very different parts of the government have needs for data analytics that really put the Big in Big-Data, sometimes several orders of magnitude larger than commonly found in industry.

AWS

AWS Government Big Data Cloud

Expanding the Cloud - Introducing the AWS Asia Pacific (Tokyo.

All Things Distributed

MARCH 2, 2011

Driving Storage Costs Down for AWS Customers. Expanding the Cloud - The AWS Storage Gateway. a Fast and Scalable NoSQL Database Service Designed for Internet Scale Applications. Job Openings in AWS - Senior Leader in Database Services. Driving down the cost of Big-Data analytics. Syndication. or rss feed.

AWS

AWS Cloud Games Latency

Driving Bandwidth Cost Down for AWS Customers. - All Things.

All Things Distributed

JUNE 29, 2011

Driving Storage Costs Down for AWS Customers. Expanding the Cloud - The AWS Storage Gateway. a Fast and Scalable NoSQL Database Service Designed for Internet Scale Applications. Job Openings in AWS - Senior Leader in Database Services. Driving down the cost of Big-Data analytics. Syndication. or rss feed.

AWS

AWS Retail Innovation Strategy

Free at Last - A Fully Self-Sustained Blog Running in Amazon S3.

All Things Distributed

FEBRUARY 23, 2011

Driving Storage Costs Down for AWS Customers. Expanding the Cloud - The AWS Storage Gateway. a Fast and Scalable NoSQL Database Service Designed for Internet Scale Applications. Job Openings in AWS - Senior Leader in Database Services. Driving down the cost of Big-Data analytics. Syndication. or rss feed.

AWS

AWS Storage Big Data Servers

AWS Elastic Beanstalk: A Quick and Simple Way into the Cloud - All.

All Things Distributed

JANUARY 19, 2011

Flexibility is one of the key principles of Amazon Web Services - developers can select any programming language and software package, any operating system, any middleware and any database to build systems and applications that meet their requirements. Driving Storage Costs Down for AWS Customers. Syndication. or rss feed.

AWS

AWS Cloud Java Scalability

DROAM - Dreaming about Cheap Data Roaming - All Things.

All Things Distributed

JANUARY 11, 2011

Driving Storage Costs Down for AWS Customers. Expanding the Cloud - The AWS Storage Gateway. a Fast and Scalable NoSQL Database Service Designed for Internet Scale Applications. Job Openings in AWS - Senior Leader in Database Services. Driving down the cost of Big-Data analytics. Syndication. or rss feed.

Wireless

Wireless AWS Internet Internet

From the Archives - Gapingvoid's Nobody Cares - All Things.

All Things Distributed

FEBRUARY 27, 2011

Driving Storage Costs Down for AWS Customers. Expanding the Cloud - The AWS Storage Gateway. a Fast and Scalable NoSQL Database Service Designed for Internet Scale Applications. Job Openings in AWS - Senior Leader in Database Services. Driving down the cost of Big-Data analytics. Syndication. or rss feed.

AWS

AWS Storage Big Data Scalability

Reboot - All Things Distributed

All Things Distributed

SEPTEMBER 29, 2010

Driving Storage Costs Down for AWS Customers. Expanding the Cloud - The AWS Storage Gateway. a Fast and Scalable NoSQL Database Service Designed for Internet Scale Applications. Job Openings in AWS - Senior Leader in Database Services. Driving down the cost of Big-Data analytics. Syndication. or rss feed.

AWS

AWS Storage Big Data Scalability

Spot Instances - Increased Control - All Things Distributed

All Things Distributed

JULY 11, 2011

Driving Storage Costs Down for AWS Customers. Expanding the Cloud - The AWS Storage Gateway. a Fast and Scalable NoSQL Database Service Designed for Internet Scale Applications. Job Openings in AWS - Senior Leader in Database Services. Driving down the cost of Big-Data analytics. Syndication. or rss feed.

AWS

AWS Storage Cloud Big Data

New Route 53 and ELB features: IPv6, Zone Apex, WRR and more.

All Things Distributed

MAY 24, 2011

Driving Storage Costs Down for AWS Customers. Expanding the Cloud - The AWS Storage Gateway. a Fast and Scalable NoSQL Database Service Designed for Internet Scale Applications. Job Openings in AWS - Senior Leader in Database Services. Driving down the cost of Big-Data analytics. Syndication. or rss feed.

Internet

Internet Internet AWS Scalability

APAC Summer Tour - All Things Distributed

All Things Distributed

JULY 3, 2011

Driving Storage Costs Down for AWS Customers. Expanding the Cloud - The AWS Storage Gateway. a Fast and Scalable NoSQL Database Service Designed for Internet Scale Applications. Job Openings in AWS - Senior Leader in Database Services. Driving down the cost of Big-Data analytics. Syndication. or rss feed.

AWS

AWS Storage Big Data Cloud

Around the World in 28 Days - All Things Distributed

All Things Distributed

SEPTEMBER 30, 2010

Mumbai on October 20 with Patni , Inxs Technologies and India Games. Driving Storage Costs Down for AWS Customers. Expanding the Cloud - The AWS Storage Gateway. a Fast and Scalable NoSQL Database Service Designed for Internet Scale Applications. Job Openings in AWS - Senior Leader in Database Services.

AWS

AWS Storage Cloud Best Practices

Expanding the Cloud - Cluster Compute Instances for Amazon EC2.

All Things Distributed

JULY 13, 2010

During my academic career, I spent many years working on HPC technologies such as user-level networking interfaces, large scale high-speed interconnects, HPC software stacks, etc. Driving Storage Costs Down for AWS Customers. Expanding the Cloud - The AWS Storage Gateway. Job Openings in AWS - Senior Leader in Database Services.

Cloud

Cloud AWS Automotive Latency

I am looking for new application and platform services - All Things.

All Things Distributed

APRIL 23, 2010

Driving Storage Costs Down for AWS Customers. Expanding the Cloud - The AWS Storage Gateway. a Fast and Scalable NoSQL Database Service Designed for Internet Scale Applications. Job Openings in AWS - Senior Leader in Database Services. Driving down the cost of Big-Data analytics. Syndication. or rss feed.

AWS

AWS Storage Cloud Big Data

This week in review: GPUs, Zombies, Biomimicry and Tom Waits.

All Things Distributed

NOVEMBER 19, 2010

Driving Storage Costs Down for AWS Customers. Expanding the Cloud - The AWS Storage Gateway. a Fast and Scalable NoSQL Database Service Designed for Internet Scale Applications. Job Openings in AWS - Senior Leader in Database Services. Driving down the cost of Big-Data analytics. Syndication. or rss feed.

AWS

AWS Cloud Benchmarking Storage

Expanding the Cloud with DNS - Introducing Amazon Route 53 - All.

All Things Distributed

DECEMBER 5, 2010

The Domain Name System is a wonderful practical piece of technology; it is a fundamental building block of our modern internet. Driving Storage Costs Down for AWS Customers. Expanding the Cloud - The AWS Storage Gateway. a Fast and Scalable NoSQL Database Service Designed for Internet Scale Applications. Syndication.

Cloud

Cloud Internet Internet AWS

Expanding the Cloud - New AWS Region: US-West (Northern.

All Things Distributed

DECEMBER 3, 2009

Driving Storage Costs Down for AWS Customers. Expanding the Cloud - The AWS Storage Gateway. a Fast and Scalable NoSQL Database Service Designed for Internet Scale Applications. Job Openings in AWS - Senior Leader in Database Services. Driving down the cost of Big-Data analytics. Syndication. or rss feed.

AWS

AWS Cloud Latency Storage

Data lakehouse innovations advance the three pillars of observability for more collaborative analytics

Dynatrace

FEBRUARY 16, 2023

The goal is to turn more data into insights so the whole organization can make data-driven decisions and automate processes. Grail data lakehouse delivers massively parallel processing for answers at scale Modern cloud-native computing is constantly upping the ante on data volume, variety, and velocity.

Analytics

Analytics Innovation Metrics Database

Mastering Hybrid Cloud Strategy

Scalegrid

MARCH 14, 2024

Integrating technology from private and public clouds and on-premises resources within one hybrid cloud platform creates an integrated IT infrastructure that leverages the strengths of each component. Ready to take your database management to the next level with ScaleGrid’s cutting-edge solutions?

Strategy

Strategy Cloud Artificial Intelligence Infrastructure

Powerful New Amazon EC2 Boot Features - All Things Distributed

All Things Distributed

DECEMBER 3, 2009

with new security patches installed), or add new user data. While the instance is stopped it does not accrue any usage hours and customers are only charged for the storage associated with the Amazon EBS volume. Driving Storage Costs Down for AWS Customers. Expanding the Cloud - The AWS Storage Gateway. Syndication.

AWS

AWS Storage Operating System Cloud

Expanding the Cloud - Opening the AWS Asia Pacific (Singapore.

All Things Distributed

APRIL 28, 2010

You can begin using Amazon Elastic Compute Cloud (Amazon EC2), Amazon Simple Storage Service (Amazon S3), Amazon SimpleDB, Amazon Relational Database Service (Amazon RDS), Amazon Simple Queue Service (Amazon SQS), Amazon Simple Notification Service (Amazon SNS), Amazon CloudWatch, and Amazon CloudFront from the Singapore Region beginning today.

AWS

AWS Cloud Latency Storage

The Amazon.com 2010 Shareholder Letter Focusses on Technology.

All Things Distributed

APRIL 27, 2011

The Amazon.com 2010 Shareholder Letter Focusses on Technology. In the 2010 Shareholder Letter Jeff Bezos writes about the unique technologies developed at Amazon.com over the years. Given that I have frequently written about many of these technologies on this blog I asked investor relations to be allowed to reprint it here.

Technology

Technology Technology AWS Storage

Expanding the Cloud - Amazon EC2 Spot Instances - All Things.

All Things Distributed

DECEMBER 13, 2009

Consistently we have lowered compute, storage and bandwidth prices based on such cost savings. There is one technological requirement for the applications that run as Spot Instances: they need to be able to handle that their computation can be stopped and restarted based on the Spot Price in relation to the customers pricing limit.

Cloud

Cloud AWS Storage Innovation

Bulldozer: Batch Data Moving from Data Warehouse to Online Key-Value Stores

The Netflix TechBlog

OCTOBER 27, 2020

Data scientists and engineers collect this data from our subscribers and videos, and implement data analytics models to discover customer behaviour with the goal of maximizing user joy. The processed data is typically stored as data warehouse tables in AWS S3.

Latency

Latency Storage Big Data Tuning

Expanding the Cloud: Introducing Amazon QuickSight

All Things Distributed

OCTOBER 7, 2015

However, the data infrastructure to collect, store and process data is geared toward developers (e.g., In AWS’ quest to enable the best data storage options for engineers, we have built several innovative database solutions like Amazon RDS, Amazon RDS for Aurora, Amazon DynamoDB, and Amazon Redshift.

Cloud

Cloud Big Data AWS Analytics

Expanding the Cloud - Amazon S3 Reduced Redundancy Storage.

All Things Distributed

MAY 18, 2010

Expanding the Cloud - Amazon S3 Reduced Redundancy Storage. Today a new storage option for Amazon S3 has been launched: Amazon S3 Reduced Redundancy Storage (RRS). This new storage option enables customers to reduce their costs by storing non-critical, reproducible data at lower levels of redundancy. Comments ().

Storage

Storage Cloud AWS Scalability

New AWS feature: Run your website from Amazon S3 - All Things.

All Things Distributed

FEBRUARY 17, 2011

Since a few days ago this weblog serves 100% of its content directly out of the Amazon Simple Storage Service (S3) without the need for a web server to be involved. Driving Storage Costs Down for AWS Customers. Expanding the Cloud - The AWS Storage Gateway. Job Openings in AWS - Senior Leader in Database Services.

AWS

AWS Website Storage Servers

In-Stream Big Data Processing

Kubernetes in the wild report 2023

Trending Sources

What is a Distributed Storage System

Driving down the cost of Big-Data analytics - All Things Distributed

The Need for Real-Time Device Tracking

Expanding the Cloud ? Managing Cold Storage with Amazon Glacier

Why MySQL Could Be Slow With Large Tables

Job Openings in AWS - Senior Leader in Database Services - All.

Delta: A Data Synchronization and Enrichment Platform

Expanding the Cloud - AWS Import/Export Support for Amazon EBS.

Track Thousands of Assets in a Time of Crisis Using Real-Time Digital Twins

Track Thousands of Assets in a Time of Crisis Using Real-Time Digital Twins

No Server Required - Jekyll & Amazon S3 - All Things Distributed

Hacking with AWS at The Next Web Hackaton - All Things Distributed

Redis vs Memcached in 2024

Expanding the Cloud - Introducing Amazon ElastiCache - All Things.

Introducing the AWS South America - All Things Distributed

The AWS GovCloud (US) Region - All Things Distributed

Expanding the Cloud - Introducing the AWS Asia Pacific (Tokyo.

Driving Bandwidth Cost Down for AWS Customers. - All Things.

Free at Last - A Fully Self-Sustained Blog Running in Amazon S3.

AWS Elastic Beanstalk: A Quick and Simple Way into the Cloud - All.

DROAM - Dreaming about Cheap Data Roaming - All Things.

From the Archives - Gapingvoid's Nobody Cares - All Things.

Reboot - All Things Distributed

Spot Instances - Increased Control - All Things Distributed

New Route 53 and ELB features: IPv6, Zone Apex, WRR and more.

APAC Summer Tour - All Things Distributed

Around the World in 28 Days - All Things Distributed

Expanding the Cloud - Cluster Compute Instances for Amazon EC2.

I am looking for new application and platform services - All Things.

This week in review: GPUs, Zombies, Biomimicry and Tom Waits.

Expanding the Cloud with DNS - Introducing Amazon Route 53 - All.

Expanding the Cloud - New AWS Region: US-West (Northern.

Data lakehouse innovations advance the three pillars of observability for more collaborative analytics

Mastering Hybrid Cloud Strategy

Powerful New Amazon EC2 Boot Features - All Things Distributed

Expanding the Cloud - Opening the AWS Asia Pacific (Singapore.

The Amazon.com 2010 Shareholder Letter Focusses on Technology.

Expanding the Cloud - Amazon EC2 Spot Instances - All Things.

Bulldozer: Batch Data Moving from Data Warehouse to Online Key-Value Stores

Expanding the Cloud: Introducing Amazon QuickSight

Expanding the Cloud - Amazon S3 Reduced Redundancy Storage.

New AWS feature: Run your website from Amazon S3 - All Things.

Stay Connected