Big Data, Database, Systems and Technology - Technology Performance Pulse

In-Stream Big Data Processing

Highly Scalable

AUGUST 20, 2013

The shortcomings and drawbacks of batch-oriented data processing were widely recognized by the Big Data community quite a long time ago. This system has been designed to supplement and succeed the existing Hadoop-based system that had too high latency of data processing and too high maintenance costs.

Big Data

Big Data Processing Lambda Database

Kubernetes in the wild report 2023

Dynatrace

JANUARY 16, 2023

While Kubernetes is still a relatively young technology, a large majority of global enterprises use it to run business-critical applications in production. Findings provide insights into Kubernetes practitioners’ infrastructure preferences and how they use advanced Kubernetes platform technologies. Java, Go, and Node.js

Open Source

Open Source Java Operating System Programming

Driving down the cost of Big-Data analytics - All Things Distributed

All Things Distributed

AUGUST 18, 2011

Werner Vogels weblog on building scalable and robust distributed systems. Driving down the cost of Big-Data analytics. The Amazon Elastic MapReduce (EMR) team announced today the ability to seamlessly use Amazon EC2 Spot Instances with their service, significantly driving down the cost of data analytics in the cloud.

Big Data

Big Data Analytics AWS Cloud

A guide to Autonomous Performance Optimization

Dynatrace

SEPTEMBER 15, 2020

Stefano started his presentation by showing how much cost and performance optimization is possible when knowing how to properly configure your application runtimes, databases, or cloud environments: Correct configuration of JVM parameters can save up to 75% resource utilization while delivering same or better performance!

Performance

Performance Java Metrics Cloud

What is a Distributed Storage System

Scalegrid

FEBRUARY 8, 2024

A distributed storage system is foundational in today’s data-driven landscape, ensuring data spread over multiple servers is reliable, accessible, and manageable. This guide delves into how these systems work, the challenges they solve, and their essential role in businesses and technology.

Storage

Storage Systems Big Data Azure

Seer: leveraging big data to navigate the complexity of performance debugging in cloud microservices

The Morning Paper

MAY 14, 2019

Seer: leveraging big data to navigate the complexity of performance debugging in cloud microservices Gan et al., Seer is an online system that observes the behaviour of cloud applications (using the DeathStarBench microservices for the evaluation) and predicts when QoS violations may be about to occur. ASPLOS’19.

Big Data

Big Data Cloud Performance Hardware

The Need for Real-Time Device Tracking

ScaleOut Software

JULY 19, 2021

How are we managing the torrent of telemetry that flows into analytics systems from these devices? Incoming data is saved into data storage (historian database or log store) for query by operational managers who must attempt to find the highest priority issues that require their attention. The list goes on.

IoT

IoT Analytics Big Data Architecture

Job Openings in AWS - Senior Leader in Database Services - All.

All Things Distributed

AUGUST 19, 2011

Werner Vogels weblog on building scalable and robust distributed systems. Job Openings in AWS - Senior Leader in Database Services. This week it is an opening for senior leaders with AWS Database Services. The ideal candidate will be someone who has built and ran large scale distributed systems and/or databases.

AWS

AWS Database Storage Scalability

DynamoDB for Location Data: Geospatial querying on DynamoDB datasets

All Things Distributed

SEPTEMBER 5, 2013

Over the past few years, two important trends that have been disrupting the database industry are mobile applications and big data. The explosive growth in mobile devices and mobile apps is generating a huge amount of data, which has fueled the demand for big data services and for high scale databases.

Big Data

Big Data Mobile Latency Database

Delta: A Data Synchronization and Enrichment Platform

The Netflix TechBlog

OCTOBER 15, 2019

Another issue exists for the capture of schema changes, where some systems, like MySQL, don’t support transactional schema changes [1][2]. By their nature, they can only rely on the lowest common denominator of participating systems. That system quickly grew very complex and became difficult to maintain.

Transportation

Transportation Architecture Processing Storage

Expanding the Cloud – An AWS Region is coming to Hong Kong

All Things Distributed

JUNE 20, 2017

The new region will give Hong Kong-based businesses, government organizations, non-profits, and global companies with customers in Hong Kong, the ability to leverage AWS technologies from data centers in Hong Kong. The new AWS Asia Pacific (Hong Kong) Region will have three Availability Zones and be ready for customers for use in 2018.

AWS

AWS Logistics Cloud Social Media

What is IT operations analytics? Extract more data insights from more sources

Dynatrace

MAY 1, 2023

Additionally, ITOA gathers and processes information from applications, services, networks, operating systems, and cloud infrastructure hardware logs in real time. Then, big data analytics technologies, such as Hadoop, NoSQL, Spark, or Grail, the Dynatrace data lakehouse technology, interpret this information.

Analytics

Analytics Artificial Intelligence Big Data Open Source

Track Thousands of Assets in a Time of Crisis Using Real-Time Digital Twins

ScaleOut Software

APRIL 3, 2020

What’s missing is a flexible, fast, and easy-to-use software system that can be quickly adapted to track these assets in real time and provide immediate answers for logistics managers. Within seconds, the software performs aggregate analysis of this data for all real-time digital twins.

Logistics

Logistics Analytics Scalability Cloud

Track Thousands of Assets in a Time of Crisis Using Real-Time Digital Twins

ScaleOut Software

APRIL 3, 2020

What’s missing is a flexible, fast, and easy-to-use software system that can be quickly adapted to track these assets in real time and provide immediate answers for logistics managers. Within seconds, the software performs aggregate analysis of this data for all real-time digital twins.

Logistics

Logistics Analytics Scalability Cloud

Use Digital Twins for the Next Generation in Telematics

ScaleOut Software

NOVEMBER 24, 2020

Real-Time Digital Twins Can Add Important New Capabilities to Telematics Systems and Eliminate Scalability Bottlenecks. The volume of incoming telemetry challenges current telematics systems to keep up and quickly make sense of all the data. Current Telematics Architecture.

Analytics

Analytics Architecture Scalability Software Architecture

AWS Elastic Beanstalk: A Quick and Simple Way into the Cloud - All.

All Things Distributed

JANUARY 19, 2011

Werner Vogels weblog on building scalable and robust distributed systems. Flexibility is one of the key principles of Amazon Web Services - developers can select any programming language and software package, any operating system, any middleware and any database to build systems and applications that meet their requirements.

AWS

AWS Cloud Java Operating System

Expanding the Cloud - Introducing Amazon ElastiCache - All Things.

All Things Distributed

AUGUST 22, 2011

Werner Vogels weblog on building scalable and robust distributed systems. There are many success stories about the effectiveness of caching in many different scenarios; next to helping applications achieving fast and predictable performance, it often protects databases from requests bursts and brownouts under overload conditions.

Cloud

Cloud Cache AWS Storage

Expanding the Cloud - AWS Import/Export Support for Amazon EBS.

All Things Distributed

JULY 7, 2011

Werner Vogels weblog on building scalable and robust distributed systems. Amazon Import/Export is an important tool for customers to accelerate moving large amounts of data into the AWS storage systems. a Fast and Scalable NoSQL Database Service Designed for Internet Scale Applications. All Things Distributed.

AWS

AWS Cloud Storage Internet

Hacking with AWS at The Next Web Hackaton - All Things Distributed

All Things Distributed

MARCH 24, 2011

Werner Vogels weblog on building scalable and robust distributed systems. Over the past years The Next Web Conference has become a premier conference on internet life and its technologies. a Fast and Scalable NoSQL Database Service Designed for Internet Scale Applications. Driving down the cost of Big-Data analytics.

AWS

AWS Internet Internet Storage

Introducing the AWS South America - All Things Distributed

All Things Distributed

DECEMBER 14, 2011

Werner Vogels weblog on building scalable and robust distributed systems. a Fast and Scalable NoSQL Database Service Designed for Internet Scale Applications. Job Openings in AWS - Senior Leader in Database Services. Driving down the cost of Big-Data analytics. All Things Distributed. Expanding the Cloud â??

AWS

AWS Latency Storage Big Data

The AWS GovCloud (US) Region - All Things Distributed

All Things Distributed

AUGUST 16, 2011

Werner Vogels weblog on building scalable and robust distributed systems. Government and Big Data. One particular early use case for AWS GovCloud (US) will be massive data processing and analytics. AWS GovCloud (US) will be used by several of these agencies to help them with their Bigger-than-Big-Data needs.

AWS

AWS Government Big Data Cloud

Expanding the Cloud ? Managing Cold Storage with Amazon Glacier

All Things Distributed

AUGUST 20, 2012

Werner Vogels weblog on building scalable and robust distributed systems. From selecting the right technology, to maintaining multi-site facilities, to dealing with exponential and often unpredictable growth, to ensuring long-term digital integrity, digital archiving can be a major headache. All Things Distributed. Comments ().

Storage

Storage Cloud AWS Media

Expanding the Cloud with DNS - Introducing Amazon Route 53 - All.

All Things Distributed

DECEMBER 5, 2010

Werner Vogels weblog on building scalable and robust distributed systems. I am very excited that today we have launched Amazon Route 53, a high-performance and highly-available Domain Name System (DNS) service. Naming is one of the fundamental concepts in Distributed Systems. By Werner Vogels on 05 December 2010 02:00 PM.

Cloud

Cloud Internet Internet AWS

No Server Required - Jekyll & Amazon S3 - All Things Distributed

All Things Distributed

AUGUST 17, 2011

Werner Vogels weblog on building scalable and robust distributed systems. Amazon S3 is much more than just storage; the network and distributed systems infrastructure to ensure that content can be served fast and at high rates without customers impacting each other, is amazing. Job Openings in AWS - Senior Leader in Database Services.

Servers

Servers Social Media AWS Website

Expanding the Cloud - Introducing the AWS Asia Pacific (Tokyo.

All Things Distributed

MARCH 2, 2011

Werner Vogels weblog on building scalable and robust distributed systems. a Fast and Scalable NoSQL Database Service Designed for Internet Scale Applications. Job Openings in AWS - Senior Leader in Database Services. Driving down the cost of Big-Data analytics. All Things Distributed. Comments (). APAC Summer Tour.

AWS

AWS Cloud Games Latency

Driving Bandwidth Cost Down for AWS Customers. - All Things.

All Things Distributed

JUNE 29, 2011

Werner Vogels weblog on building scalable and robust distributed systems. a Fast and Scalable NoSQL Database Service Designed for Internet Scale Applications. Job Openings in AWS - Senior Leader in Database Services. Driving down the cost of Big-Data analytics. All Things Distributed. Comments (). APAC Summer Tour.

AWS

AWS Retail Innovation Strategy

Free at Last - A Fully Self-Sustained Blog Running in Amazon S3.

All Things Distributed

FEBRUARY 23, 2011

Werner Vogels weblog on building scalable and robust distributed systems. a Fast and Scalable NoSQL Database Service Designed for Internet Scale Applications. Job Openings in AWS - Senior Leader in Database Services. Driving down the cost of Big-Data analytics. All Things Distributed. Comments (). APAC Summer Tour.

AWS

AWS Storage Big Data Servers

Expanding the Cloud - Cluster Compute Instances for Amazon EC2.

All Things Distributed

JULY 13, 2010

Werner Vogels weblog on building scalable and robust distributed systems. During my academic career, I spent many years working on HPC technologies such as user-level networking interfaces, large scale high-speed interconnects, HPC software stacks, etc. Not just for HPC but for mission critical enterprise systems such as OLTP.

Cloud

Cloud AWS Automotive Latency

Redis vs Memcached in 2024

Scalegrid

MARCH 28, 2024

Advanced Redis Features Showdown Big data center concept, cloud database, server power station of the future. Data transfer technology. Cube or box Block chain of abstract financial data. Elevating Redis with ScaleGrid’s Managed DBaaS Database management can be intricate.

Cache

Cache Storage Scalability Architecture

40+ Best Web Development Blogs of 2018

KeyCDN

OCTOBER 2, 2018

New technologies, libraries, and techniques are continuously emerging and if you want to stay on top of all these changes you will need to immerse yourself in communities, groups, and conferences all revolving around web development. Use it to find inspiration, learn how to pitch projects or simply keep up with new technologies.

Development

Development Website Design Code

From the Archives - Gapingvoid's Nobody Cares - All Things.

All Things Distributed

FEBRUARY 27, 2011

Werner Vogels weblog on building scalable and robust distributed systems. a Fast and Scalable NoSQL Database Service Designed for Internet Scale Applications. Job Openings in AWS - Senior Leader in Database Services. Driving down the cost of Big-Data analytics. All Things Distributed. Comments (). APAC Summer Tour.

AWS

AWS Storage Big Data Scalability

DROAM - Dreaming about Cheap Data Roaming - All Things.

All Things Distributed

JANUARY 11, 2011

Werner Vogels weblog on building scalable and robust distributed systems. DROAM - Dreaming about Cheap Data Roaming. The one thing that I have always struggled with during my travels are the data plans of the cell phone companies. a Fast and Scalable NoSQL Database Service Designed for Internet Scale Applications.

Wireless

Wireless AWS Internet Internet

Reboot - All Things Distributed

All Things Distributed

SEPTEMBER 29, 2010

Werner Vogels weblog on building scalable and robust distributed systems. a Fast and Scalable NoSQL Database Service Designed for Internet Scale Applications. Job Openings in AWS - Senior Leader in Database Services. Driving down the cost of Big-Data analytics. All Things Distributed. Comments (). APAC Summer Tour.

AWS

AWS Storage Big Data Scalability

Data Movement in Netflix Studio via Data Mesh

The Netflix TechBlog

JULY 26, 2021

This happens at an unprecedented scale and introduces many interesting challenges; one of the challenges is how to provide visibility of Studio data across multiple phases and systems to facilitate operational excellence and empower decision making. End-to-End Schema Evolution Schema is a key component of Data Mesh.

Big Data

Big Data Government Analytics Processing

Spot Instances - Increased Control - All Things Distributed

All Things Distributed

JULY 11, 2011

Werner Vogels weblog on building scalable and robust distributed systems. a Fast and Scalable NoSQL Database Service Designed for Internet Scale Applications. Job Openings in AWS - Senior Leader in Database Services. Driving down the cost of Big-Data analytics. All Things Distributed. Comments (). APAC Summer Tour.

AWS

AWS Storage Cloud Big Data

New Route 53 and ELB features: IPv6, Zone Apex, WRR and more.

All Things Distributed

MAY 24, 2011

Werner Vogels weblog on building scalable and robust distributed systems. a Fast and Scalable NoSQL Database Service Designed for Internet Scale Applications. Job Openings in AWS - Senior Leader in Database Services. Driving down the cost of Big-Data analytics. All Things Distributed. Comments (). APAC Summer Tour.

Internet

Internet Internet AWS Scalability

APAC Summer Tour - All Things Distributed

All Things Distributed

JULY 3, 2011

Werner Vogels weblog on building scalable and robust distributed systems. a Fast and Scalable NoSQL Database Service Designed for Internet Scale Applications. Job Openings in AWS - Senior Leader in Database Services. Driving down the cost of Big-Data analytics. All Things Distributed. APAC Summer Tour.

AWS

AWS Storage Big Data Cloud

Around the World in 28 Days - All Things Distributed

All Things Distributed

SEPTEMBER 30, 2010

Werner Vogels weblog on building scalable and robust distributed systems. Mumbai on October 20 with Patni , Inxs Technologies and India Games. a Fast and Scalable NoSQL Database Service Designed for Internet Scale Applications. Job Openings in AWS - Senior Leader in Database Services. All Things Distributed. Comments ().

AWS

AWS Storage Cloud Best Practices

I am looking for new application and platform services - All Things.

All Things Distributed

APRIL 23, 2010

Werner Vogels weblog on building scalable and robust distributed systems. a Fast and Scalable NoSQL Database Service Designed for Internet Scale Applications. Job Openings in AWS - Senior Leader in Database Services. Driving down the cost of Big-Data analytics. All Things Distributed. Comments (). APAC Summer Tour.

AWS

AWS Storage Cloud Big Data

Mastering Hybrid Cloud Strategy

Scalegrid

MARCH 14, 2024

Understanding Hybrid Cloud Strategy A hybrid cloud merges the capabilities of public and private clouds into a singular, coherent system. This combination allows for the fluid movement of data and applications across different environments, facilitating shared workloads seamlessly. The tool must be compatible with your current systems.

Strategy

Strategy Cloud Artificial Intelligence Infrastructure

Expanding the Cloud - New AWS Region: US-West (Northern.

All Things Distributed

DECEMBER 3, 2009

Werner Vogels weblog on building scalable and robust distributed systems. a Fast and Scalable NoSQL Database Service Designed for Internet Scale Applications. Job Openings in AWS - Senior Leader in Database Services. Driving down the cost of Big-Data analytics. All Things Distributed. Comments (). APAC Summer Tour.

AWS

AWS Cloud Latency Storage

This week in review: GPUs, Zombies, Biomimicry and Tom Waits.

All Things Distributed

NOVEMBER 19, 2010

Werner Vogels weblog on building scalable and robust distributed systems. Big news this week was of course the launch of Cluster GPU instances for Amazon EC2. a Fast and Scalable NoSQL Database Service Designed for Internet Scale Applications. Job Openings in AWS - Senior Leader in Database Services. Comments ().

AWS

AWS Cloud Benchmarking Storage

The Amazon.com 2010 Shareholder Letter Focusses on Technology.

All Things Distributed

APRIL 27, 2011

Werner Vogels weblog on building scalable and robust distributed systems. The Amazon.com 2010 Shareholder Letter Focusses on Technology. In the 2010 Shareholder Letter Jeff Bezos writes about the unique technologies developed at Amazon.com over the years. By Werner Vogels on 27 April 2011 12:51 AM. Comments ().

Technology

Technology Technology AWS Storage

Expanding the Cloud - Opening the AWS Asia Pacific (Singapore.

All Things Distributed

APRIL 28, 2010

Werner Vogels weblog on building scalable and robust distributed systems. There are four main reasons to do so: Performance - For many applications and services, data access latency to end users is important. You need to be able to place your systems in locations where you can minimize the distance to your most important customers.

AWS

AWS Cloud Latency Storage

In-Stream Big Data Processing

Kubernetes in the wild report 2023

Trending Sources

Driving down the cost of Big-Data analytics - All Things Distributed

A guide to Autonomous Performance Optimization

What is a Distributed Storage System

Seer: leveraging big data to navigate the complexity of performance debugging in cloud microservices

The Need for Real-Time Device Tracking

Job Openings in AWS - Senior Leader in Database Services - All.

DynamoDB for Location Data: Geospatial querying on DynamoDB datasets

Delta: A Data Synchronization and Enrichment Platform

Expanding the Cloud – An AWS Region is coming to Hong Kong

What is IT operations analytics? Extract more data insights from more sources

Track Thousands of Assets in a Time of Crisis Using Real-Time Digital Twins

Track Thousands of Assets in a Time of Crisis Using Real-Time Digital Twins

Use Digital Twins for the Next Generation in Telematics

AWS Elastic Beanstalk: A Quick and Simple Way into the Cloud - All.

Expanding the Cloud - Introducing Amazon ElastiCache - All Things.

Expanding the Cloud - AWS Import/Export Support for Amazon EBS.

Hacking with AWS at The Next Web Hackaton - All Things Distributed

Introducing the AWS South America - All Things Distributed

The AWS GovCloud (US) Region - All Things Distributed

Expanding the Cloud ? Managing Cold Storage with Amazon Glacier

Expanding the Cloud with DNS - Introducing Amazon Route 53 - All.

No Server Required - Jekyll & Amazon S3 - All Things Distributed

Expanding the Cloud - Introducing the AWS Asia Pacific (Tokyo.

Driving Bandwidth Cost Down for AWS Customers. - All Things.

Free at Last - A Fully Self-Sustained Blog Running in Amazon S3.

Expanding the Cloud - Cluster Compute Instances for Amazon EC2.

Redis vs Memcached in 2024

40+ Best Web Development Blogs of 2018

From the Archives - Gapingvoid's Nobody Cares - All Things.

DROAM - Dreaming about Cheap Data Roaming - All Things.

Reboot - All Things Distributed

Data Movement in Netflix Studio via Data Mesh

Spot Instances - Increased Control - All Things Distributed

New Route 53 and ELB features: IPv6, Zone Apex, WRR and more.

APAC Summer Tour - All Things Distributed

Around the World in 28 Days - All Things Distributed

I am looking for new application and platform services - All Things.

Mastering Hybrid Cloud Strategy

Expanding the Cloud - New AWS Region: US-West (Northern.

This week in review: GPUs, Zombies, Biomimicry and Tom Waits.

The Amazon.com 2010 Shareholder Letter Focusses on Technology.

Expanding the Cloud - Opening the AWS Asia Pacific (Singapore.

Stay Connected