Big Data, Database and Scalability - Technology Performance Pulse

What is Greenplum Database? Intro to the Big Data Database

Scalegrid

MAY 13, 2020

Greenplum Database is a massively parallel processing (MPP) SQL database that is built and based on PostgreSQL. It can scale towards a multi-petabyte level data workload without a single issue, and it allows access to a cluster of powerful servers that will work together within a single SQL interface where you can view all of the data.

Big Data

Big Data Database Artificial Intelligence Open Source

What Should You Know About Graph Database’s Scalability?

DZone

JANUARY 20, 2023

Having a distributed and scalable graph database system is highly sought after in many enterprise scenarios. Do Not Be Misled Designing and implementing a scalable graph database system has never been a trivial task.

Scalability

Scalability Big Data Hardware Internet

In-Stream Big Data Processing

Highly Scalable

AUGUST 20, 2013

The shortcomings and drawbacks of batch-oriented data processing were widely recognized by the Big Data community quite a long time ago. It is clear that distributed in-stream data processing has something to do with query processing in distributed relational databases. Basics of Distributed Query Processing.

Big Data

Big Data Processing Lambda Database

What is a Distributed Storage System

Scalegrid

FEBRUARY 8, 2024

Key Takeaways Distributed storage systems benefit organizations by enhancing data availability, fault tolerance, and system scalability, leading to cost savings from reduced hardware needs, energy consumption, and personnel. Variations within these storage systems are called distributed file systems.

Storage

Storage Systems Big Data Azure

What is IT operations analytics? Extract more data insights from more sources

Dynatrace

MAY 1, 2023

Then, big data analytics technologies, such as Hadoop, NoSQL, Spark, or Grail, the Dynatrace data lakehouse technology, interpret this information. Here are the six steps of a typical ITOA process : Define the data infrastructure strategy. Identify data use cases and develop a scalable delivery model with documentation.

Analytics

Analytics Artificial Intelligence Big Data Open Source

Redis vs Memcached in 2024

Scalegrid

MARCH 28, 2024

In this comparison of Redis vs Memcached, we strip away the complexity, focusing on each in-memory data store’s performance, scalability, and unique features. Redis is better suited for complex data models, and Memcached is better suited for high-throughput, string-based caching scenarios. Data transfer technology.

Cache

Cache Storage Scalability Architecture

Kubernetes in the wild report 2023

Dynatrace

JANUARY 16, 2023

The strongest Kubernetes growth areas are security, databases, and CI/CD technologies. Through effortless provisioning, a larger number of small hosts provide a cost-effective and scalable platform. Strongest Kubernetes growth areas are security, databases, and CI/CD technologies. Java, Go, and Node.js

Open Source

Open Source Java Operating System Programming

What is cloud monitoring? How to improve your full-stack visibility

Dynatrace

JANUARY 11, 2023

As cloud and big data complexity scales beyond the ability of traditional monitoring tools to handle, next-generation cloud monitoring and observability are becoming necessities for IT teams. Database monitoring. This ensures the database queries are performant, while also identifying host problems. Website monitoring.

Cloud

Cloud Monitoring Best Practices Infrastructure

Mastering Hybrid Cloud Strategy

Scalegrid

MARCH 14, 2024

This approach allows companies to combine the security and control of private clouds with public clouds’ scalability and innovation potential. Ready to take your database management to the next level with ScaleGrid’s cutting-edge solutions? A hybrid cloud strategy could be your answer.

Strategy

Strategy Cloud Artificial Intelligence Infrastructure

Driving down the cost of Big-Data analytics - All Things Distributed

All Things Distributed

AUGUST 18, 2011

Werner Vogels weblog on building scalable and robust distributed systems. Driving down the cost of Big-Data analytics. The Amazon Elastic MapReduce (EMR) team announced today the ability to seamlessly use Amazon EC2 Spot Instances with their service, significantly driving down the cost of data analytics in the cloud.

Big Data

Big Data Analytics AWS Cloud

Conducting log analysis with an observability platform and full data context

Dynatrace

APRIL 20, 2023

With the extent of observability data going beyond human capacity to manage, Grail is the first purpose-built causational data lakehouse that allows for immediate answers with cost-efficient, scalable storage. With Grail, teams can choose what’s relevant at any time to ensure data is always on hand. .

Analytics

Analytics Infrastructure Storage Efficiency

The Need for Real-Time Device Tracking

ScaleOut Software

JULY 19, 2021

Incoming data is saved into data storage (historian database or log store) for query by operational managers who must attempt to find the highest priority issues that require their attention. The best they can usually do in real-time using general purpose tools is to filter and look for patterns of interest.

IoT

IoT Analytics Big Data Architecture

Job Openings in AWS - Senior Leader in Database Services - All.

All Things Distributed

AUGUST 19, 2011

Werner Vogels weblog on building scalable and robust distributed systems. Job Openings in AWS - Senior Leader in Database Services. This week it is an opening for senior leaders with AWS Database Services. This week it is an opening for senior leaders with AWS Database Services. All Things Distributed. Comments ().

AWS

AWS Database Storage Scalability

Should You Use ClickHouse as a Main Operational Database?

Percona

JANUARY 14, 2019

What if we use ClickHouse (which is a columnar analytical database) as our main datastore? Well, typically, an analytical database is not a replacement for a transactional or key/value datastore. However, ClickHouse is super efficient for timeseries and provides “sharding” out of the box (scalability beyond one node).

Database

Database Analytics Blockchain Healthcare

Use Digital Twins for the Next Generation in Telematics

ScaleOut Software

NOVEMBER 24, 2020

Real-Time Digital Twins Can Add Important New Capabilities to Telematics Systems and Eliminate Scalability Bottlenecks. Every few seconds, the application servers collect batches of snapshots and write them to the database where they can be queried by dispatchers managing the fleet.

Analytics

Analytics Architecture Scalability Software Architecture

Expanding the Cloud – An AWS Region is coming to Hong Kong

All Things Distributed

JUNE 20, 2017

After the launch of the AWS APAC (Hong Kong) Region, there will be 19 Availability Zones in Asia Pacific for customers to build flexible, scalable, secure, and highly available applications. In 2010, we opened our first AWS Region in Singapore and since then have opened additional regions: Japan, Australia, China, Korea, and India.

AWS

AWS Logistics Cloud Social Media

NoSQL Data Modeling Techniques

Highly Scalable

MARCH 1, 2012

NoSQL databases are often compared by various non-functional criteria, such as scalability, performance, and consistency. At the same time, NoSQL data modeling is not so well studied and lacks the systematic theory found in relational databases. The second one is database-managed indexes, at least in some implementations.

Database

Database Ecommerce Efficiency Engineering

How LinkedIn Serves Over 4.8 Million Member Profiles per Second

InfoQ

JULY 3, 2023

LinkedIn introduced Couchbase as a centralized caching tier for scaling member profile reads to handle increasing traffic that has outgrown their existing database cluster. The new solution achieved over 99% hit rate, helped reduce tail latencies by more than 60% and costs by 10% annually. By Rafal Gancarz

Cache

Cache Latency Traffic Database

Track Thousands of Assets in a Time of Crisis Using Real-Time Digital Twins

ScaleOut Software

APRIL 3, 2020

An innovative new software approach called “real-time digital twins” running on a cloud-hosted, highly scalable, in-memory computing platform can help address this challenge. These questions can be answered using the latest data as it streams in from the field. What are real-time digital twins and why are they useful here?

Logistics

Logistics Analytics Scalability Cloud

Track Thousands of Assets in a Time of Crisis Using Real-Time Digital Twins

ScaleOut Software

APRIL 3, 2020

An innovative new software approach called “real-time digital twins” running on a cloud-hosted, highly scalable, in-memory computing platform can help address this challenge. These questions can be answered using the latest data as it streams in from the field. What are real-time digital twins and why are they useful here?

Logistics

Logistics Analytics Scalability Cloud

Expanding the Cloud - Introducing Amazon ElastiCache - All Things.

All Things Distributed

AUGUST 22, 2011

Werner Vogels weblog on building scalable and robust distributed systems. There are many success stories about the effectiveness of caching in many different scenarios; next to helping applications achieving fast and predictable performance, it often protects databases from requests bursts and brownouts under overload conditions.

Cloud

Cloud Cache AWS Storage

The AWS GovCloud (US) Region - All Things Distributed

All Things Distributed

AUGUST 16, 2011

Werner Vogels weblog on building scalable and robust distributed systems. Government and Big Data. One particular early use case for AWS GovCloud (US) will be massive data processing and analytics. The scalability, flexibility and the elasticity of AWS makes it an ideal environment for the agencies to run their analytics.

AWS

AWS Government Big Data Cloud

Fast key-value stores: an idea whose time has come and gone

The Morning Paper

JUNE 23, 2019

Coupled with stateless application servers to execute business logic and a database-like system to provide persistent storage, they form a core component of popular data center service archictectures. We’ve seen similar high marshalling overheads in big data systems too.) Fetching too much data in a single query (i.e.,

Cache

Cache Latency Google Lambda

Register for AWS re: Invent - All Things Distributed

All Things Distributed

JULY 16, 2012

Werner Vogels weblog on building scalable and robust distributed systems. There are sessions in many different categories: Architecture, Big Data, HPC, Computer & Networking, Storage, Databases, Security, Tools & Languages, Media Sharing & Content Delivery, Managing AWS Resources, Enterprise IT, Mobile, Start-up, and more.

AWS

AWS Big Data Media Storage

Introducing the AWS South America - All Things Distributed

All Things Distributed

DECEMBER 14, 2011

Werner Vogels weblog on building scalable and robust distributed systems. a Fast and Scalable NoSQL Database Service Designed for Internet Scale Applications. Job Openings in AWS - Senior Leader in Database Services. Driving down the cost of Big-Data analytics. All Things Distributed. Comments ().

AWS

AWS Latency Storage Big Data

Even more amazing papers at VLDB 2019 (that I didn’t have space to cover yet)

The Morning Paper

SEPTEMBER 19, 2019

MongoDB is an important database, and this paper explains the tunable (per-operation) consistency models that MongoDB provides and how they are implemented under the covers. Microsoft have a paper describing their new recovery mechanism in Azure SQL Database , the key feature being that it can recovery in constant time.

Blockchain

Blockchain Hardware Google Analytics

Expanding the Cloud - AWS Import/Export Support for Amazon EBS.

All Things Distributed

JULY 7, 2011

Werner Vogels weblog on building scalable and robust distributed systems. a Fast and Scalable NoSQL Database Service Designed for Internet Scale Applications. Job Openings in AWS - Senior Leader in Database Services. Driving down the cost of Big-Data analytics. All Things Distributed. Comments ().

AWS

AWS Cloud Storage Internet

AWS Elastic Beanstalk: A Quick and Simple Way into the Cloud - All.

All Things Distributed

JANUARY 19, 2011

Werner Vogels weblog on building scalable and robust distributed systems. Flexibility is one of the key principles of Amazon Web Services - developers can select any programming language and software package, any operating system, any middleware and any database to build systems and applications that meet their requirements.

AWS

AWS Cloud Java Operating System

No Server Required - Jekyll & Amazon S3 - All Things Distributed

All Things Distributed

AUGUST 17, 2011

Werner Vogels weblog on building scalable and robust distributed systems. If you have a largely static site you can rely on the enormous power of S3 to make serving your content highly scalable and storing it extremely durable. a Fast and Scalable NoSQL Database Service Designed for Internet Scale Applications.

Servers

Servers Social Media AWS Website

Expanding the Cloud - Introducing the AWS Asia Pacific (Tokyo.

All Things Distributed

MARCH 2, 2011

Werner Vogels weblog on building scalable and robust distributed systems. a Fast and Scalable NoSQL Database Service Designed for Internet Scale Applications. Job Openings in AWS - Senior Leader in Database Services. Driving down the cost of Big-Data analytics. All Things Distributed. Comments ().

AWS

AWS Cloud Games Latency

Driving Bandwidth Cost Down for AWS Customers. - All Things.

All Things Distributed

JUNE 29, 2011

Werner Vogels weblog on building scalable and robust distributed systems. a Fast and Scalable NoSQL Database Service Designed for Internet Scale Applications. Job Openings in AWS - Senior Leader in Database Services. Driving down the cost of Big-Data analytics. All Things Distributed. Comments ().

AWS

AWS Retail Innovation Strategy

Hacking with AWS at The Next Web Hackaton - All Things Distributed

All Things Distributed

MARCH 24, 2011

Werner Vogels weblog on building scalable and robust distributed systems. It is likely that the Amazon Web Services will be used by many of the participants for their compute, storage, database and other cloud resource needs. a Fast and Scalable NoSQL Database Service Designed for Internet Scale Applications. Comments ().

AWS

AWS Internet Internet Storage

Free at Last - A Fully Self-Sustained Blog Running in Amazon S3.

All Things Distributed

FEBRUARY 23, 2011

Werner Vogels weblog on building scalable and robust distributed systems. a Fast and Scalable NoSQL Database Service Designed for Internet Scale Applications. Job Openings in AWS - Senior Leader in Database Services. Driving down the cost of Big-Data analytics. All Things Distributed. Comments ().

AWS

AWS Storage Big Data Servers

DROAM - Dreaming about Cheap Data Roaming - All Things.

All Things Distributed

JANUARY 11, 2011

Werner Vogels weblog on building scalable and robust distributed systems. DROAM - Dreaming about Cheap Data Roaming. The one thing that I have always struggled with during my travels are the data plans of the cell phone companies. a Fast and Scalable NoSQL Database Service Designed for Internet Scale Applications.

Wireless

Wireless AWS Internet Internet

New Route 53 and ELB features: IPv6, Zone Apex, WRR and more.

All Things Distributed

MAY 24, 2011

Werner Vogels weblog on building scalable and robust distributed systems. a Fast and Scalable NoSQL Database Service Designed for Internet Scale Applications. Job Openings in AWS - Senior Leader in Database Services. Driving down the cost of Big-Data analytics. All Things Distributed. Comments ().

Internet

Internet Internet AWS Scalability

From the Archives - Gapingvoid's Nobody Cares - All Things.

All Things Distributed

FEBRUARY 27, 2011

Werner Vogels weblog on building scalable and robust distributed systems. a Fast and Scalable NoSQL Database Service Designed for Internet Scale Applications. Job Openings in AWS - Senior Leader in Database Services. Driving down the cost of Big-Data analytics. All Things Distributed. Comments ().

AWS

AWS Storage Big Data Scalability

Reboot - All Things Distributed

All Things Distributed

SEPTEMBER 29, 2010

Werner Vogels weblog on building scalable and robust distributed systems. a Fast and Scalable NoSQL Database Service Designed for Internet Scale Applications. Job Openings in AWS - Senior Leader in Database Services. Driving down the cost of Big-Data analytics. All Things Distributed. Comments ().

AWS

AWS Storage Big Data Scalability

Spot Instances - Increased Control - All Things Distributed

All Things Distributed

JULY 11, 2011

Werner Vogels weblog on building scalable and robust distributed systems. a Fast and Scalable NoSQL Database Service Designed for Internet Scale Applications. Job Openings in AWS - Senior Leader in Database Services. Driving down the cost of Big-Data analytics. All Things Distributed. Comments ().

AWS

AWS Storage Cloud Big Data

APAC Summer Tour - All Things Distributed

All Things Distributed

JULY 3, 2011

Werner Vogels weblog on building scalable and robust distributed systems. a Fast and Scalable NoSQL Database Service Designed for Internet Scale Applications. Job Openings in AWS - Senior Leader in Database Services. Driving down the cost of Big-Data analytics. All Things Distributed. APAC Summer Tour.

AWS

AWS Storage Big Data Cloud

Expanding the Cloud ? Managing Cold Storage with Amazon Glacier

All Things Distributed

AUGUST 20, 2012

Werner Vogels weblog on building scalable and robust distributed systems. Data is retrieved by scheduling a job, which typically completes within 3 to 5 hours. Amazon Glacier integrates seamlessly with other AWS services such as Amazon S3 and the different AWS Database services. All Things Distributed. Expanding the Cloud â??

Storage

Storage Cloud AWS Media

I am looking for new application and platform services - All Things.

All Things Distributed

APRIL 23, 2010

Werner Vogels weblog on building scalable and robust distributed systems. a Fast and Scalable NoSQL Database Service Designed for Internet Scale Applications. Job Openings in AWS - Senior Leader in Database Services. Driving down the cost of Big-Data analytics. All Things Distributed. Comments ().

AWS

AWS Storage Cloud Big Data

Around the World in 28 Days - All Things Distributed

All Things Distributed

SEPTEMBER 30, 2010

Werner Vogels weblog on building scalable and robust distributed systems. a Fast and Scalable NoSQL Database Service Designed for Internet Scale Applications. Job Openings in AWS - Senior Leader in Database Services. Driving down the cost of Big-Data analytics. All Things Distributed. Comments ().

AWS

AWS Storage Cloud Best Practices

Expanding the Cloud - Cluster Compute Instances for Amazon EC2.

All Things Distributed

JULY 13, 2010

Werner Vogels weblog on building scalable and robust distributed systems. Cluster Compute Instances for Amazon EC2 change the HPC game for two important reasons: Access to scalable on-demand capacity. a Fast and Scalable NoSQL Database Service Designed for Internet Scale Applications. All Things Distributed. Comments ().

Cloud

Cloud AWS Automotive Latency

Expanding the Cloud with DNS - Introducing Amazon Route 53 - All.

All Things Distributed

DECEMBER 5, 2010

Werner Vogels weblog on building scalable and robust distributed systems. Often these namespaces are hierarchical in nature such that it becomes easier to manage them and to decentralize control, which makes the system more scalable. a Fast and Scalable NoSQL Database Service Designed for Internet Scale Applications.

Cloud

Cloud Internet Internet AWS

What is Greenplum Database? Intro to the Big Data Database

What Should You Know About Graph Database’s Scalability?

Trending Sources

In-Stream Big Data Processing

What is a Distributed Storage System

What is IT operations analytics? Extract more data insights from more sources

Redis vs Memcached in 2024

Kubernetes in the wild report 2023

What is cloud monitoring? How to improve your full-stack visibility

Mastering Hybrid Cloud Strategy

Driving down the cost of Big-Data analytics - All Things Distributed

Conducting log analysis with an observability platform and full data context

The Need for Real-Time Device Tracking

Job Openings in AWS - Senior Leader in Database Services - All.

Should You Use ClickHouse as a Main Operational Database?

Use Digital Twins for the Next Generation in Telematics

Expanding the Cloud – An AWS Region is coming to Hong Kong

NoSQL Data Modeling Techniques

How LinkedIn Serves Over 4.8 Million Member Profiles per Second

Track Thousands of Assets in a Time of Crisis Using Real-Time Digital Twins

Track Thousands of Assets in a Time of Crisis Using Real-Time Digital Twins

Expanding the Cloud - Introducing Amazon ElastiCache - All Things.

The AWS GovCloud (US) Region - All Things Distributed

Fast key-value stores: an idea whose time has come and gone

Register for AWS re: Invent - All Things Distributed

Introducing the AWS South America - All Things Distributed

Even more amazing papers at VLDB 2019 (that I didn’t have space to cover yet)

Expanding the Cloud - AWS Import/Export Support for Amazon EBS.

AWS Elastic Beanstalk: A Quick and Simple Way into the Cloud - All.

No Server Required - Jekyll & Amazon S3 - All Things Distributed

Expanding the Cloud - Introducing the AWS Asia Pacific (Tokyo.

Driving Bandwidth Cost Down for AWS Customers. - All Things.

Hacking with AWS at The Next Web Hackaton - All Things Distributed

Free at Last - A Fully Self-Sustained Blog Running in Amazon S3.

DROAM - Dreaming about Cheap Data Roaming - All Things.

New Route 53 and ELB features: IPv6, Zone Apex, WRR and more.

From the Archives - Gapingvoid's Nobody Cares - All Things.

Reboot - All Things Distributed

Spot Instances - Increased Control - All Things Distributed

APAC Summer Tour - All Things Distributed

Expanding the Cloud ? Managing Cold Storage with Amazon Glacier

I am looking for new application and platform services - All Things.

Around the World in 28 Days - All Things Distributed

Expanding the Cloud - Cluster Compute Instances for Amazon EC2.

Expanding the Cloud with DNS - Introducing Amazon Route 53 - All.

Stay Connected