All Things Distributed

AUGUST 18, 2011

Werner Vogels weblog on building scalable and robust distributed systems. Driving down the cost of Big-Data analytics. The Amazon Elastic MapReduce (EMR) team announced today the ability to seamlessly use Amazon EC2 Spot Instances with their service, significantly driving down the cost of data analytics in the cloud.

Big Data 112

Big Data Analytics AWS Scalability 112

Scalegrid

MARCH 14, 2024

Understanding Hybrid Cloud Strategy A hybrid cloud merges the capabilities of public and private clouds into a singular, coherent system. This combination allows for the fluid movement of data and applications across different environments, facilitating shared workloads seamlessly. The tool must be compatible with your current systems.

Strategy 130

Strategy Cloud Artificial Intelligence Infrastructure 130

The Morning Paper

MAY 14, 2019

Seer: leveraging big data to navigate the complexity of performance debugging in cloud microservices Gan et al., Seer is an online system that observes the behaviour of cloud applications (using the DeathStarBench microservices for the evaluation) and predicts when QoS violations may be about to occur. ASPLOS’19.

Big Data 45

Big Data Cloud Performance Hardware 45

ScaleOut Software

JULY 19, 2021

How are we managing the torrent of telemetry that flows into analytics systems from these devices? Incoming data is saved into data storage (historian database or log store) for query by operational managers who must attempt to find the highest priority issues that require their attention. The list goes on.

IoT 78

IoT Analytics Big Data Architecture 78

Dynatrace

APRIL 11, 2023

Log4Shell required many organizations to take devices and applications offline to prevent malicious attackers from gaining access to IT systems and sensitive data. As a result, organizations need to be vigilant in identifying and addressing vulnerabilities to protect their systems and data.

Cloud 186

Cloud DevOps Open Source Retail 186

The Morning Paper

OCTOBER 26, 2020

As a production system within Microsoft capturing around a quadrillion events and indexing 16 trillion search keys per day it would be interesting in its own right, but there’s a lot more to it than that. These two narratives of reference architecture and ingestion/indexing system are interwoven throughout the paper.

Cloud 104

Cloud Big Data Latency Architecture 104

Dynatrace

SEPTEMBER 15, 2020

Stefano started his presentation by showing how much cost and performance optimization is possible when knowing how to properly configure your application runtimes, databases, or cloud environments: Correct configuration of JVM parameters can save up to 75% resource utilization while delivering same or better performance!

Performance 185

Performance Java Metrics Cloud 185

Dynatrace

JUNE 29, 2022

A hybrid cloud, however, combines public infrastructure and services with on-premises resources or a private data center to create a flexible, interconnected IT environment. Hybrid environments provide more options for storing and analyzing ever-growing volumes of big data and for deploying digital services.

Infrastructure 181

Infrastructure Cloud Azure AWS 181

All Things Distributed

AUGUST 19, 2011

Werner Vogels weblog on building scalable and robust distributed systems. Job Openings in AWS - Senior Leader in Database Services. This week it is an opening for senior leaders with AWS Database Services. The ideal candidate will be someone who has built and ran large scale distributed systems and/or databases.

AWS 77

AWS Database Storage Scalability 77

Dynatrace

JUNE 1, 2020

The variables that can impact the performance of an application vary; from coding errors or ‘bugs’ in the software, database slowdowns, hosting and network performance, to operating system and device type support. And I’m sure we’ve all experienced frustration when an application crashes, is slow to load, or doesn’t load at all.

Artificial Intelligence 182

Artificial Intelligence Social Media Monitoring IoT 182

Dynatrace

MARCH 11, 2021

I took a big-data-analysis approach, which started with another problem visualization. This is what I wanted to optimize and avoid and many traditional (or homegrown) systems aren’t doing this. This time defines when Dynatrace will invoke the integration to 3 rd party system for a problem after it has been detected.

Tuning 263

Tuning Architecture Monitoring Big Data 263

Dynatrace

JUNE 1, 2020

The variables that can impact the performance of an application vary; from coding errors or ‘bugs’ in the software, database slowdowns, hosting and network performance, to operating system and device type support. And I’m sure we’ve all experienced frustration when an application crashes, is slow to load, or doesn’t load at all.

Monitoring 132

Monitoring Performance Social Media Artificial Intelligence 132

All Things Distributed

SEPTEMBER 5, 2013

Over the past few years, two important trends that have been disrupting the database industry are mobile applications and big data. The explosive growth in mobile devices and mobile apps is generating a huge amount of data, which has fueled the demand for big data services and for high scale databases.

Big Data 100

Big Data Mobile Latency Database 100

The Netflix TechBlog

OCTOBER 15, 2019

Another issue exists for the capture of schema changes, where some systems, like MySQL, don’t support transactional schema changes [1][2]. By their nature, they can only rely on the lowest common denominator of participating systems. That system quickly grew very complex and became difficult to maintain.

Transportation 192

Transportation Architecture Processing Storage 192

Highly Scalable

MARCH 1, 2012

NoSQL databases are often compared by various non-functional criteria, such as scalability, performance, and consistency. At the same time, NoSQL data modeling is not so well studied and lacks the systematic theory found in relational databases. Document databases advance the BigTable model offering two significant improvements.

Database 279

Database Ecommerce Efficiency Engineering 279

ScaleOut Software

NOVEMBER 24, 2020

Real-Time Digital Twins Can Add Important New Capabilities to Telematics Systems and Eliminate Scalability Bottlenecks. The volume of incoming telemetry challenges current telematics systems to keep up and quickly make sense of all the data. Current Telematics Architecture.

Analytics 52

Analytics Architecture Scalability Software Architecture 52

Testsigma

MAY 7, 2020

IBM Big Data and Analytics Hub website cited a case study, where a US insurance company was estimating 15% of their testing efforts to be just test data collection for the backend system and the frontend system. The test data management for the company had become a big problem and had to be solved.

Testing 60

Testing Storage Database Processing 60

The Morning Paper

SEPTEMBER 19, 2019

MongoDB is an important database, and this paper explains the tunable (per-operation) consistency models that MongoDB provides and how they are implemented under the covers. Microsoft have a paper describing their new recovery mechanism in Azure SQL Database , the key feature being that it can recovery in constant time. Yes please!

Blockchain 46

Blockchain Hardware Google Analytics 46

All Things Distributed

JANUARY 6, 2016

Mirae Asset Global Investments improved its web service environment and reduced annual management costs by 50% by consolidating the management of all web services, including servers, network, database, and security. Many of these enterprises are assisted by our extensive partner ecosystem in Korea.

AWS 110

AWS Cloud Games Latency 110

The Morning Paper

JUNE 23, 2019

Coupled with stateless application servers to execute business logic and a database-like system to provide persistent storage, they form a core component of popular data center service archictectures. Why are developers using RInK systems as part of their design? Fetching too much data in a single query (i.e.,

Cache 79

Cache Latency Google Lambda 79

All Things Distributed

JUNE 20, 2017

Beyond running their web properties and applications, Next Digital also uses Amazon RDS (database), Amazon ElastiCache (caching), and Amazon Redshift (data warehousing). Next Digital operates on AWS in a more highly available and fault-tolerant environment than their previous colocation solution.

AWS 149

AWS Logistics Cloud Social Media 149

All Things Distributed

JULY 16, 2012

Werner Vogels weblog on building scalable and robust distributed systems. There are sessions in many different categories: Architecture, Big Data, HPC, Computer & Networking, Storage, Databases, Security, Tools & Languages, Media Sharing & Content Delivery, Managing AWS Resources, Enterprise IT, Mobile, Start-up, and more.

AWS 85

AWS Big Data Media Storage 85

All Things Distributed

AUGUST 22, 2011

Werner Vogels weblog on building scalable and robust distributed systems. There are many success stories about the effectiveness of caching in many different scenarios; next to helping applications achieving fast and predictable performance, it often protects databases from requests bursts and brownouts under overload conditions.

Cloud 117

Cloud Cache AWS Storage 117

All Things Distributed

JANUARY 19, 2011

Werner Vogels weblog on building scalable and robust distributed systems. Flexibility is one of the key principles of Amazon Web Services - developers can select any programming language and software package, any operating system, any middleware and any database to build systems and applications that meet their requirements.

AWS 115

AWS Cloud Java Scalability 115

ScaleOut Software

APRIL 3, 2020

What’s missing is a flexible, fast, and easy-to-use software system that can be quickly adapted to track these assets in real time and provide immediate answers for logistics managers. Within seconds, the software performs aggregate analysis of this data for all real-time digital twins.

Logistics 52

Logistics Analytics Scalability Cloud 52

ScaleOut Software

APRIL 3, 2020

What’s missing is a flexible, fast, and easy-to-use software system that can be quickly adapted to track these assets in real time and provide immediate answers for logistics managers. Within seconds, the software performs aggregate analysis of this data for all real-time digital twins.

Logistics 52

Logistics Analytics Scalability Cloud 52

All Things Distributed

JULY 7, 2011

Werner Vogels weblog on building scalable and robust distributed systems. Amazon Import/Export is an important tool for customers to accelerate moving large amounts of data into the AWS storage systems. a Fast and Scalable NoSQL Database Service Designed for Internet Scale Applications. All Things Distributed.

AWS 98

AWS Cloud Storage Internet 98

All Things Distributed

AUGUST 16, 2011

Werner Vogels weblog on building scalable and robust distributed systems. Government and Big Data. One particular early use case for AWS GovCloud (US) will be massive data processing and analytics. AWS GovCloud (US) will be used by several of these agencies to help them with their Bigger-than-Big-Data needs.

AWS 113

AWS Government Big Data Cloud 113

All Things Distributed

DECEMBER 14, 2011

Werner Vogels weblog on building scalable and robust distributed systems. a Fast and Scalable NoSQL Database Service Designed for Internet Scale Applications. Job Openings in AWS - Senior Leader in Database Services. Driving down the cost of Big-Data analytics. All Things Distributed. Expanding the Cloud â??

AWS 115

AWS Latency Storage Big Data 115

All Things Distributed

AUGUST 20, 2012

Werner Vogels weblog on building scalable and robust distributed systems. It requires substantial upfront capital investments in cold data storage systems such as tape robots and tape libraries, then thereâ??s Data is retrieved by scheduling a job, which typically completes within 3 to 5 hours. All Things Distributed.

Storage 122

Storage Cloud AWS Media 122

All Things Distributed

AUGUST 17, 2011

Werner Vogels weblog on building scalable and robust distributed systems. Amazon S3 is much more than just storage; the network and distributed systems infrastructure to ensure that content can be served fast and at high rates without customers impacting each other, is amazing. Job Openings in AWS - Senior Leader in Database Services.

Servers 126

Servers Social Media AWS Website 126

All Things Distributed

DECEMBER 5, 2010

Werner Vogels weblog on building scalable and robust distributed systems. I am very excited that today we have launched Amazon Route 53, a high-performance and highly-available Domain Name System (DNS) service. Naming is one of the fundamental concepts in Distributed Systems. By Werner Vogels on 05 December 2010 02:00 PM.

Cloud 117

Cloud Internet Internet AWS 117

All Things Distributed

MARCH 2, 2011

Werner Vogels weblog on building scalable and robust distributed systems. a Fast and Scalable NoSQL Database Service Designed for Internet Scale Applications. Job Openings in AWS - Senior Leader in Database Services. Driving down the cost of Big-Data analytics. All Things Distributed. Comments ().

AWS 114

AWS Cloud Games Latency 114

All Things Distributed

JUNE 29, 2011

Werner Vogels weblog on building scalable and robust distributed systems. a Fast and Scalable NoSQL Database Service Designed for Internet Scale Applications. Job Openings in AWS - Senior Leader in Database Services. Driving down the cost of Big-Data analytics. All Things Distributed. Comments ().

AWS 106

AWS Retail Innovation Strategy 106

Brendan Gregg

APRIL 30, 2018

USENIX’s LISA conference is the premier event for topics in production system engineering. We both have had long careers supporting system administration, and LISA has always felt like a homecoming, reuniting with old friends while welcoming newcomers. Join us for 3 days in Nashville at LISA'18. Post by Brendan Gregg and Rikki Endsley.

DevOps 43

DevOps Network Best Practices Programming 43

All Things Distributed

MARCH 24, 2011

Werner Vogels weblog on building scalable and robust distributed systems. It is likely that the Amazon Web Services will be used by many of the participants for their compute, storage, database and other cloud resource needs. a Fast and Scalable NoSQL Database Service Designed for Internet Scale Applications. Comments ().

AWS 96

AWS Internet Internet Storage 96

All Things Distributed

FEBRUARY 23, 2011

Werner Vogels weblog on building scalable and robust distributed systems. a Fast and Scalable NoSQL Database Service Designed for Internet Scale Applications. Job Openings in AWS - Senior Leader in Database Services. Driving down the cost of Big-Data analytics. All Things Distributed. Comments ().

AWS 79

AWS Storage Big Data Servers 79