All Things Distributed

AUGUST 18, 2011

Werner Vogels weblog on building scalable and robust distributed systems. Driving down the cost of Big-Data analytics. The Amazon Elastic MapReduce (EMR) team announced today the ability to seamlessly use Amazon EC2 Spot Instances with their service, significantly driving down the cost of data analytics in the cloud.

Big Data 111

Big Data Analytics AWS Cloud 111

Scalegrid

MARCH 28, 2024

In this comparison of Redis vs Memcached, we strip away the complexity, focusing on each in-memory data store’s performance, scalability, and unique features. Redis is better suited for complex data models, and Memcached is better suited for high-throughput, string-based caching scenarios.

Cache 130

Cache Storage Scalability Architecture 130

The Morning Paper

MAY 14, 2019

Seer: leveraging big data to navigate the complexity of performance debugging in cloud microservices Gan et al., Seer is an online system that observes the behaviour of cloud applications (using the DeathStarBench microservices for the evaluation) and predicts when QoS violations may be about to occur. ASPLOS’19.

Big Data 45

Big Data Cloud Performance Hardware 45

Dynatrace

MARCH 24, 2023

Containers enable developers to package microservices or applications with the libraries, configuration files, and dependencies needed to run on any infrastructure, regardless of the target system environment. This means organizations are increasingly using Kubernetes not just for running applications, but also as an operating system.

Infrastructure 191

Infrastructure Open Source Operating System Cloud 191

The Netflix TechBlog

APRIL 29, 2019

Content is placed on the network of servers in the Open Connect CDN as close to the end user as possible, improving the streaming experience for our customers and reducing costs for both Netflix and our Internet Service Provider (ISP) partners. We also use Python to detect sensitive data using Lanius.

Open Source 279

Open Source Network Infrastructure Big Data 279

Dynatrace

OCTOBER 17, 2022

Artificial intelligence for IT operations, or AIOps, combines big data and machine learning to provide actionable insight for IT teams to shape and automate their operational strategy. The four stages of data processing. There are four stages of data processing: Collect raw data. Analyze the data.

DevOps 202

DevOps Artificial Intelligence Healthcare Innovation 202

The Morning Paper

OCTOBER 26, 2020

On the surface this is a paper about fast data ingestion from high-volume streams, with indexing to support efficient querying. As a production system within Microsoft capturing around a quadrillion events and indexing 16 trillion search keys per day it would be interesting in its own right, but there’s a lot more to it than that.

Cloud 104

Cloud Big Data Latency Architecture 104

ScaleOut Software

JULY 19, 2021

How are we managing the torrent of telemetry that flows into analytics systems from these devices? Incoming data is saved into data storage (historian database or log store) for query by operational managers who must attempt to find the highest priority issues that require their attention. The list goes on.

IoT 78

IoT Analytics Big Data Architecture 78

Highly Scalable

MARCH 1, 2012

This aspect of NoSQL is well-studied both in practice and theory because specific non-functional properties are often the main justification for NoSQL usage and fundamental results on distributed systems like the CAP theorem apply well to NoSQL systems. Many techniques that are described below are perfectly applicable to this model.

Database 279

Database Ecommerce Efficiency Engineering 279

High Scalability

APRIL 14, 2020

has hours of system design content. They also do live system design discussions every week. Scrapinghub is hiring a Senior Software Engineer (Big Data/AI). Check out Educative.io's bestselling new 4-course learning track: Scalability and System Design for Developers. Who's Hiring? InterviewCamp.io Apply here.

Education 105

Education Software Engineering Scalability Engineering 105

High Scalability

MAY 12, 2020

has hours of system design content. They also do live system design discussions every week. Scrapinghub is hiring a Senior Software Engineer (Big Data/AI). Check out Educative.io's bestselling new 4-course learning track: Scalability and System Design for Developers. Who's Hiring? InterviewCamp.io Apply here.

Education 102

Education Software Engineering Scalability Engineering 102

High Scalability

APRIL 28, 2020

has hours of system design content. They also do live system design discussions every week. Scrapinghub is hiring a Senior Software Engineer (Big Data/AI). Check out Educative.io's bestselling new 4-course learning track: Scalability and System Design for Developers. Who's Hiring? InterviewCamp.io Apply here.

Education 102

Education Software Engineering Scalability Engineering 102

High Scalability

MARCH 30, 2020

has hours of system design content. They also do live system design discussions every week. Scrapinghub is hiring a Senior Software Engineer (Big Data/AI). Level up on in-demand technologies and prep for your interviews on Educative.io, featuring popular courses like the bestselling Grokking the System Design Interview.

Education 105

Education Software Engineering Engineering Big Data 105

High Scalability

MAY 26, 2020

has hours of system design content. They also do live system design discussions every week. Scrapinghub is hiring a Senior Software Engineer (Big Data/AI). Check out Educative.io 's bestselling new 4-course learning track: Scalability and System Design for Developers. Who's Hiring? InterviewCamp.io Apply here.

Education 100

Education AWS Software Engineering Scalability 100

High Scalability

MARCH 24, 2020

Scrapinghub is hiring a Senior Software Engineer (Big Data/AI). You will be designing and implementing distributed systems : large-scale web crawling platform, integrating Deep Learning based web data extraction components, working on queue algorithms, large datasets, creating a development platform for other company departments, etc.

Education 105

Education Software Engineering Engineering Big Data 105

Dynatrace

JUNE 1, 2020

The variables that can impact the performance of an application vary; from coding errors or ‘bugs’ in the software, database slowdowns, hosting and network performance, to operating system and device type support. Continuous Automation. With its ease of use, and unlimited number of users, you can eliminate silos and accelerate teamwork.

Artificial Intelligence 184

Artificial Intelligence Social Media Monitoring IoT 184

ACM Sigarch

MAY 31, 2023

Introduction Memory systems are evolving into heterogeneous and composable architectures. Heterogeneous and Composable Memory (HCM) offers a feasible solution for terabyte- or petabyte-scale systems, addressing the performance and efficiency demands of emerging big-data applications. The recently announced CXL3.0

Latency 52

Latency Hardware Cache Architecture 52

All Things Distributed

DECEMBER 13, 2016

The council has deployed IoT Weather Stations in Schools across the City and is using the sensor information collated in a Data Lake to gain insights on whether the weather or pollution plays a part in learning outcomes. The British Government is also helping to drive innovation and has embraced a cloud-first policy for technology adoption.

AWS 166

AWS Cloud Artificial Intelligence IoT 166

Dynatrace

JUNE 1, 2020

The variables that can impact the performance of an application vary; from coding errors or ‘bugs’ in the software, database slowdowns, hosting and network performance, to operating system and device type support. Continuous Automation. With its ease of use, and unlimited number of users, you can eliminate silos and accelerate teamwork.

Monitoring 132

Monitoring Performance Social Media Artificial Intelligence 132

All Things Distributed

AUGUST 17, 2011

Werner Vogels weblog on building scalable and robust distributed systems. No Server Required - Jekyll & Amazon S3. Amazon S3 is much more than just storage; the network and distributed systems infrastructure to ensure that content can be served fast and at high rates without customers impacting each other, is amazing.

Servers 126

Servers Social Media AWS Website 126

SQL Server According to Bob

JANUARY 15, 2020

SQL Server BDC Hints and Tips: The node’s Journal can be your best friend If you are new to Kubernetes (K8’s), like I am, you quickly learn that the servicing node’s journal holds a wealth of information. ​​ For example, my master-0, SQL Server, pod was getting evicted and restarted.

Servers 40

Servers Metrics Big Data Operating System 40

All Things Distributed

JANUARY 11, 2011

Werner Vogels weblog on building scalable and robust distributed systems. DROAM - Dreaming about Cheap Data Roaming. The one thing that I have always struggled with during my travels are the data plans of the cell phone companies. for the device rental and the first day of data, and â?¬3.50 Comments ().

Wireless 107

Wireless AWS Internet Internet 107

ScaleOut Software

NOVEMBER 24, 2020

Real-Time Digital Twins Can Add Important New Capabilities to Telematics Systems and Eliminate Scalability Bottlenecks. The volume of incoming telemetry challenges current telematics systems to keep up and quickly make sense of all the data. Current Telematics Architecture.

Analytics 52

Analytics Architecture Scalability Software Architecture 52

The Morning Paper

JUNE 23, 2019

Coupled with stateless application servers to execute business logic and a database-like system to provide persistent storage, they form a core component of popular data center service archictectures. Why are developers using RInK systems as part of their design? Fetching too much data in a single query (i.e.,

Cache 79

Cache Latency Google Lambda 79

Alex Podelko

DECEMBER 19, 2019

What’s that VM/Server Doing? Computing System Congestion Management Using Exponential Smoothing Forecasting by James Brady, State of Nevada. – System performance management is an important topic – and James is going to share a practical method for it. by Debbie Sheetz, MBI Solutions.

Efficiency 100

Efficiency Artificial Intelligence Scalability Performance 100

All Things Distributed

JANUARY 6, 2016

Mirae Asset Global Investments improved its web service environment and reduced annual management costs by 50% by consolidating the management of all web services, including servers, network, database, and security. Many of these enterprises are assisted by our extensive partner ecosystem in Korea.

AWS 110

AWS Cloud Games Latency 110

All Things Distributed

APRIL 4, 2017

The new region will give Nordic-based businesses, government organisations, non-profits, and global companies with customers in the Nordics, the ability to leverage the AWS technology infrastructure from data centers in Sweden. The new AWS EU (Stockholm) Region will have three Availability Zones and will be ready for customers to use in 2018.

AWS 116

AWS Airlines Latency Games 116

All Things Distributed

FEBRUARY 17, 2011

Werner Vogels weblog on building scalable and robust distributed systems. Since a few days ago this weblog serves 100% of its content directly out of the Amazon Simple Storage Service (S3) without the need for a web server to be involved. I had held out implementing an alternative to my simple blog server that had. Comments ().

AWS 134

AWS Website Storage Servers 134

All Things Distributed

JULY 7, 2011

Werner Vogels weblog on building scalable and robust distributed systems. AWS Import/Export transfers data off of storage devices using Amazons high-speed internal network and bypassing the Internet. With this new functionality AWS Import/Export now supports importing data directly into Amazon EBS snapshots. Comments ().

AWS 96

AWS Cloud Storage Internet 96

All Things Distributed

DECEMBER 5, 2010

Werner Vogels weblog on building scalable and robust distributed systems. I am very excited that today we have launched Amazon Route 53, a high-performance and highly-available Domain Name System (DNS) service. Naming is one of the fundamental concepts in Distributed Systems. Authoritative servers hold the definitive mappings.

Cloud 117

Cloud Internet Internet AWS 117

All Things Distributed

AUGUST 22, 2011

Werner Vogels weblog on building scalable and robust distributed systems. Systems that make extensive use of caching almost all report a significant reduction in the cost of their database tier. Driving down the cost of Big-Data analytics. No Server Required - Jekyll & Amazon S3. All Things Distributed.

Cloud 117

Cloud Cache AWS Storage 117

All Things Distributed

AUGUST 16, 2011

Werner Vogels weblog on building scalable and robust distributed systems. For example a number of our European customers are subject to data residency requirements when it comes to PII data and they use the EU Region to meet to those requirements. Government and Big Data. All Things Distributed. Comments ().

AWS 111

AWS Government Big Data Cloud 111

All Things Distributed

FEBRUARY 23, 2011

Werner Vogels weblog on building scalable and robust distributed systems. In a follow up to the last blog post I have removed the last two dependencies this blog had on running a server somewhere: comments are now served by Disqus and search is now handled by Bing. Driving down the cost of Big-Data analytics.

AWS 78

AWS Storage Big Data Servers 78

All Things Distributed

JANUARY 19, 2011

Werner Vogels weblog on building scalable and robust distributed systems. Flexibility is one of the key principles of Amazon Web Services - developers can select any programming language and software package, any operating system, any middleware and any database to build systems and applications that meet their requirements.

AWS 115

AWS Cloud Java Scalability 115

All Things Distributed

APRIL 27, 2011

Werner Vogels weblog on building scalable and robust distributed systems. To our shareowners: Random forests, naïve Bayesian estimators, RESTful services, gossip protocols, eventual consistency, data sharding, anti-entropy, Byzantine quorum, erasure coding, vector clocks. All Things Distributed. Comments ().

Technology 151

Technology Technology AWS Storage 151

All Things Distributed

DECEMBER 14, 2011

Werner Vogels weblog on building scalable and robust distributed systems. Additionally, it allows them to keep their data inside of Brazil. Driving down the cost of Big-Data analytics. No Server Required - Jekyll & Amazon S3. All Things Distributed. Expanding the Cloud â?? Comments ().

AWS 114

AWS Latency Storage Big Data 114

All Things Distributed

MARCH 2, 2011

Werner Vogels weblog on building scalable and robust distributed systems. With the new Tokyo Region companies that are required to meet certain compliance, control, and data locality requirements can now achieve these certifications: customers can now choose to keep their data entirely within the Tokyo Region. Comments ().

AWS 112

AWS Cloud Games Latency 112