All Things Distributed

AUGUST 20, 2012

Managing Cold Storage with Amazon Glacier. With the introduction of Amazon Glacier , IT organizations now have a solution that removes the headaches of digital archiving and provides extremely low cost storage. Data is retrieved by scheduling a job, which typically completes within 3 to 5 hours. All Things Distributed.

Storage 122

Storage Cloud AWS Media 122

The Netflix TechBlog

DECEMBER 21, 2020

At this scale, we can gain a significant amount of performance and cost benefits by optimizing the storage layout (records, objects, partitions) as the data lands into our warehouse. We built AutoOptimize to efficiently and transparently optimize the data and metadata storage layout while maximizing their cost and performance benefits.

Storage 203

Storage Latency Efficiency Data Engineering 203

Percona

JANUARY 14, 2019

What if we use ClickHouse (which is a columnar analytical database) as our main datastore? Well, typically, an analytical database is not a replacement for a transactional or key/value datastore. Although such databases can be very efficient with counts and averages, some queries will be slow or simply non existent. Processed 4.15

Database 40

Database Analytics Blockchain Healthcare 40

All Things Distributed

JULY 7, 2011

AWS Import/Export transfers data off of storage devices using Amazons high-speed internal network and bypassing the Internet. With this new functionality AWS Import/Export now supports importing data directly into Amazon EBS snapshots. Driving Storage Costs Down for AWS Customers. Syndication. or rss feed.

AWS 98

AWS Cloud Storage Internet 98

The Morning Paper

SEPTEMBER 19, 2019

MongoDB is an important database, and this paper explains the tunable (per-operation) consistency models that MongoDB provides and how they are implemented under the covers. Microsoft have a paper describing their new recovery mechanism in Azure SQL Database , the key feature being that it can recovery in constant time.

Blockchain 46

Blockchain Hardware Google Analytics 46

The Morning Paper

JUNE 23, 2019

Coupled with stateless application servers to execute business logic and a database-like system to provide persistent storage, they form a core component of popular data center service archictectures. We’ve seen similar high marshalling overheads in big data systems too.) See e.g. ‘ caching data with Spring.’

Cache 79

Cache Latency Google Lambda 79

All Things Distributed

JULY 16, 2012

There are sessions in many different categories: Architecture, Big Data, HPC, Computer & Networking, Storage, Databases, Security, Tools & Languages, Media Sharing & Content Delivery, Managing AWS Resources, Enterprise IT, Mobile, Start-up, and more.

AWS 85

AWS Big Data Media Storage 85

All Things Distributed

MARCH 2, 2011

Japanese companies and consumers have become used to low latency and high-speed networking available between their businesses, residences, and mobile devices. With the launch of the Asia Pacific (Tokyo) Region, companies can now leverage the AWS suite of infrastructure web services directly connected to Japanese networks. Syndication.

AWS 114

AWS Cloud Games Latency 114

All Things Distributed

AUGUST 17, 2011

As some of you may remember I was pretty excited when Amazon Simple Storage Service (S3) released its website feature such that I could serve this weblog completely from S3. Driving Storage Costs Down for AWS Customers. Expanding the Cloud - The AWS Storage Gateway. Job Openings in AWS - Senior Leader in Database Services.

Servers 126

Servers Social Media AWS Website 126

All Things Distributed

AUGUST 16, 2011

Government and Big Data. One particular early use case for AWS GovCloud (US) will be massive data processing and analytics. Several agencies of very different parts of the government have needs for data analytics that really put the Big in Big-Data, sometimes several orders of magnitude larger than commonly found in industry.

AWS 113

AWS Government Big Data Cloud 113

Brendan Gregg

APRIL 30, 2018

Apart from networking, attending conferences like LISA in person is an effective way to upgrade your skills: you can block out work interruptions and absorb new knowledge that's been neatly summarized into sessions. We first met each other at LISA, in addition to making many other important industry connections over the years.

DevOps 43

DevOps Network Best Practices Programming 43

Brendan Gregg

APRIL 29, 2018

Apart from networking, attending conferences like LISA in person is an effective way to upgrade your skills: you can block out work interruptions and absorb new knowledge that's been neatly summarized into sessions. We first met each other at LISA, in addition to making many other important industry connections over the years.

DevOps 40

DevOps Network Best Practices Programming 40

All Things Distributed

JANUARY 19, 2011

Flexibility is one of the key principles of Amazon Web Services - developers can select any programming language and software package, any operating system, any middleware and any database to build systems and applications that meet their requirements. Driving Storage Costs Down for AWS Customers. Syndication. or rss feed.

AWS 115

AWS Cloud Java Scalability 115

All Things Distributed

JULY 13, 2010

Customers with complex computational workloads such as tightly coupled, parallel processes, or with applications that are very sensitive to network performance, can now achieve the same high compute and networking performance provided by custom-built infrastructure while benefiting from the elasticity, flexibility and cost advantages of Amazon EC2.

Cloud 119

Cloud AWS Automotive Latency 119

All Things Distributed

DECEMBER 5, 2010

The naming system that we are all most familiar with in the internet is the Domain Name System (DNS) that manages the naming of the many different entities in our global network; its most common use is to map a name to an IP address, but it also provides facilities for aliases, finding mail servers, managing security keys, and much more.

Cloud 117

Cloud Internet Internet AWS 117

Highly Scalable

MAY 1, 2012

Let us start with a simple example that illustrates capabilities of probabilistic data structures: Let us have a data set that is simply a heap of ten million random integer values and we know that it contains not more than one million distinct values (there are many duplicates). what is the cardinality of the data set)?

Analytics 191

Analytics Traffic Big Data Efficiency 191

All Things Distributed

MAY 18, 2010

Expanding the Cloud - Amazon S3 Reduced Redundancy Storage. Today a new storage option for Amazon S3 has been launched: Amazon S3 Reduced Redundancy Storage (RRS). This new storage option enables customers to reduce their costs by storing non-critical, reproducible data at lower levels of redundancy. Comments ().

Storage 72

Storage Cloud AWS Scalability 72

Abhishek Tiwari

DECEMBER 27, 2017

ETL refers to extract, transform, load and it is generally used for data warehousing and data integration. ETL is a product of the relational database era and it has not evolved much in last decade. There are several emerging data trends that will define the future of ETL in 2018. Machine learning meets data integration.

Big Data 49

Big Data Artificial Intelligence Storage Hardware 49

All Things Distributed

APRIL 27, 2011

We use high-performance transactions systems, complex rendering and object caching, workflow and queuing systems, business intelligence and data analytics, machine learning and pattern recognition, neural networks and probabilistic decision making, and a wide variety of other techniques. Driving Storage Costs Down for AWS Customers.

Technology 151

Technology Technology AWS Storage 151

All Things Distributed

FEBRUARY 17, 2011

Since a few days ago this weblog serves 100% of its content directly out of the Amazon Simple Storage Service (S3) without the need for a web server to be involved. I have used a bucket policy to make all documents world readable, but you could create one that restricts it to referrers, network address range, time of day, etc.

AWS 134

AWS Website Storage Servers 134

All Things Distributed

DECEMBER 9, 2010

By supporting such large object sizes, Amazon S3 better enables a variety of interesting big data use cases. You can create parallel uploads to better utilize your available bandwidth and even stream data into Amazon S3 as its being created. Driving Storage Costs Down for AWS Customers. Syndication. or rss feed.

AWS 82

AWS Big Data Scalability Storage 82

All Things Distributed

FEBRUARY 24, 2010

Amazon SimpleDB has launched today with a new set of features giving the customer more control over which consistency and concurrency models to use in their database operations. These new features will make it easier to transition those applications to SimpleDB that are designed with traditional database tools in mind. Consistent Read.

AWS 70

AWS Latency Database Scalability 70

All Things Distributed

NOVEMBER 15, 2010

Starting today Amazon EMR can take advantage of the Cluster Compute and Cluster GPU instances, giving customers ever more powerful components to base the large scale data processing and analysis on. Driving Storage Costs Down for AWS Customers. Expanding the Cloud - The AWS Storage Gateway. Where to go from here? Syndication.

AWS 136

AWS Latency Programming Architecture 136

All Things Distributed

APRIL 19, 2010

Amazon Cloudfront is the Content Delivery Network (CDN) that is dead simple to use both from a technology and a business point of view. Driving Storage Costs Down for AWS Customers. Expanding the Cloud - The AWS Storage Gateway. a Fast and Scalable NoSQL Database Service Designed for Internet Scale Applications.

Media 61

Media AWS Storage Big Data 61