All Things Distributed

AUGUST 18, 2011

Driving down the cost of Big-Data analytics. The Amazon Elastic MapReduce (EMR) team announced today the ability to seamlessly use Amazon EC2 Spot Instances with their service, significantly driving down the cost of data analytics in the cloud. The posting on the AWS developer blog also has some more background.

Big Data 111

Big Data Analytics AWS Cloud 111

Uber Engineering

AUGUST 14, 2019

Once identified, … The post Less is More: Engineering Data Warehouse Efficiency with Minimalist Design appeared first on Uber Engineering Blog. In our experience, optimizing for operational efficiency requires answering one key question: for which tables does the maintenance cost supersede utility?

Efficiency 110

Efficiency Engineering Design Storage 110

Dynatrace

JULY 6, 2022

As organizations continue to adopt multicloud strategies, the complexity of these environments grows, increasing the need to automate cloud engineering operations to ensure organizations can enforce their policies and architecture principles. This requires significant data engineering efforts, as well as work to build machine-learning models.

Artificial Intelligence 193

Artificial Intelligence Tuning Strategy Big Data 193

Uber Engineering

APRIL 5, 2018

Three years ago, Uber Engineering adopted Hadoop as the storage ( HDFS ) and compute ( YARN ) infrastructure for our organization’s big data analysis.

Systems 109

Systems Big Data Storage Infrastructure 109

Dynatrace

SEPTEMBER 15, 2020

Without automation, Performance engineers and developers can no longer ensure that applications perform as planned, and costs are minimized.”. Supported technologies include cloud services, big data, databases, OS, containers, and application runtimes like the JVM.

Performance 189

Performance Java Metrics Cloud 189

Dynatrace

JULY 6, 2020

Our customers have frequently requested support for this first new batch of services, which cover databases, big data, networks, and computing. See the health of your big data resources at a glance. Azure HDInsight supports a broad range of use cases including data warehousing, machine learning, and IoT analytics.

Azure 149

Azure Cloud Big Data Virtualization 149

Scalegrid

MAY 13, 2020

It can scale towards a multi-petabyte level data workload without a single issue, and it allows access to a cluster of powerful servers that will work together within a single SQL interface where you can view all of the data. Greenplum features a cost-based query optimizer for large-scale, big data workloads.

Big Data 321

Big Data Database Artificial Intelligence Open Source 321

Percona

JANUARY 19, 2023

16 years ago, our founder Peter Zaitsev covered this topic and some of the points described there are still valid, and we will cover more on this blog. With disks being faster nowadays and CPU and memory resources being cheaper, we could easily say MySQL can handle TBs of data with good performance.

Open Source 87

Open Source Storage Database Big Data 87

Dynatrace

MARCH 14, 2022

AIOps (or “AI for IT operations”) uses artificial intelligence so that big data can help IT teams work faster and more effectively. Learn more about Dynatrace’s approach to AIOps and explore Davis , our innovative AI engine at the core of our observability platform. Gartner introduced the concept of AIOps in 2016.

Healthcare 231

Healthcare Artificial Intelligence Innovation Strategy 231

Uber Engineering

DECEMBER 17, 2019

Uber’s busy 2019 included our billionth delivery of an Uber Eats order, 24 million miles covered by bike and scooter riders on our platform, and trips to top destinations such as the Empire State Building, the Eiffel Tower, and the … The post Uber’s Data Platform in 2019: Transforming Information to Intelligence appeared first on Uber Engineering (..)

Engineering 144

Engineering Big Data Infrastructure Analytics 144

DZone

AUGUST 4, 2022

In this blog, I will discuss the availability of Spark-Radiant 1.0.4, Spark-Radiant will help optimize performance and cost considering catalyst optimizer rules, enhance auto-scaling in Spark, collect important metrics related to a Spark job, Bloom filter index in Spark, etc. Spark-Radiant is now available and ready to use.

Performance 246

Performance Metrics Availability Big Data 246

Uber Engineering

DECEMBER 10, 2019

Michelangelo , Uber’s machine learning (ML) platform, powers machine learning model training across various use cases at Uber, such as forecasting rider demand , fraud detection , food discovery and recommendation for Uber Eats , and improving the accuracy of … The post Productionizing Distributed XGBoost to Train Deep Tree Models with Large (..)

Engineering 122

Engineering Big Data Architecture 122

Uber Engineering

OCTOBER 30, 2018

Cluster management, a common software infrastructure among technology companies, aggregates compute resources from a collection of physical hosts into a shared resource pool, amplifying compute power and allowing for the flexible use of data center hardware.

Hardware 103

Hardware Infrastructure Engineering Technology 103

All Things Distributed

SEPTEMBER 5, 2013

Over the past few years, two important trends that have been disrupting the database industry are mobile applications and big data. The explosive growth in mobile devices and mobile apps is generating a huge amount of data, which has fueled the demand for big data services and for high scale databases.

Big Data 98

Big Data Mobile Latency Database 98

Uber Engineering

OCTOBER 30, 2018

Cluster management, a common software infrastructure among technology companies, aggregates compute resources from a collection of physical hosts into a shared resource pool, amplifying compute power and allowing for the flexible use of data center hardware.

Hardware 64

Hardware Infrastructure Engineering Technology 64

Rigor

SEPTEMBER 11, 2019

Reading time 16 min Whether you’re a web performance expert, an evangelist for the culture of performance, a web engineer incorporating performance into your process, or someone new to the web performance entirely, you probably identify as curious, excited about new ideas, and always learning. Jake Archibald. Rick Byers. Rick Byers.

Performance 89

Performance Education Google Website 89

All Things Distributed

AUGUST 20, 2012

With Amazon Glacier any organization now has access to the same data archiving capabilities as the worldâ??s We see many young businesses engaging in large-scale big-data collection activities, and storing all this data can become rather expensive over time- archiving their historical data sets in Amazon Glacier is an ideal solution.

Storage 122

Storage Cloud AWS Media 122

Testsigma

MAY 26, 2021

Scenario 4: Image-based data-driven testing. Photo management applications, e-commerce websites, search engines, healthcare, disaster monitoring, robotics, etc. The test data is then run on the algorithm to decide the accuracy, predictability, and precision of the spam detection algorithm. Conclusion.

Testing 70

Testing Healthcare Performance Testing Website 70

Rigor

JANUARY 13, 2020

When it comes to web content, you can easily find what you need through many different paths, from search engines and social media to playlists and blogs, jumping from one source to another with just a tap of a finger.

Performance 75

Performance Social Media Website Website Performance 75

All Things Distributed

AUGUST 17, 2011

A good example is the comments section on this blog; a few lines of JavaScript and these pages have a dynamic nature with comments, trackbacks and social media discussion showing up as they happen. But I really want a setup that allows me to thinker with the blog where ever I am (e.g. at 30,000 feet). Which suits me just fine.

Servers 126

Servers Social Media AWS Website 126

Testsigma

JULY 14, 2020

If there is any update required either in test scripts or test data, it is hassle-free because both test data and test scripts are placed separately with no dependency on each other. Simply running the automation scripts on a massive test data set, without a proper vision is such a waste of time, effort and resources.

Testing 63

Testing Artificial Intelligence DevOps Big Data 63

All Things Distributed

DECEMBER 8, 2016

It adopted Amazon Redshift, Amazon EMR and AWS Lambda to power its data warehouse, big data, and data science applications, supporting the development of product features at a fraction of the cost of competing solutions. Kik Interactive is a Canadian chat platform with hundreds of millions of users around the globe.

AWS 155

AWS Cloud Lambda Innovation 155

All Things Distributed

NOVEMBER 19, 2010

Big news this week was of course the launch of Cluster GPU instances for Amazon EC2. There were blog posts by Jeff Barr The Cluster GPU Instance and James Hamilton HPC in the Cloud with GPGPUs , as well as my background posting: Expanding the Cloud - Adding the Incredible Power of the Amazon EC2 Cluster GPU Instances. Contact Info.

AWS 67

AWS Cloud Benchmarking Storage 67

The Netflix TechBlog

NOVEMBER 20, 2023

It also improves the engineering productivity by simplifying the existing pipelines and unlocking the new patterns. In this blog post, we talk about the landscape and the challenges in workflows at Netflix. Backfill: Backfilling datasets is a common operation in big data processing. data arrives too late to be useful).

Processing 210

Processing Big Data Efficiency Engineering 210

The Netflix TechBlog

JULY 21, 2022

We at Netflix, as a streaming service running on millions of devices, have a tremendous amount of data about device capabilities/characteristics and runtime data in our big data platform. With large data, comes the opportunity to leverage the data for predictive and classification based analysis.

Big Data 179

Big Data Cache Engineering Data Engineering 179

All Things Distributed

JANUARY 19, 2011

and Engine Yard , Springsource users have CloudFoundry. More information on the launch can be found on the AWS developer blog. blog comments powered by Disqus. he posts material that doesnt belong on this blog or on twitter. Driving down the cost of Big-Data analytics. Contact Info. Werner Vogels.

AWS 115

AWS Cloud Java Scalability 115

All Things Distributed

SEPTEMBER 30, 2010

There is huge variety in exiting architectures and I am often impressed about the ingenuity of the engineers in how to best transform the application if "Lift & Shift" is not an option. blog comments powered by Disqus. he posts material that doesnt belong on this blog or on twitter. Driving down the cost of Big-Data analytics.

AWS 75

AWS Storage Cloud Best Practices 75

Dynatrace

OCTOBER 25, 2022

Mike Gualtieri of Forrester Research, who coined the term NoOps , once said in his blog post, “NoOps means that application developers will never have to speak with an operations professional again.”. Davis®— the Dynatrace purpose-built AI engine —further uses event proximity and baselined data for automatic correlation and noise suppression.

DevOps 222

DevOps Big Data Cloud Innovation 222

All Things Distributed

JULY 13, 2010

Cluster Computer Instances are similar to other Amazon EC2 instances but have been specifically engineered to provide high performance compute and networking. Jeff Barr in his blog post on the AWS developer blog has additional details and there are some great testimonials of early Cluster Compute Instances customers in the press release.

Cloud 118

Cloud AWS Automotive Latency 118

All Things Distributed

DECEMBER 5, 2010

There is more than one Werner Vogels in this world and although I never get emails, snail mail or phones calls for any of my peers, I am sure they are somewhat frustrated if they type in our name in a search engine :-). If you want to learn more about Route 53 visit [link] and read the blog post at the AWS Developer weblog.

Cloud 117

Cloud Internet Internet AWS 117

The Netflix TechBlog

MARCH 25, 2019

Now, imagine yourself in the role of a software engineer responsible for a micro-service which publishes data consumed by few critical customer facing services (e.g. You are about to make structural changes to the data and want to know who and what downstream to your service will be impacted.

Infrastructure 219

Infrastructure Big Data Transportation Architecture 219

Dynatrace

AUGUST 10, 2021

And, they got the chance to do just that at our recent event, DynatraceGo, which I’ll tell you a bit more about in this blog. Vikash Chhaganlal , GM of Engineering and Infrastructure at Kiwibank said it. She dispelled the myth that more big data equals better decisions, higher profits, or more customers.

DevOps 246

DevOps Innovation Big Data Cloud 246

Dynatrace

APRIL 21, 2021

To do this effectively, you need a big data processing approach. The three challenges to optimizing Core Web Vitals is exactly why the Dynatrace Business Insights team have built the Insights Analytics Engine. The post Business Insights extends support for optimizing Core Web Vitals appeared first on Dynatrace blog.

Traffic 263

Traffic Metrics Mobile Analytics 263

All Things Distributed

APRIL 28, 2010

Currently, each AWS Region contains multiple Availability Zones, which are distinct locations that are engineered to be insulated from failures in other Availability Zones. You can also find more information on the AWS developer blog. All the RightScale services are available for the new region as well, read more on their blog.

AWS 61

AWS Cloud Latency Storage 61

The Netflix TechBlog

OCTOBER 18, 2022

As Big data and ML became more prevalent and impactful, the scalability, reliability, and usability of the orchestrating ecosystem have increasingly become more important for our data scientists and the company. In this blog, we use step and job interchangeably.

Java 202

Java Scalability Traffic Architecture 202

Dynatrace

OCTOBER 4, 2022

Dynatrace built and optimized it for Davis® AI, the game-changing Dynatrace artificial intelligence engine that processes billions of dependencies in the blink of an eye. Grail addresses today’s challenges of big data and cloud everywhere: Grail is highly scalable, cost-effective, and super-fast.

Analytics 239

Analytics Artificial Intelligence Storage Serverless 239

All Things Distributed

OCTOBER 7, 2015

However, the data infrastructure to collect, store and process data is geared toward developers (e.g., In AWS’ quest to enable the best data storage options for engineers, we have built several innovative database solutions like Amazon RDS, Amazon RDS for Aurora, Amazon DynamoDB, and Amazon Redshift. Big data challenges.

Cloud 137

Cloud Big Data AWS Analytics 137