Azure and Big Data - Technology Performance Pulse

Microsoft Azure Event Hubs

DZone

FEBRUARY 23, 2023

Introduction With big data streaming platform and event ingestion service Azure Event Hubs , millions of events can be received and processed in a single second. Any real-time analytics provider or batching/storage adaptor can transform and store data supplied to an event hub.

Azure

Azure Big Data Analytics Storage

Introduction to Azure Data Lake Storage Gen2

DZone

FEBRUARY 1, 2023

Built on Azure Blob Storage, Azure Data Lake Storage Gen2 is a suite of features for big data analytics. Azure Data Lake Storage Gen1 and Azure Blob Storage's capabilities are combined in Data Lake Storage Gen2.

Azure

Azure Storage Big Data Analytics

What is Greenplum Database? Intro to the Big Data Database

Scalegrid

MAY 13, 2020

When handling large amounts of complex data, or big data, chances are that your main machine might start getting crushed by all of the data it has to process in order to produce your analytics results. Greenplum features a cost-based query optimizer for large-scale, big data workloads. Query Optimization.

Big Data

Big Data Database Artificial Intelligence Open Source

ScyllaDB Trends – How Users Deploy The Real-Time Big Data Database

Scalegrid

NOVEMBER 25, 2019

Google Cloud does offer their own wide column store and big data database called Bigtable which is actually ranked #111, one under ScyllaDB at #110 on DB-Engines. Azure followed in third place representing 17.4% Azure followed in third place representing 17.4% AWS vs. Azure vs. GCP Click To Tweet.

Big Data

Big Data Database Open Source Azure

No need to compromise visibility in public clouds with the new Azure services supported by Dynatrace

Dynatrace

JULY 6, 2020

As adoption rates for Microsoft Azure continue to skyrocket, Dynatrace is developing a deeper integration with the platform to provide even more value to organizations that run their businesses on Azure or use it as a part of their multi-cloud strategy. Azure Batch. Azure DB for MariaDB. Azure DB for MySQL.

Azure

Azure Cloud Big Data Virtualization

What is a Distributed Storage System

Scalegrid

FEBRUARY 8, 2024

These systems enable vast amounts of data to be spread over multiple nodes, allowing for simultaneous access and boosting processing efficiency. Amazon S3 and Microsoft Azure Blob Storage leverage distributed storage solutions. These distributed storage services also play a pivotal role in big data and analytics operations.

Storage

Storage Systems Big Data Azure

Experiences with approximating queries in Microsoft’s production big-data clusters

The Morning Paper

SEPTEMBER 8, 2019

Experiences with approximating queries in Microsoft’s production big-data clusters Kandula et al., Microsoft’s big data clusters have 10s of thousands of machines, and are used by thousands of users to run some pretty complex queries. VLDB’19. For the larger more production-like query analysed in §4.2.1,

Big Data

Big Data Analytics Latency Azure

Hybrid cloud infrastructure explained: Weighing the pros, cons, and complexities

Dynatrace

JUNE 29, 2022

A hybrid cloud, however, combines public infrastructure and services with on-premises resources or a private data center to create a flexible, interconnected IT environment. Hybrid environments provide more options for storing and analyzing ever-growing volumes of big data and for deploying digital services.

Infrastructure

Infrastructure Cloud Azure AWS

What is a data lakehouse? Combining data lakes and warehouses for the best of both worlds

Dynatrace

OCTOBER 4, 2022

Agent and open technologies make it easy to ingest large volumes of observability, security, and business data. Data management. Data lakehouses take advantage of low-cost object stores like AWS S3 or Microsoft Azure Blob Storage to store and manage data cost-effectively. Data warehouses. Query language.

Artificial Intelligence

Artificial Intelligence Analytics Storage Government

Kubernetes in the wild report 2023

Dynatrace

JANUARY 16, 2023

Most Kubernetes clusters in the cloud (73%) are built on top of managed distributions from the hyperscalers like AWS Elastic Kubernetes Service (EKS), Azure Kubernetes Service (AKS), or Google Kubernetes Engine (GKE). Big data : To store, search, and analyze large datasets, 32% of organizations use Elasticsearch.

Open Source

Open Source Java Operating System Programming

What is container orchestration?

Dynatrace

MARCH 24, 2023

Part of its popularity owes to its availability as a managed service through the major cloud providers, such as Amazon Elastic Kubernetes Service , Google Kubernetes Engine , and Microsoft Azure Kubernetes Service. Likewise, Kubernetes is both an enterprise platform and managed services with Red Hat OpenShift.

Infrastructure

Infrastructure Open Source Operating System Cloud

Mastering Hybrid Cloud Strategy

Scalegrid

MARCH 14, 2024

Workloads from web content, big data analytics, and artificial intelligence stand out as particularly well-suited for hybrid cloud infrastructure owing to their fluctuating computational needs and scalability demands.

Strategy

Strategy Cloud Artificial Intelligence Infrastructure

The Need for Real-Time Device Tracking

ScaleOut Software

JULY 19, 2021

And it can maintain contextual information about every data source (like the medical history of a device wearer or the maintenance history of a refrigeration system) and keep it immediately at hand to enhance the analysis.

IoT

IoT Analytics Big Data Architecture

Microsoft Engineering loves SQLBits

SQL Server According to Bob

FEBRUARY 15, 2018

Microsoft engineering is actually sending quite a few folks over the Atlantic to come talk about SQL Server 2017, SQL Server on Linux, GDPR, Performance, Security, Azure Data Lake, Azure SQL Database, Azure SQL Data Warehouse, and Azure CosmosDB. SELECT * FROM Azure Cosmos DB – Andrew Liu.

Engineering

Engineering Azure Best Practices Servers

Even more amazing papers at VLDB 2019 (that I didn’t have space to cover yet)

The Morning Paper

SEPTEMBER 19, 2019

Microsoft have a paper describing their new recovery mechanism in Azure SQL Database , the key feature being that it can recovery in constant time. Hyper Dimension Shuffle describes how Microsoft improved the cost of data shuffling, one of the most costly operations, in their petabyte-scale internal big data analytics platform, SCOPE.

Blockchain

Blockchain Hardware Google Analytics

Using Real-Time Digital Twins for Aggregate Analytics

ScaleOut Software

JUNE 15, 2020

Instead, most applications just sift through the telemetry for patterns that might indicate exceptional conditions and forward the bulk of incoming messages to a data lake for offline scrubbing with a big data tool such as Spark. Maintain State Information for Each Data Source.

Analytics

Analytics IoT Lambda Big Data

Using Real-Time Digital Twins for Aggregate Analytics

ScaleOut Software

JUNE 15, 2020

Instead, most applications just sift through the telemetry for patterns that might indicate exceptional conditions and forward the bulk of incoming messages to a data lake for offline scrubbing with a big data tool such as Spark. Maintain State Information for Each Data Source.

Analytics

Analytics IoT Lambda Big Data

Track Thousands of Assets in a Time of Crisis Using Real-Time Digital Twins

ScaleOut Software

APRIL 3, 2020

Unlike powerful big data platforms which focus on deep and often lengthy analysis to make future projections, what real-time digital twins offer is timeliness in obtaining quick answers to pressing questions using the most current data.

Logistics

Logistics Analytics Scalability Cloud

Track Thousands of Assets in a Time of Crisis Using Real-Time Digital Twins

ScaleOut Software

APRIL 3, 2020

Unlike powerful big data platforms which focus on deep and often lengthy analysis to make future projections, what real-time digital twins offer is timeliness in obtaining quick answers to pressing questions using the most current data.

Logistics

Logistics Analytics Scalability Cloud

Technology Performance Pulse

Microsoft Azure Event Hubs

Introduction to Azure Data Lake Storage Gen2

Trending Sources

What is Greenplum Database? Intro to the Big Data Database

ScyllaDB Trends – How Users Deploy The Real-Time Big Data Database

No need to compromise visibility in public clouds with the new Azure services supported by Dynatrace

What is a Distributed Storage System

Experiences with approximating queries in Microsoft’s production big-data clusters

Hybrid cloud infrastructure explained: Weighing the pros, cons, and complexities

What is a data lakehouse? Combining data lakes and warehouses for the best of both worlds

Kubernetes in the wild report 2023

What is container orchestration?

Mastering Hybrid Cloud Strategy

The Need for Real-Time Device Tracking

Microsoft Engineering loves SQLBits

Even more amazing papers at VLDB 2019 (that I didn’t have space to cover yet)

Using Real-Time Digital Twins for Aggregate Analytics

Using Real-Time Digital Twins for Aggregate Analytics

Track Thousands of Assets in a Time of Crisis Using Real-Time Digital Twins

Track Thousands of Assets in a Time of Crisis Using Real-Time Digital Twins

Stay Connected