Optimizing Database Performance and Efficiency


It's easy for modern, distributed, high-scale applications to hide database performance and efficiency problems. It is a recursive method that can be used at any scale, from a data center with many types of databases cooperating as one, to a single server and drilling down to a single query.

MezzFS?—?Mounting object storage in Netflix’s media processing platform

The Netflix TechBlog

It’s used extensively in our media processing platform, which includes services like Archer and runs features like video encoding and title image generation on tens of thousands of Amazon EC2 instances. in Atlas , Netflix’s in-memory dimensional time series database. MezzFS?—?Mounting

Media 283

Impact of Data locality on DB workloads.


In this video I migrate a Postgres DB running PGbench benchmark. The aim of the experiment was to see how data-locality might affect a running database workload, not to generate the maximum TPS. Databases Performance Testing locality nutanix postgres

Titan Graph Database Integration with DynamoDB: World-class Performance, Availability, and Scale for New Workloads

All Things Distributed

It opens up the possibility to enjoy the value that graph databases bring to relationship-centric use cases, without worrying about managing the underlying storage. Social media apps navigate relationships between friends, photos, videos, pages, and followers. Enter graph databases.

Performance gains for postgres on Linux with hugepages


pgbench creates a ~7.4gb database using a scale-factor of 500. Then reboot the kernel I used these excellent resources How to modify the kernel command line How to enable hugepages and this great video on Linux virtual memory. Databases Tuning hugepages linux postgres tuningFor this experiment I am using Postgres v11 on Linux 3.10 kernel. The goal was to see what gains can be made from using hugepages.

Stuff The Internet Says On Scalability For October 5th, 2018

High Scalability

Hey, wake up! It's HighScalability time: Halloween is early. Do you know what's hiding inside your computer? Probably not. bloomberg ). Do you like this sort of Stuff? Please support me on Patreon. I'd really appreciate it. Know anyone looking for a simple book explaining the cloud?

Bandwidth or Latency: When to Optimise for Which

CSS Wizardry

If you’re streaming video, the difference between a 2Mb 1 connection and a 20Mb connection will surely be appreciated. When it comes to network performance, there are two main limiting factors that will slow you down: bandwidth and latency. Bandwidth is defined as….

Lerner?—?using RL agents for test case scheduling

The Netflix TechBlog

Netflix engineers run a series of tests and benchmarks to validate the device across multiple dimensions including compatibility of the device with the Netflix SDK, device performance, audio-video playback quality, license handling, encryption and security. Lerner?—?using

Jira Cloud, Jira Server, and Jira Data Center Integrations


Integration can help unify the information in one place (either a Jira or a database), so you can run reports and analytics across all of your teams. The video below shows a seamless integration between two Jira instances/teams to allow for visibility across the board.

The new MongoDB support for NServiceBus

Particular Software

As NoSQL databases have gained in popularity, MongoDB has emerged as one of the go-to choices in the field. MongoDB databases provide high availability and scalability, features which are near and dear to our hearts here at Particular, so it was an easy decision to provide a MongoDB package.

Understanding APIs: What They Are and Why They Matter


You’ve probably lost track of how many applications are on your phone or tablet – applications for shopping, ordering food, checking the weather, reading the news, tracking the stock market, and sharing images, videos, and other content with friends and family (and maybe a few strangers).

Amazon DynamoDB - From the Super Bowl to WeatherBug - All.

All Things Distributed

In the five months since it launched in January, DynamoDB , our fast and scalable NoSQL database service, has been setting AWS growth records. If we had chosen to use a more traditional database approach, we would still be in the process of designing the system for horizontal scalability in addition to negotiating license agreements.â??. s really tough to quickly scale a normal database system to handle that kind of rapid increase in load, and our system was being strained.

Games 65

Stuff The Internet Says On Scalability For June 29th, 2018

High Scalability

karmakaze : The key difference is that NoSQL was being developed and used to solve specific issues with using traditional databases. Hey, it's HighScalability time: Rockets. They're big. You won't believe how really really big they are. Corridor Crew ). Do you like this sort of Stuff?

Stuff The Internet Says On Scalability For June 15th, 2018

High Scalability

1.6x : better deep learning cluster scheduling on k8s; 100,000 : Large-scale Diverse Driving Video Database; 3rd : reddit popularity in the US; 50% : increase in Neural Information Processing System papers, AI bubble? Hey, it's HighScalability time: Scaling fake ratings.

Procella: unifying serving and analytical data at YouTube

The Morning Paper

to understand YouTube video performance) drive tens of thousands of canned (known in advance) queries per second, that need to be served with latency in the tens of milliseconds. Given sufficient memory, Procella can essentially become an in-memory database.

AWS Elastic Beanstalk: A Quick and Simple Way into the Cloud - All.

All Things Distributed

Flexibility is one of the key principles of Amazon Web Services - developers can select any programming language and software package, any operating system, any middleware and any database to build systems and applications that meet their requirements. Last week I ran into an AWS customer at CES who was enthusiastic about how his digital production workflow and video encoding is now running reliably in the cloud. Job Openings in AWS - Senior Leader in Database Services.

Java 51

Fallacy #4: The network is secure

Particular Software

Design secure systems: See what Udi has to say in Day 1 of his Advanced Distributed Systems Design video course. It's very sad that, in this modern era, the vulnerability at the top of the list is a simple injection attack , which is trivially avoided using parameterized database queries.

Stuff The Internet Says On Scalability For August 3rd, 2018

High Scalability

It was the very first time (1968) the world had ever seen a mouse, seen outline processing, seen hypertext, seen mixed text and graphics, seen real-time video conferencing. Alok Pathak : While both (Multi-AZ and Read replica) maintain a copy of database but they are different in nature.

A Decade of Dynamo: Powering the next wave of high-performance, internet-scale applications

All Things Distributed

We had an advanced team of database administrators and access to top experts within Oracle. We were pushing the limits of what was a leading commercial database at the time and were unable to sustain the availability, scalability and performance needs that our growing Amazon business demanded.

Fallacy #3: Bandwidth is infinite

Particular Software

We amaze ourselves by our ability to have a real-time video conversation with someone on the other side of the world, but then react with extreme frustration when the connection quality starts to dip and we must ask "are you there?"

Front-End Performance Checklist 2019 [PDF, Apple Pages, MS Word]

Smashing Magazine

Paddy Ganti’s script constructs two URLs (one normal and one blocking the ads), prompts the generation of a video comparison via WebPageTest and reports a delta. Are videos properly optimized? Addy Osmani recommends to replace animated GIFs with looping inline videos.

Cache 96

AWS EC2 Virtualization 2017: Introducing Nitro

Brendan Gregg

But not all workloads: some are network bound (proxies) and storage bound (databases). ## 5. It was covered in detail today in a talk by Matthew Wilson and Aaron Blasius ([CMP330]: video ).

40+ Best Web Development Blogs of 2018


Its videos and blog articles address issues such as web performance, extensible component development and the intersection of CSS with other technologies, like HTML and JavaScript. In the world of web development, those who become experts usually do so by learning from their predecessors.

Speed Matters talk at Auckland Web Dev Nights

Speed Curve

Real User Monitoring (RUM) Pingdom New Relic (also backend, database and server health monitoring) Google Analytics mPulse. Impact of performance improvements Optimizing the Critical Rendering Path for Instant Mobile Websites (Video) What are responsive websites made of? Responsive web design is solid gold Etsy - Building Resilient User Experiences Velocity Conference Videos Browser Diet References. Here's the slides from my presentation at the Auckland Web Dev Nights meetup.

How Improving Website Performance Can Help Save The Planet

Smashing Magazine

The Green Web Foundation maintains an ever-growing database of web hosts who are either wholly powered by renewable energy or are at least committed to being carbon neutral. Are we embedding YouTube videos? How Improving Website Performance Can Help Save The Planet.

DynamoDB for Location Data: Geospatial querying on DynamoDB datasets

All Things Distributed

Over the past few years, two important trends that have been disrupting the database industry are mobile applications and big data. These factors have made DynamoDB a compelling database for mobile developers, who happen to be among the biggest adopters of this technology.

Build A Seamless Spreadsheet Import Experience With The Help Of Flatfile.io

Smashing Magazine

The last thing users want is to sift through lengthy documentation or video tutorials to upload a simple spreadsheet. Broken data, which your team has to manually clean before it’s usable in your database. Build A Seamless Spreadsheet Import Experience With The Help Of Flatfile.io.

Azure 52

Expanding the Cloud: Docker Containers in Elastic Beanstalk

All Things Distributed

To get started with Docker and Elastic Beanstalk, check out this short video below or see Jeff''s post for a few samples

AWS 79

USENIX LISA 2018: CFP Now Open

Brendan Gregg

To learn more about how LISA has evolved, browse last year's LISA17 conference program , and see the slides and talk videos. Join us for 3 days in Nashville at LISA'18. Post by Brendan Gregg and Rikki Endsley.

Accelerating Data: Faster and More Scalable ElastiCache for Redis

All Things Distributed

Fast Data is an emerging industry term for information that is arriving at high volume and incredible rates, faster than traditional databases can manage. While caching continues to be a dominant use of ElastiCache for Redis, we see customers increasingly use it as an in-memory NoSQL database.

Taking DynamoDB beyond Key-Value: Now with Faster, More Flexible, More Powerful Query Capabilities

All Things Distributed

We launched DynamoDB last year to address the need for a cloud database that provides seamless scalability, irrespective of whether you are doing ten transactions or ten million transactions, while providing rock solid durability and availability.

Games 83

Stuff The Internet Says On Scalability For April 5th, 2019

High Scalability

It's HighScalability time: SHUTDOWN ABORT the last Oracle database running Amazon Fulfillment! million : emails sent during 10 day spam campaign; Quoteable Quotes: @KimZetter : Crashed Tesla vehicles sold at junk yards and auctions found to contain invasive personal info on driver, including phonebook and calendar info from drivers' paired mobile devices; also found unencrypted video showing what happened just before accident. Wake up!

Expanding the Cloud ? Managing Cold Storage with Amazon Glacier

All Things Distributed

One factor in this growth is the presence of image, audio and video archives, which are often archived in their original un-edited form. Amazon Glacier integrates seamlessly with other AWS services such as Amazon S3 and the different AWS Database services. All Things Distributed.

Driving down the cost of Big-Data analytics - All Things Distributed

All Things Distributed

If you want a quick introduction on how to get started with mixing Spot Instances with On-Demand Instances in an Amazon EMR cluster, watch this Getting Started Video. a Fast and Scalable NoSQL Database Service Designed for Internet Scale Applications. All Things Distributed.

Amazon Cloudfront is Streaming Media 2010 Editor's pick - All.

All Things Distributed

As Dan Rayburn wrote on his Business of Video blog, "Amazon will be in the drivers seat to own the market for small and medium sized content owners who need simple delivery at a great price.". a Fast and Scalable NoSQL Database Service Designed for Internet Scale Applications. Job Openings in AWS - Senior Leader in Database Services. All Things Distributed. Werner Vogels weblog on building scalable and robust distributed systems.

Media 40

5 Terabyte Object Support in Amazon S3 - All Things Distributed

All Things Distributed

This is especially true for customers managing HD video or data-intensive instruments such as genomic sequencers. a Fast and Scalable NoSQL Database Service Designed for Internet Scale Applications. Job Openings in AWS - Senior Leader in Database Services. All Things Distributed. Werner Vogels weblog on building scalable and robust distributed systems. Big Just Got Bigger - 5 Terabyte Object Support in Amazon S3. By Werner Vogels on 09 December 2010 11:30 AM. Permalink.

AWS 50