Symphonia at Velocity 2018, and more Serverless Insights

This is the latest edition of our newsletter. To receive the email version please subscribe here.

Hello friendly Serverless Insights subscribers! Summer has arrived in New York City — a time when locals scramble to find rooftops, dodge a million dripping air conditioners, and wonder how bodegas are selling pumpkin beer already. This summer also marks the 4-yearly event that is La Copa Mundial (we only get Telemundo in my apartment, not Fox Sports Network) but since the good old US of A are absent from the men’s World Cup this year, football fever is distinctly frigid. Unlike the temperature outside.

John and I (Mike) have had a fun three months. Some in-depth and unusual client work (want to encrypt a petabyte of S3 storage? We’ve got you covered), partnering with friends on a fascinating open source project, our most comprehensive training course yet, plenty of speaking engagements (see our expanding portfolio of talks that we can give to your team here), and a revisit of an old friend. We’ll get to all of those later on, but first I’m going to start the news this time with a roundup of an interesting day last week…

News from the Serverless World

Last week I was at O’Reilly’s Velocity conference in San Jose. It’s a great event full of deep technology experience, and a whole breadth of diversity. I was fortunate to be both presenting a 2-day workshop (on AWS Serverless Architectures and Continuous Deployment) as well as hosting a full-day Serverless track of talks. This track was a good state-of-the-union type affair, so I thought I’d share what was presented.

I led off with a talk titled “Crossing the Serverless Fireswamp”. I presented a series of hazards that face the brave traveler as they journey through the Serverless Adventure — mismatched scaling of components, region instability, the dreaded Cold Starts, and more. Of course just presenting dangers is not enough, and for each ‘gotcha’ I also presented a number of mitigations. My overall point was that, yes, there are still difficulties with Serverless, but with a positive attitude, and some help from increasingly effective platforms and communities, most of these are surmountable.

Next up Lynn Langit gave a talk on Serverless SQL, updated for 2018. Lynn always gives deep insights to this area, especially with her current passion for bioinformatics. Summarizing the current offerings in this area from both Google and AWS, including demonstrations of both Glue and Athena from the latter, Lynn went on to show some inspiring ideas of combining Serverless SQL queries with Jupyter notebooks. Great stuff!

Donna Malayeri gave a wonderful talk at a Serverless conference in New York last year, showing a sequence of surprising capabilities of Microsoft Azure. Since then Donna’s been bringing her expertise to Pulumi, a startup promising to make infrastructure automation much more friendly and less, well, YAML’ey. In her talk at Velocity, Donna introduced a framework for helping decide what Serverless tooling is best for your team, and with it compared Serverless Framework, Terraform, Claudia.js, and Pulumi’s own offering. This was a good, balanced, assessment, and it was fascinating to see Pulumi’s “code over template” approach. Timely, as well, since in a keynote that same morning Bryan Liles stated “we need to move past templating” for application configuration. I wholeheartedly agree. Pulumi released their product this week, and I’ll be looking more at it.

There was a strong Kubernetes theme to the whole conference, so it was good for the Serverless track to represent, with Soam Vasani and Timirah Jones from Platform 9 — makers of Fission — the “FaaS on K8S” project. Soam and Timirah talked about the different ways to consider function composition with FaaS. This type of talk is close to my heart since I strongly feel we need to do a lot of deep thinking (and subsequent knowledge sharing) of design and architecture patterns with Serverless. This talk, and others like it, are going to be key as we move our industry forward in that regard.

Closing out the day we were glad to have Erica Windisch, CTO of IOPipe, join us, to give us her latest thoughts on “The State of Statelessness” . As she pointed out : with Serverless, yes, there are still servers somewhere, and so it is with state. When building an application targeted to a FaaS platform it’s key to understand all the state-management options available to you in order to build a clean, high-performance, design.

I’m glad that there are an increasing number of Serverless themed conferences now, and it was fascinating to have a day dedicated to the subject, with excellent speakers, in a top-quality infrastructure-themed conference.

News from the Symphonia World

Plenty of updates from John and I, so I’ll get stuck in.

One of the catalysts for starting Symphonia was the massive interest in my article on “Serverless Architectures” that is published on Martin Fowler’s site. Two years, and half a million plus views, later, it was starting to show its age. A top-to-bottom review has brought it right up to date — you can read details of the updates here.

While we like to speak, write, and teach about Serverless, our consulting has a broader remit. From data engineering, to cost management, via conversations about team dynamics and architecture, we like to get involved with all-things-cloud-and-DevOps related at our clients. This has proved especially true in the last couple of months, as we helped a company update it’s entire AWS infrastructure in a number of critical ways. A particularly thorny part of this was encrypting billions of objects in a petabyte’s worth of S3 storage. This is not as easy a task as one might imagine. We’re always looking for ways to use our Serverless knowledge to good effect, and even in this task our experience with AWS Athena reduced the effort requirement significantly.

In the talk I mentioned above from Lynn Langit, she described a process of using AWS Glue, S3, and various other tools to build a compelling Data Lake platform. But how do companies put all of these tools together? Over the past 9 months John’s been helping our friends at Beyondsoft with their new open-source Serverless data lake project — ConvergDB. ConvergDB is designed to create and manage Serverless data lakes with a DevOps friendly workflow. Users describe the structure and behavior of their data, then ConvergDB creates the infrastructure and scripts to do the heavy lifting of optimization and transformation. Beyondsoft are offering a 3-day quick start to demonstrate the power of this approach — you can get in touch with them here.

Finally, I mentioned above the training course I gave at Velocity, and it was one of our most in-depth yet. Running the gamut of AWS Serverless technologies, a healthy smattering of Architecture, and a backbone of effective Continuous Delivery techniques, this is a launchpad for Serverless success. We’d love to share this course with your company — drop us a line and we’d be happy to chat about doing exactly that.

Closing out

Thanks for reading our latest newsletter, and we trust you found something useful here. We continue to enjoy your suggestions for improvements and additions for future issues.

Our mission at Symphonia is to improve the effectiveness of organizations through Serverless, DevOps, and related practices and philosophies. Please email us at johnandmike@symphonia.io to chat about how we might help you, or just to share ideas how the Cloud is helping your team, and your business.

To receive the email version of this newsletter please subscribe here.