…but developer productivity and commercial flexibility are bigger
This post goes into technology a little more than usual so the TL;DR version for busy CIO’s is:
- Please make sure some of your teams are experimenting with serverless technology from Amazon or other providers.
The rest of this post explains why Amazon is leading the transformation of software development with an obscure bit of technology from Google.
A long time in the making
Great ideas can sometimes simmer in obscurity for many years until someone finds the right commercial and technical package to ignite them. Then the old idea can burst into the headlines as a brand new innovation. This is happening now with a collection of ideas being branded as “serverless” computing. Amazon Web Services attracted all the headlines in 2014 when they released their Lambda service. I suspect Amazon will claim the bulk of serverless revenues over the next few years but the ideas and technology behind Lambda are at least a decade old.
Node.js is small and fast enough to be disposable. It is economical to start a node.js server in a container just to run a trivial bit of software - a simple function - and then throw it away once the function has completed. The whole cycle might last just a few tenths of a second. This is just what Amazon needed as the basis for its new serverless offering. Lambda now supports a range of languages and tools but still uses node.js as the small, fast workhorse behind the scenes. But cloud service performance is not the only, or even the most important, factor. Developer performance and commercial flexibility are also big considerations.
This means you can demonstrate an application or service on a relatively low specification laptop, trial and scale it at very low cost as cloud functions and switch to dedicated virtual servers (in the cloud or in your own data centres) when demand is stable or the scale makes Lambda’s pricing model less economical. I am not sure that Amazon set out to achieve this kind of portability or that Amazon’s competitors have realised how significant this is.
Try before you buy
This sounds too good to be true but the tools and frameworks are maturing rapidly and it is already a realistic option for interested, part-time, amateur developers - like me.
I’ve recently ported a web app that I use for my business over to node.js on Lambda. With a few configuration settings the same software will run in a virtual machine on my laptop or a physical or virtual server in the cloud. In fact, I can run different parts of the app on different infrastructure e.g. serve end-user parts over Lambda and keep admin features on private machines. As a bonus, my level of use is probably low enough to stay within Amazon’s free usage tier.
And the runner-up prize goes to…
Here is a final word of commiseration for Google - they had the right ideas but at the wrong time. Unfortunately, when Google launched its serverless offering, Google App Engine, back in 2008, node.js did not exist and many popular development frameworks did not support the serverless model very well. App Engine was good but something developed for the Google platform would not work very well, if at all, on private servers or another public cloud platform. Google had to back-track and offer virtual machine services alongside Amazon, Microsoft and others. Google is now moving slowly to offer a Lambda look-alike service. This time, Google is giving a lot more attention to portability and other major players should do the same.
An even more technical footnote
I had already worked with node.js and express for some voluntary work so I jumped straight to the the aws-serverless-express library to get started with AWS Lambda. The deployed version of my app makes use of other AWS services, such as S3, DynamoDB, API Gateway and Cloud Front, as well as Lambda but the platform specific details are taken care of by one or two configuration files. The application software will run on any instance of node.js. I found Keith Cirkel’s post about npm really useful in deciphering what AWS had done in their library and then extending it to avoid storing any sensitive information, like access keys, in files locally or in the cloud.