Will AWS be Crushed Under its Own Weight?

Pace of innovation

The pace of innovation inside Amazon’s AWS is impressive, blinding and maybe even a bit nauseating. While looking for charts about how many services or features AWS has had over the years, this telling one from re:Invent (it’s a couple of years old) popped up.

It looks like it is mislabeled because the chart shows the number of new capabilities per year and says “new capabilities daily” but what they’re saying with that label is that as of 2016, there were at least three new capabilities added to AWS per day! Extrapolating to 2018 we’re probably somewhere between five and eight new capabilities a day.

So if it felt difficult to keep up with AWS in 2014, and like a full-time job to keep up with it in 2016, now in 2018 it’s just downright impossible. Going out on a limb here, I’ll say no one can know all of AWS anymore, which is a problem for everyone. Not the least of which is AWS. And already within AWS there are some subtle signs that they recognize this issue. For example, Lightsail is meant to cover up some of the getting-started complexity that comes with provisioning new EC2 machines. Elastic Beanstalk is another AWS service that came along to give developers a shortcut for making deployment environments. But even with AWS making these helper services, there’s still a need to understand what’s really going on. For example, when you look at your bill, and you’ve been using Elastic Beanstalk, your statement won’t have a single line item for your Beanstalk usage, it will have lots of line items for all the core services that Beanstalk itself is composed of. If you don’t have at least some inkling of what those are, then you won’t understand your bill, and you definitely won’t be able to predict it.

The ever-rising bar of expectations

In its own tutorials and explainer videos, AWS likes to equate its services to building blocks. You put many of them together to build a solution. You might use five or six different blocks representing storage, another several representing computing power, and another few for network or I/O. When they’re all stacked just right, they make an application. But as applications and user expectations both get more sophisticated, a single deployment may use 10 to 15 or even 25 to 30 AWS services and within each of those services, the app may be using dozens of AWS capabilities. That’s a lot to keep track of, especially for teams that are struggling to keep pace with cloud innovation and really are better off focusing on innovation within their domains.

So what’s the problem with this? There are two major issues, the first being security. If the tools developers use are changing too rapidly and require too much of their attention span to learn, then they’re going to start making mistakes. Security mistakes are already happening all the time because people aren’t correctly using security tools and best practices even as AWS adds more features and capabilities to its security solutions. Security mistakes hurt us all, and the number of them will go up as AWS offerings get more plentiful.

A second significant problem is stifling innovation. If developers are spending all their time keeping up with AWS, that’s time they’re not spending building the next big thing in retail or hospitality or healthcare or payments.

Send IaaS to the back

So what’s going to happen? First, let’s talk about what won’t happen. AWS is not going to start sunsetting services or streamlining its offerings. In general, once an AWS service sees the light of day, it’s here for good. This is likely tied to Amazon’s relentless focus on customers. Taking away services would definitely displease customers who depend on them, so Amazon leaves services in place. They may move the focus of feature growth to other services, but they won’t turn a service off. This means that Amazon itself, through its own culture, will not be the ones to solve the problem of runaway complexity.

The main thing we’ll see is that AWS will become something fewer and fewer companies touch directly. We’ve already seen that AWS is not good at making user-friendly software nor do they excel at making understandable, easy-to-consume documentation. Don’t believe me? Just try to understand this description of what ECS is if you’ve never used it. Don’t expect them to figure out how to become developer friendly overnight. But we should expect them to continue creating capabilities and services that companies want at an ever-accelerating pace. Third parties are going to have to make these services understandable and usable for the masses. And third parties are going to stitch together AWS services in creative ways that solve problems for specific domains. They will own the difficult task of keeping up with AWS so that companies focused on specific fields, like healthcare or retail, don’t have to spend all their time on devops and infrastructure.

A new market is already emerging

This new crop of companies creating value-add services on top of AWS already exists, but there aren’t any clear winners — yet. Some examples are ContainerShip, which manages container deployments for AWS (as well as other clouds); AlertLogic, which is building security-focused solutions specifically for pieces and parts of AWS; and Heptio, which is creating managed Kubernetes offerings on top of bare EC2 clusters or on top of AWS’s new EKS service. In the next few years, expect to see dozens more domain and problem-specific companies creating specialized collections of AWS building blocks that then become the defacto way to build applications for downstream companies.

One of the best ways for this new crop of companies to compete and win market share is on price transparency and predictability. Customers will likely be willing to pay a premium for the peace of mind of knowing what they will pay for specific solutions at various scales. Companies that create solutions with AWS services as building blocks will be able to take advantage of the benefit of their own scale to arrive at this predictability — not to mention they can factor in their own expertise as a value add to compensate for the additional margin they seek over AWS’s base pricing.

It will be really interesting to see whether AWS lets these companies stay independent or tries to buy them all. Either way, AWS has in a way already fallen over under its own weight. Don’t expect it to make significant headway in improving its approachability. Meanwhile, the next round of companies that will act as the entry point to AWS are already writing code and winning customers.