10. Will AWS Be Crushed Under It’s Own Weight? (Part 01)

In episode 10 of Mobycast, Jon and Chris consider a reality where AWS crushes under its own weight. Welcome to Mobycast, a weekly conversation about containerization, Docker, and modern software deployment. Let’s jump right in.

Jon: This week, we’re going to talk about AWS again. The last few episodes we talked about various things that causes users on AWS and things that we’re looking at using of AWS. One of the things that came out of that was a conversation that Chris and I had in Brazil that’s just about AWS overall and the speed that which is growing and changing and how hard it is to keep up with all the new stuff that AWS is putting out.

I guess we’ll call these ideas that AWS will be crushed by its own weight. If they continue growing at the pace they’re growing at now and nothing changes, I don’t think that people will be able to keep up; I don’t think that it’s going to work for them. I think we can speculate a little bit about that. Let’s just start by talking about how fast it is growing and what we think will happen if nothing changes. Maybe Chris you can talk about that a little bit.

Chris: Sure, yeah. I’ve seen the grass. I don’t have them in front of me.

Jon: I can tell you one thing is that, 2016, they were up to 500 some capabilities and then the year after that it was 700 something. It’s not a straight line; it’s definitely a curved line up into the right and turning up. They’re accelerating.

Chris: Yeah, absolutely. There’s two ways of measuring it. One is the number of services that you see on the AWS console. I think that’s gone from a handful and when in the early days of AWS2, it’s about 150 now just on the AWS console. And then AWS will break it out and say—capabilities as well. Major features of those high level services as well. I think they were up over a thousand added in that year. They’re basically shifting three a day.

Pretty tremendous pace and obviously, they’re not going to slow down. Amazon is on a hiring bench, their aspirations are limitless. They are getting into each and every vertical there is out there and as they do that, there is more and more core services that they need to build those business and that’s AWS. They’ll continue expanding that. The pace is not going to slow down.

Jon: Let’s just think about that for a second. Three new capabilities a day, maybe even more than that, that means, if they take the weekends off, if you’re a developer, a CIO, or CTO, that’s 15 new things a week that may help you get your job done. Just even knowing what those are is one thing.

Rich: Can we define what the capability means in this context, are we talking about a new service, so it has a brand name for it or is it just something that’s a feature added to another service?

Chris: It’s a feature that warrants telling users about. Amazon, they basically publish, releases, that dictates here’s what new this week—type thing. I think it’s anything that is big enough to warrant inclusion in that; they would define as a capability.

Jon: Right. Rich, if one of the capabilities is – this service supports uploading by PDF document, just for example. That could be something that could really be useful to somebody. Knowing about that and having that as part of your body of knowledge in your tool belts could be great. But the real point I was trying to make is that I don’t think anybody has the mental space for keeping of top of 15 new things from AWS every week. Let alone, all the other things that you’re supposed to know about that have nothing to do with AWS like changes in your development language, changes with another third-party tools that you use, changes in the way people think about doing agile programming and project management, just all the things that are happening. Keeping up with 15 AWS capabilities a week is just unattainable, it’s kind of impossible.

Chris: I was just going to say that maybe just to try that example a little bit more—just kind of what the capability is. A very real world example recently of AWS announced that they released a new service discoverability feature in ECS where they’re now integrating ECS with Route 53 to do basically DNS updates as a way for applications to access your services instead of going through a load balancer.

Actually, like a pretty huge feature for AWS—it’s one of those three a day that they’re doing. Something like that actually has pretty big ramifications to someone that is using ECS.

Jon: Great example. Much more relevant to what we typically talk about. Thank you.

What’ll happen? Let’s just imagine if nothing changes, what’s going to happen to AWS and people that use it?

Chris: It’s going to get more complicated. There is more services, each one of those services would become richer, more features. I think if nothing else happens at the very least, people will just find their comfort zone of the services that they’re using to lay with and they’ll stick to that set and everything else will be just noise for them. That’s barring anything else, that’s what you have to do.

Jon: That’s exactly what I was looking for. It’s an important thing and it really matters to their market. Just before we got on this call, I was having a conversation of Slack group called Hangouts. Somebody said something a little snarky about ECS. Basically, the way the conversation was that somebody said, “Wow, with EKS,” which is AWS’s new Kubernetes based clustering and orchestration system, “With EKS coming out, what’s going to happen to all these people who have dedicated their careers to managing Kubernetes?”

The answer’s supposed to be, “Oh, they’re going to have to retool and find another jobs.” But somebody snarkily said that, “Well, if EKS is anything like ECS, then nothing. People will just go about doing the same thing they’ve always been doing.”

Underneath that snark and sarcasm was just this assumption that ECS was no good, that it’s a useless tool. The expectation would be that EKS would also be a useless tool. But then I challenged that, I said, “Wait a minute. We’re having some pretty good luck with ECS and I’m just curious what you don’t like about it, what’s not working for you at ECS? And why would you say that?” The answer was, “It was so narrowly focused, and it’s just kind of a toy and a lot of AWS services are pretty narrowly focused compared to their more open competitors.”

Other people within Hangouts, immediately jumped to ECS’s defense and said, “No, no, no. Wait, wait. You admit not having use ECS in a year and a half and it’s come a long way since then. It’s got all these features now and all these capabilities. It’s tied into the way. Load balancers work in a way that is super powerful and we’ve been loving it.” Everybody jumped on and defended ECS.

The point I’m trying to make here is that somebody evaluated ECS about a year and a half ago, decided it wasn’t for him and then moved on and really got entrenched to using Kubernetes. Meanwhile, ECS has been improving super fast. And now, it’s a really good tool. That person just didn’t know that and it’s not that person’s fault. He just hasn’t been able to keep up with the pace of AWS improvements. His view of the world is completely outdated now and I think he is the absolute example of what’s happening to everybody.

This is actually bad for AWS, because people are not keeping up with AWS then they have no idea. They may tell each other things that are not true about AWS and then cause people to not select AWS as their solution when maybe it would be a good solution for them.

I’m just trying to think what else might happen to AWS. I guess another thing would be full on services—complete services are just dying on the vine. That’s probably more natural. Have noticed that’s ever happened with any of those services? Can you think of, Chris, any services that they have just decided to sunset because they weren’t good enough?

Chris: I’m sure they have. I can’t think any of hand other than if I were to bet, I would say Beanstalk is definitely one of those services that we may see go away.

Jon: That’s a good example of one. Still, they have probably got a lot of usage and lot of people on it.

Chris: They do have a noSQL database that’s like Dynamodb lite . I would imagine that the use cases that support this dwindling rapidly and that service will go away. Absolutely, there is pruning that happens. This is no different than the company with a suite of products and some products are going to winners in the marketplace and other ones are not. It’s going to constantly be evolving. Absolutely, there will be some that wither away and get pruned off the AWS Tree, while many new branches are growing.

Jon: As we talk about this, one thing that strikes me is that AWS in general does seem to keep support for a lot of their products indefinitely. I think about Google, it feels like half the time I’m reading about new Google Services and the other half of the time I’m reading about Google Services that are going away. That doesn’t seem to be the case with AWS. It seems like you’re only ever reading about new ones, maybe because they start so good at vacuuming in development talent that they just continue to fan out and just maintain everything and new stuff at the same time.

Rich: I wonder if that has something to do with how AWS actually found itself into the market. AWS wasn’t called that but it was being used for Amazon.com for a long time before the rest of us started to use it, whereas, most Google products were really always built for other people to use.

I wonder if its growth doesn’t really matter or the fact that they have these products that we don’t use may continue to be used because it powers the Amazon.com like infrastructure itself. I think it’s an interesting thing about AWS, is if they’re launching three features per day, to what degree is that for everyone else and to what degree is that for them internally? Do they really need to have success at every corner with these products?

Chris: There’s no doubt that Amazon itself is big customer aid of US, although, it’s not the biggest customer. It is just one out there. I think it’s probably safe to say that just about every service or capability that AWS launches—it actually is coming from customer input. There are people out there that absolutely want that.

Maybe Amazon—they haven’t had this problem of constantly deprecating services just because they have some minimum threshold user base using it, probably doesn’t cost too terribly much to keep it going, especially considering their talent pool and just how many folks they have working for Amazon now. There is a bit of all that investment of just everything that they’re doing and the core infrastructure types that make it more economical to run those things that are less popular.

Jon: I feel this creates two business opportunities. Honestly, I do think that they’re making a little bit of a mistake by fanning out and growing, at breakneck speed without making any kind of prioritization clear and with addressing the overall navigatability problem of their products and services. This is a problem for them, it’s unapproachable.

This morning, I was having a meeting with one of the developers and he’s excited that he’s been doing a lot of backend work and feels he’s making good strides there but he felt one of his biggest weaknesses is, “We have to speed on AWS and all of the things that we view in there. It’s like, “Yeah, join the club, man.” Even everybody feels that way. From you all the way to me and Chris and everybody on the team it feels like we’re always a little behind them at AWS.

That constant feeling of feeling a little behind on it is not a good one as a software developer. You want to feel you’re maybe only slightly behind or you only have a couple things to catch up on but not like the boat is sailing away from you.

The two business opportunities that this creates are one—there’s a couple of other major cloud providers out there, Microsoft and Google, Azure, and G Cloud—I guess it’s called. Those have an opportunity to present themselves in a different light. I can already see Microsoft trying to present itself in a much more approachable way. I don’t have a lot of data there or detail. Do you have any more? Do you feel that that’s correct, my intuition that Azure is trying to make itself look a little more approachable than AWS?

Chris: I think Azure has a different target customer base, it’s more important to them. They’re definitely more enterprise and they have a lot more of folks trying to get them off of having on-prem compute to move into the cloud, they are much better in the hybrid space, just the enterprise tools, and MyCreation and whatnot versus AWS—it’s really built from the groundout. It’s really targeted on the web companies and startups and whatnot. Not having that, is like starting from day one in the cloud as opposed to having this story where you’re more established business that you not need to transition to the cloud.

I think there are different customer segments so that dictates how each of those businesses grow with, as you’re taking some different approaches than AWS.  That said, Azure for the most part, along with Google, they’re playing catch up to AWS, so AWS comes out with a major service that the marketplace is guaranteed 18 months later, Azure and Google have it. It happened with Lambda, it’s just across the board. They’re playing catch up.

Jon: I guess it remains to be seen whether people are aware of this problem, aware of the unapproachability of the proliferation of cloud services and whether either Google or Azure will take advantage of their opportunity to make it more approachable. At least for my perspective, in terms of the marketing that Azure has done—while I agree that they are marketing to that enterprise and absolutely they’re trying to get people off on prime service into the cloud, just something about it, maybe it’s the colors, the soft clouds. There’s something softer about it, something that feels a little bit inviting, “Join this,” more comfortable place to be in. I’m interested to see if they start to pull in web companies, that’ll be pretty fascinating.

Chris: I think I’m going to give mine too with Microsoft. They are a package software company. They take product design very seriously, they always have. They are basically a retail company. They’re building products for the retail space. That’s part of their deep DNA and that’s how they building at Azure. That’s why their software and their UIs are much more thought out and designed and integrated and consistent versus Amazon’s. It’s like, “You know what, we’re just going to give you APIs and we’re not going to have any consistency across UIs, which one of these services, some of them are going to be command lines, some of them are going to look like a high school project.”

But they’re going to get the job done. It’s not going to really going to slow you down but that’s just something that they’re prioritizing on. They’re not saying, it doesn’t need to be pretty and consistent and all integrated, instead we’re just going to give you the APIs that you need to get the job done.

Jon: One of the things that we wanted to do is a lot of these conversations have been around an hour and we realized that people’s time is pretty valuable and they may not want to listen to us every week, pontificate for an hour. We decided that we want to keep this down to more like about 20 minutes. Right now, we’re wrapping up on a thought.

AWS is going out of control and these other cloud companies have an opportunity. There’s still more to discuss here. I think there’s another business opportunity that I’d like to talk about. Something we can wait for the next episode of Mobycast to do that.

Chris: Sounds good.

Jon: Alright. Thank you, Chris and Rich for joining me this week and looking forward to talking to you again.

Chris: Take care. Great. Thanks guys.

Jon: Bye.

Dear listener, you made it to the end. We appreciate your time and invite you to continue the conversation with us online. This episode, along with the show notes and other valuable resources is available at mobycast.fm/10. If you have any questions or additional insights, we encourage you to leave us a comment there. Thank you. We’ll see you again next week.