11. Will AWS Be Crushed Under It’s Own Weight (Part 02)

In Episode 11 of Mobycast, we continue last week’s conversation regarding the negative implications of AWS’ rapid growth. Welcome to Mobycast, a weekly conversation about containerization, Docker, and modern software deployment. Let’s jump right in.

Jon: Welcome to number 11 of Mobycast, Rich and Chris.

Rich: Hey.

Chris: Hey. How’s it going, guys?

Jon: It’s going well here. I’m excited to do this one because I think we left a lot on the table with last week’s conversation about AWS collapsing under its own weight. But before we get started, I just want to find out from each of you what you’ve been up to this past week. Rich, what have you been up to?

Rich: This is the first week that I have felt that I’ve been able to take a breath in a while. We started out on a strategy at the end of last year that has really started to kick in and work. The downside to that is, we really have to start focusing on processes and stuff like that.

Last three months have been pretty stressful but I feel I’m finally above water, days are feeling normal again, I’m getting out and hitting some golf balls, so life is good.

Jon: That sounds pretty good. How about you, Chris?

Chris: I took a trip out to sunny San Diego recently to meet with a client and kind of have a collaborative onsite design session, working out a roadmap for things for us to work on, and kind of start to dig into how do we start building out some visualization capabilities for all the data that’s been collected by the software that we have been building. Kind of investigating couple of different prongs there for what might be the best way, especially given the fact that right now the data malls are pretty complicated from many disparate sources. That provides some unique challenges for us, having fun with that.

Jon: But more importantly, did you have any fish tacos or at least a burrito?

Chris: Unfortunately, no, but what I did have, though, I did get In-N-Out. Here in Seattle, there is no In-N-Out Burger, so when I’m in California, that’s definitely high on the to-do list. I indulge in a Double-Double and Fries, Animal Style.

Jon: As for me, a couple of weeks ago I told you bout that I’ve been reading a book by Ryan Holiday called Conspiracy, about how Peter Thiel took on Gawker, and that was so good. I didn’t remember the guy’s name was Ryan Holiday at the time, and then later I told Rich, “Oh yeah, it’s by Ryan Holiday,” and Rich was like, “Woah. No way!” He was the Director of Marketing for American Apparel and he’s written this other crazy book called Trust Me, I’m Lying. I read that one mostly this week and it just blew my mind.

It was written in 2012 and aside from maybe one paragraph in the entire book, that thing is relevant like it feels it was written in 2016. It really just talks about the danger of fake media, how to create fake news, and what will happen in a world with the kinds of pressures that we have on writers to produce cheap content as fast as humanly possible. It was eye-opening. I definitely recommend that if you’re listening and check out Trust Me, I’m Lying and then take all the news you read with a grain of salt.

Rich: That’s good advice.

Jon: All right, we’re going to continue talking about AWS becoming just too big for its own good and having too many capabilities for people to digest, and what’s going to happen with that. Last week, we kind of ended up talking about that there are two business opportunities that I had in my mind. One was for the other cloud providers be better at competing and maybe show a different face than what AWS is showing to the world.

The other business opportunity that I was seeing is just this possibility for third parties to put services together in a way that’s digestible and really for AWS to kind of move into the background.

I’m actually just remembering that there’s another part of this that I want to talk about that I want Rich to introduce. But let’s start with this third party businesses idea and maybe Chris, you can just discuss what you think third parties might be able to do to start to address this problem.

Chris: As things become more complicated, not even just more complicated, just as the volume increases for various services and capabilities that are out there by AWS, it kind of become impossible for any one person to keep up on all of that.

It’s not that they’re not smart enough or mentally capable of doing that. It’s just that there’s not enough hours in the day, so to keep track of all the best practices of using these things—all of these services they are not isolated, you put them together like Legos, building blocks to actually build what it is that you need for applications. You’re not going to use probably just one servicek you’re probably going to need to kind of put together 4, 5, 10 of these services or more.

There’s more of these things that is going up the stack and it’s more maybe application-level or subsystem-level building blocks of services that are composed of the baseline services.

Jon: Chris, I want to just put a pin in what you just talked about in terms of these building blocks of services, and underline something that you started with, which is not having time and maybe it’s not a good idea to spend all of that time that it would take to keep up with AWS. I just want to underline that.

If you are a CIO or a CTO and you’re spending a good part of your week just staying on top of what’s going on with AWS, or a good part of your quarter, that’s time that you’re not spending thinking about innovation for your domain, your own company. That’s got a real downside.

If AWS gets too complicated, companies are not going to be able to be as innovative if they have to know it all. I just want to underline that and then now, we can go back to this concept of building blocks and putting them together.

Chris: Think of them as recipes where there are going to be some pretty common recipes that folks can put together and kind of just say, “Here is a solution for you to use.” That way, you don’t have to figure out what is the best practice for stringing these six things together to do near-line low-latency archival of block-level data that is across multi-region, instead of knowing the intimate details of how to architect that. Like now, someone else have done the legwork for you and make sure that it’s up-to-date with the latest and greatest.

I think there’s lots of opportunity there that that’s going to be one of the biggest ways to address this pain of this furious pace of innovation in the cloud space with AWS and its services and capabilities, at least a void for folks to come in and to help basically be some shoulders that you can stay on top of and not have to be an expert at every single service, every single capability.

Instead, if you have a job to do and you have solutions that need built and basically you want to do that as quickly and efficiently and as best and brief as possible, getting help from third parties.

Amazon itself, they have a whole team. They call them Answers and they’re basically cloud formation stacks for doing these kinds of things. But even that is pretty low-level, it requires some heavy lifting.

I think there’s going to be more companies kind of in this space that provide the value-added services. It really becomes pretty straightforward and turnkey to use these things.

Jon: One of the things related to these third parties that I think that they can really address, that’s a pain point for Amazon is understanding billing and predicting billing. If you go to Amazon’s Help Desk, what did you call it again? What do they call it again? Not their Help Desk, but their solution or recommendation team. What was that called again?

Chris: Their consulting solution architects division, one of things they offer, they call it the AWS  Answers. These are cloud formation stacks for various different common use cases.

Jon: We’re actually using one of those for HIPAA right now. If you use those, then you’re still kind of stuck with, well, “I’ve gotten this many EC2 machines. I helped plan doing this much network traffic, I’ll need load balancers, I’ll need some lambda functions running, just adding and adding and adding and adding services and services and services. It’s this unbelievable nightmare to understand what the billing even is, and then what the billing is likely to be under different loads.

I think that is an area where third parties just have an incredible potential for demystifying and getting advantage over using their AWS. If they can make the pricing transparent, “This is what you want, this is what it costs,” as opposed to, “Here’s the 15 different ways this is going to cost you.” That’s going to drive adoption towards them, I think.

Chris: Yeah, there’s no doubt that’s one of the top concerns that folks would have is, “How much is this going to cost me and I don’t want to be surprised by a large bill,” just kind of having that predictability. That is very difficult right now.

It’s really difficult to understand, how do you build things so that they are the most cost-effective? Amazon gives you a ton of information on this. There are a bunch of various techniques for you to use. Spot instances or reserved instances, how you combine those two things instead of using on-demand instances.

You can save thousands of dollars a month off of your AWS bill, but it’s pretty difficult, pretty challenging. There’s no AWS Answers confirmation stack for that that I’m aware of. Just again, another opportunity in there, just all around billing, both in the inside into that and of predictability as well as like, “Hey, save me money.”

Jon: These third parties, they can improve on billing and predictability, they can improve around approachability and solution-providing. There’s so much room right now for third parties to jump in. This is a kind of nascent area. There’s absolutely companies that are already doing this, but I don’t think that there are any winners.

There’s not the one company that’s like, “Hey, we have 50 different AWS solutions and we are the guerrillas, the best at delivering, easy-to-use AWS solution.” But can you think of any companies that are starting to make headway in this area?

Chris: There’s none that stick out right now. It’s kind of ironic in a way, because there have been companies that tried to do this in the past and they didn’t do it well. We’ve kind of seen their offerings and their presence in the space, and it depends on the space and whatnot.

Rackspace is a company that comes to mind. They were, and I think they still do provide kind of managed AWS services. They will provide you a solution architect and will help you figure out what it is to build, but that whole experience wasn’t nearly as powerful as I think it could be or should be. I think there’s been some V1/V2 of this that the best is yet to come and there’s going to be a lot more, a lot better solutions, more delivered as a service, as opposed to the consulting and basically people.

You’ve already seen Amazon do some of this themselves with products like—Beanstalk was definitely one of those things that they launched to kind of make this much easier to folks to consume a whole bunch of various AWS services to deliver a web application quickly. That’s kind of being superseded by newer technologies now, but that was the problem that they were addressing there. They also have, more recently, Lightsail as a service to help you very quickly spin up EC2 instances, just get it up and running.

Amazon recognizes this. They know it definitely benefits them to take the complexity out of these products and services to get people up running as quickly as possible, so that they can start charging them as quickly as possible.

I think what Amazon does in this space will be very limited compared to what should be done and what could be done to a much greater extent.

Jon: You take the example of Lightsail, it’s in the list of services right underneath EC2. It’s not clear what it is. There’s no like, “Hey look here.” You just want to use a machine quickly. “This is where you should come.” There’s none of that. You’re just as likely to click on EC2 as you are on Lightsail. They both get you to the same end, you have a machine that’s running in the cloud, but there wouldn’t be a way that you would know that.

That actually kind of brings to mind that the user console, the administrative interface that AWS has and I think Rich, you had a story that relates to this and why AWS might not be able to solve this problem.

Rich: Yeah, and just to kick that off, Amazon has never been known for their UI, ever. Even on amazon.com, it still doesn’t look pretty, but it is one of the most converting websites on the planet.

I think that the biggest problem for a company like Amazon to really start investing in UI and UX is that it’s probably not part of their culture. It would be a really difficult thing for them to accomplish, especially now that they have all of these different services that would need to sort of adhere to the same UI, even though maybe some of them are slightly different.

The story that I always talk about is very recently, Atlassian updated the UI for Jira. A couple of years earlier, they acquired Trello and now, Jira looks like Trello. But about three months prior to that transition, I had decided that I was going to learn the complexity of Jira because I had heard so much about its power, and it was true.

Once you figured it out, it was very powerful and it was worth the learning experience. But then they transitioned to this new UI, which is undoubtedly more beautiful. It changed the way that you had to do things in order to take advantage of this power, so I was completely lost again.

In one day, although they allowed you to go back to the original or the classic view—eventually they were going to censor that—I was either going to need to relearn everything or I was going to leave, and I left.

The irony of that is that I use Trello now. Trello is simple. I can deal with the UI there, but Jira was so complex that I lost basically any foothold that I had to be efficient in the program.

Salesforce did the same thing very recently. They have a beautiful UI, but they’re smart enough to know that never change that experience on people, so the classic editor will never actually be sunset. Basecamp does the same thing. Whenever they change things, they allow you to go back. Google was around 15 years before they actually invested in UI and there was a lot of opinion about that.

I think it comes down to the idea that UI is not necessarily a subjective thing, but it does have a level of art in it, which is clearly subjective and I think that it is a very dangerous thing when you can train somebody on a poor user experience.

The beauty of the admin is that you’re not trying to convert people anymore. They’ve already paid, so if they find value in your product, they will learn that UI. The only thing that they can do is make it easier for new customers. But regardless, all of the legacy customers will be affected by it, and mostly in a negative way.

The idea of these third party companies can come in and create their own subjectivity as to how things should look is great, because every customer will be a new customer and so, they won’t need to worry about the way that things work with AWS.

I think that that opportunity is something that I would suspect Amazon really wouldn’t want to compete against, because it’s something that inherently is not a part of their overlying culture because they’d never really implement it in any of their products.

Job: I agree with that, Rich, and I think that this is going to be an interesting thing to watch. I think it’s almost like what’s happened with computer programming in general, where maybe many years ago, you had to know what was happening in terms of chip architecture and assembly languages in order to write good software, and then eventually you just didn’t need to know that stuff anymore.

I think the cloud may be sort of the same way where, in order to get business applications written, those of us that are in the business of building business applications, might not have to know anything about what’s happening on AWS.

We may be able to get our most innovative solutions out there by letting other keep her down the stack companies deal with AWS directly and provide us the tools and services that we need to create innovative solutions for business. It’ll be interesting to watch this transition take place.

I think we can wrap up pretty much with this, but I just want to let you know that I don’t want to end it myself. I do want to give Chris one more opportunity to say anything else that’s on your mind about AWS falling over under its own weight and what will happen in the future?

Chris: I definitely think the pace of innovation is definitely going to increase. We’re all going to have to figure out ways to mix that blend of getting stuff done versus kind of being aware of what’s out there.

For me, it kind of a core underlying principle I’ve had just for anything I do, is just identifying what is your core competency? What is it that makes you different? This is for any business or company. What is the secret sauce? What is your competitive advantage that you have that makes you unique and allows you to win? That’s what you should be spending most of your time.

If that means that being an expert in S3, DynamoDB, SQS and ECS, they may not be part of it. As the landscape evolves, kind of taking advantage of these other sources that allow you to get up to speed with those things without spending a lot of time so you can focus on your core competencies is going to be key to thriving in this environment of constant innovation as opposed to being crushed by the weight.

Jon: I think that’s a great way to end it. Thanks again, Chris and Rich, for another great conversation, and I’ll talk to you next week.

Chris: Thanks guys. Bye.

Well dear listener, you made it to the end. We appreciate your time and invite you to continue the conversation with us online. This episode, along with the show notes and other valuable resources is available at mobycast.fm/11. If you have any questions or additional insights, we encourage you to leave us a comment there. Thank you and we’ll see you again next week.