85. Serverless Containers with ECS Fargate – Part 1

Voiceover: The Amazon Elastic Container Service, or ECS is the container orchestration system built by AWS. ECS features tight integration with many AWS services and is a powerful choice for running containers in the cloud. We first discussed ECS back in Episode Three of Mobycast, but so much has changed in the 18 months since then, bringing many major improvements. One new feature is Fargate, which brings the serverless concept to containers.
With Fargate, you no longer need to manage servers, you don’t pay for idle CPU, and there’s the promise of seamless scaling. Sounds perfect, right? Well, as with most things that sound too good to be true, there are some limitations and gotchas. In this episode of Mobycast, Jon and Chris kick off a three part series where we dive deep into Fargate. We learn the ins and outs of running containers without managing servers and discuss the trade offs we make when ditching those servers. Welcome to Mobycast, a show about the techniques and technologies used by the best cloud native software teams. Each week your hosts Jon Christensen and Chris Hickman pick a software concept and dive deep to figure it out.

Jon: Welcome, Chris. It’s another episode of Mobycast.

Chris: Hey, Jon it’s good to be back.

Jon: Yeah, good to have you back. Let’s start. Because we’re doing a new series today, we have a little bit of time we’re not deep in the middle of something that we’ve got to jump into right away. So is there anything new going on with you?

Chris: I am deep in the throes of a bathroom remodel.

Jon: Oh so fun.

Chris: So fun. Yeah. So I took on the task of doing the demo. That was quite the job for sure. So it was getting rid of the… we had a built in vanity with double sinks with bath tub, shower. So just tearing that all apart and then once that’s done then fixing some rot in the sub floor and moving the drain for the tub to meet what the new tub needs to be, and decided to move that drain of four inches so just a lot, a lot of work but it’s coming along and now I’ve kind of handed over the reins to someone else, and so they are now taking it from here. So I can now recover a little bit of my body.

Jon: Excellent. Yeah and bathrooms still are mostly on-prem.

Chris: I don’t know how that would work doing the bathroom as a service.

Jon: Yep. In the cloud. Yeah, I don’t have anything new. I don’t think to say, really, just everyone in Colorado where I am is talking about how today is in the 70s and tomorrow it’s supposed to be like 17 and snowing. There’ll be a 60 degree drop overnight, and that’ll be fun, but I’ll believe it when I see it. Lately, at least here in Colorado, the disaster forecasts have been all wind and no real bite. I’m expecting some wind tomorrow and kind of cool temperatures.

Chris: Yeah, it’s really, really cold here in Seattle. It’s that wet cold. So it’s not, I think in the mornings it’s been the upper of 30s, and then today, I don’t think it’s going to get much above like 48, 49 but it’s a wet cold.

Jon: That’s the worst. The worst.

Chris: You just feels so much colder now.

Jon: Especially if it’s windy at all. Oh, get me out of there.

Chris: Yeah.

Jon: All right, so this week, we’re going to start talking about UCS. And UCS is a service of AWS. It’s a container orchestration service. And I think I have a question, Chris, if somebody is listening, who’s already decided they’re going to do Kubernetes, or pretty much decided that they’re not going to use ECS, is there any reason for them to listen to this ETFs episode? Or, and we might even do a couple episodes on ECS. Is there any reason for them to listen to this, or should they just skip it?

Chris: No, I mean, I think I would urge you to stick around and take a listen just because, I mean, a lot of these techniques and just ideas apply to containers in general, and just how you go about scheduling, running, monitoring those containers in production. So ECS is Amazon’s orchestration system for containers. It does support Kubernetes as well. It supports a serverless model, which is what we’re really going to dive into today, as well as kind of like the original flavor of ECS where you’re backing it with the cluster of EC2 instances. So, I would say yeah, absolutely. There’s gonna be information here that hopefully is useful, and enter-

Jon: Entertaining, that’s where you’re going.

Chris: Yeah, yeah.

Jon: If entertaining.

Chris: If we’re not here to educate, we’re here to entertain. How’s that?

Jon: Cool, cool. Alright, so, yeah. So another thing about this week’s episode is that we’re probably going to refer to some things about containers, and how they work, that rely on knowledge from the last series that we just finished up on VMs and containers. So if you haven’t listened to that, and it may be worth at least listening through or reading the transcripts, just so nothing surprises you when we talk about containers.

Chris: Absolutely. Yep, definitely building on that. And then, you know, this was one of the original episodes that we did with Mobycast, right? This is back, way back in Episode Three of Mobycast, we talked about ECS and the basics. And so, definitely a lot has changed since then. And that’s why we’re kind of circling back to it now. But following that same arc, right? We started off with containers and virtual machines, and what’s the differences between them? And then we quickly went into, “Okay, now that we know what containers are, how do you actually use them in practice?” And so we did the episode on ECS. And so now we’re kind of doing the same thing, where we revisited the containers versus VMs, this time taking a much more closer look at it.
And now we’re going to do the same thing with ECS but also really, there’s been so much development in this space, and so much that’s new. Like it just really warrants to go back, go and look at, and see what’s happened in 18 months since we last talked about it.

Jon: Yeah, reminds me of something that I read recently or saw online. You know, there’s a guy named Cory Quinn and he is involved in about 80 to 90% of every conversation that happens about it, AWS on the internet. And he was saying that somebody asked him what are some of the biggest mistakes that people make in the cloud specifically with AWS and, and I agreed with his point. His main point that he made was that people make the mistake of learning a service and then thinking they know it, you know, then six months goes by, a year goes by, and they haven’t spent time updating their knowledge. And really, they don’t know it anymore.
After six months to a year, the services typically changed significantly. So it’s been, you know, like Chris said, What, 17, 16, 17 months since we did the ECS episode. It’s not the same ECS anymore. So, agree with Cory coin on that one.

Chris: Absolutely. We talked about this so many times, right? The pace of innovation is just accelerating at breakneck speed. So it’s the number of new, just announcements, it’s like, it’s 1500 or 2000 announcements a year of product, and that’s just from AWS. But [inaudible 00:07:58] what’s going on right? So, scores of those that relate to things, just a particular technology or service like ECS.
So yeah, you have to stay on top of it. Otherwise, you could be an expert in ECS circa 2018, And you can continue using it that way, probably, for some time. But-

Jon: Yeah, I mean, they never really changed. It’s always backwards compatible. Yeah.

Chris: But it’s, you’re not getting all the benefits anymore.

Jon: Great. Alright, so let’s get into the nitty gritty. Go ahead, Chris.

Chris: Yes, so maybe just first, just to kind of frame this, is just do a quick recap of what we did talk about in Episode Three in that, just what is ECS? Kind of like the main components, right? So again, ECS is stands for the Elastic Container Service. It’s the orchestration system for containers provided by AWS. And there’s multiple orchestration systems out there again, Kubernetes being one of the most popular one out there that, you know, most everyone has heard of.
But one of the real benefits of ECS, is its deep integration with other Amazon services, right? So if you’re in AWS, it makes a lot of sense to be looking very hard at ECS, if you’re running containers on, inside Amazon. And so, some of the core components in ECS that we will be talking about, a cluster. And so this provides a logical grouping for your containers basically, to be grouped together.
When we first talked about this, the first go around, the cluster was actually a physical thing, right? It was actually a collection of EC2 instances.

Jon: Yeah.

Chris: On which your containers would run. And so now we need to change this definition a little bit and say, it’s really a logical grouping, because we’re gonna throw in the, the Fargate launch type now, which essentially is a serverless model of running, so you don’t have to have EC2 instances that you spin up yourself. Instead, you can just say, you have access to these virtual resources, if you will, right? So, so that’s a cluster.
Again, if you’re going to be running on your own EC2s, then those are going to be defined by launch configuration and auto scale group. And again, that’s, we kind of talked about that in depth and Episode Three. Another key component definition is service. So this is basically a description, that allows you to run and maintain a specified number of containers. And it’s really for long running applications.
So you may have an API service, a microservice, web service. Some kind of piece of, some sort of code that you want always running. And you want to make sure that you have like X number of instances of it, or you want to be able to scale it up or scale it down, based upon load and demand. You want it to be monitored so that if failures happen, the scheduler will then go ahead and see that replace it. So a service is that definition that says that puts those characteristics to that of how you want your containers to be managed for you.

Jon: So Chris, I’m a little surprised that you decided to mention service before task this time around, because I guess the way I’ve always thought about ECS is, a task is a really easy thing for me to understand. It’s like one shot at running a container. The container has a command in it that it does. And that’s, that you basically say, hey, ECS, this, this container and its command, that’s a task, I’ll give it a name. And that’s your task. When you run this task, you’re going to go execute that container.
And then a service is basically saying, do this task. Always, make sure it’s always running. And you can, have more than one of them running, or just one or whatever. But you basically say, turn this task, which may be node process or something like that, or maybe some Python service or whatever, but it’s a task and it’s basically saying, “Keep it going, keep this thing running, and maybe scale it.”
So that for me, building from tasks to service was as easy. Was there a reason you decided to switch orders? Or was it just random?

Chris: Just I think, for me, it’s just kind of like a hierarchy thing, right? A cluster is at the top. A cluster is just a collection of-

Jon: Oh, got it. Got it. Got it.

Chris: Zero to end services and those services are then a collection of tasks, right? So just from a hierarchy standpoint. And I think most the time when we are using something like ECS, we are thinking in terms of services. In addition to servers, there are two other ways to run task on ECS. You can run them one off. Just, hey, here’s, I just want to run this task, and let it complete, and that’s it.
Or you can also have a scheduled task. So you can say, “At this certain frequency, I want this thing to be kicked off and run and whatnot.” Which is really just a special case of the run once. But those are less frequent, right? Usually when we’re when we’re thinking about that, it’s more like in the case of these long running services. So it’s like, “Oh, I need to run my web service.”

Jon: Got it.

Chris: It’s not like I need to run my container, or I need to run my node task. It’s like, “No, I need my, my web service. And it’s going to be needs a top TLS, and it needs to connect to this database, or what not.

Jon: Oh, sorry. Yeah. And so, I’m going to beat this to death. Just because, I think for me, and hopefully for a few listeners, it’ll make sense to go the other direction. And the reason it makes sense to go the other direction is because we learned what containers are. And then we learned what they do. They run, some command there, or process, and so that process, would give it a name and tell AWS about it, and tell AWS what Docker image it is. And AWS has decided that’s a task. And okay, so we get that, that’s easy.
Now if you want to keep that thing running, and maybe have three or four, or 300 or 400 of them going at once, and always going on and never less than two of them, say, and that’s a service. And then if you actually want to know where those services are running? On like what physical computers, and as you said, these days, you may not even know what physical computers they are, if you’re using Fargate, but if you are using these EC2 instances, and that’s actually literally a cluster, in the traditional sense of the word. Or in the New Age sense of the word. It’s just a logical cluster for Fargate, but like building up from that, in that direction just sort of makes sense to me from where we’ve been to now. So, there

Chris: And there you go. Yeah. And it’s again, it’s a task service, they’re all intertwined, and the truth, I mean at the end of the day, you have to create your test definition before you create your service anyhow, type thing.

Jon: Mm-hmm (affirmative). Mm-hmm (affirmative). Cool, cool. Now that we’ve argued about it, hopefully everybody’s like, “Oh, yeah, the ECS has tests and services and clusters.

Chris: Yes.

Jon: Next.

Chris: Yeah. And so those are the main components we talked about back there in Episode Three. And then we also kind of walked through just a simple deployment, like how do you actually do this, and so we talked about how first you’re going to create your Docker image, then you’re going to publish that image to some repository, whether it be Docker Hub, or use AWS’s repository, which is called ECR Elastic Container Registry, repository. And then once that image is now published as an artifact, you then would create your task definition revision to say, “Hey, I now want my task to use this new version of the image instead of the previous one.” And then after that, I would then update the ECS service to use that new task definition version. Right? So that was the simple, simple case of just really like, how do you deploy on ECS? And that’s what we were able to cover in that first episode.

Jon: I just there was a little bit of mixing in what you were just saying that I wanted to tease out just to make sure that we’re real specific. Deployment, that I guess what you were talking about, would be a deployment where you had already done a deployment, and you were doing an update, because you had mentioned, you wanted to use a new, a new ID or a new instance, or a new image, as opposed to just you know, “Hey, use this image.” On the very first deployment, you’ll be picking your Docker image and saying, “This is the thing I want to use for my task.” And not until you update it, will you be changing that?

Chris: Yeah.

Jon: You might not even realize you said that.

Chris: Yeah.

Jon: I just wanted to clarify.

Chris: It’s a bootstrap thing, right? In order to talk about the case where you’re doing it for the first time, you’re actually building up everything. You’re actually going and create in the ECS service itself and, and specifying all those parameters and then giving it the test definition, and then it’s just… So there’s that bootstrap process. And then after that, you don’t have to do it anymore. Now deployment is build your new image, publish it, make your new task definition that is referring to the new image and then you just update your service.

Jon: There we go. Cool.

Chris: So that’s kind of the just the kind of like the more common thing that folks are going to do the steady state because you’re not doing that that one off-

Jon: Right.

Chris: Process of creating, doing the bootstrap.

Jon: So in like 10 minutes, we just easily breezed through ECS. It’s such a fun little service, happy little clouds, happy little trees. But I have to say that I was watching somebody whose title was chief architect over in another Slack that I’m involved in, doing what we just did on his own. And, I watched the questions come in, like, “How do you do this? How do you do that? What about my terraform? What about this? I don’t understand why this isn’t working with these roles. What did you say? What is that about?” You know, auto scale a group, and, and just so many questions, and it really took a couple of weeks of questions before, before, I was very clear that he had got his first production instance up.
And I guess I say that just because this stuff is hard. There’s a lot of choices to make. There’s a lot of things that are really opaque and the AWS console, and then if you’re terraforming it, or using cloud formation, that too is something to learn, like how do you refer to an auto scale group or launch configuration and cloud formation? Is that inside your ECS area, or is that outside of it? There’s just so much here, so that easy, easy peasy Breezy 10 minutes of, “This is the high level ECS.” Is, I think actually would have helped that person, would help anybody that’s about to do what that person was just doing.
I feel bad. I was, during the time that he was struggling with it, I knew we were going to do these, and I knew it was going to be too little too late. So I didn’t even mention it. I didn’t want him to feel bad. Like, “Ugh.” Because he’s got a schedule he’s got I mean, he can’t wait for us to release Mobycast, like six weeks after I tell him it’s coming. But yeah, this would have helped him I’m sure.

Chris: Yeah. And we talked about like the general outline of ECS and just how to go and, create a service, and deploy it and whatnot and kind of breezed over it about 10 minutes, but, as you mentioned, I mean, there’s a lot of ground there that’s covered. And that’s just, like if you talk about the surface area, of what ECS touches. I mean, that covers maybe 30% of the entire surface area. Right?
So they’re like, and in a way, rightly so. Right? Because I mean, we’ve talked about how containers are really important now to a modern cloud native software development and running these, your applications, your code in production. And you have to have the reliability, the scalability, everything we talked about when we went through the well architected framework, right? I mean, that’s all required of it. And so there’s all these other pieces that are around that core of ECS to support that. And again, lots of innovation happened here, lots of updates, again, just to name a few things that have happened over the past 18 months so Fargate, which is the server lists way of dealing with ECS containers.
There’s tasked networking, which is a pretty cool advanced networking technology from a AWS that we’re going to get into. We’ll touch on today. But we’ll we’ll dive deep into it in a later episode, I think because it’s just so meaty, and it’s so interesting. There’s things like Container Insights, which is like just really robust system of metrics to give you some great observe ability, what’s going on in your containers,

Jon: Love that telemetry.

Chris: Love that telemetry, and service discovery. And its integration with cloud map is another big, big feature, that it’s almost like again, it’s just keep rolling this stuff out. And so just service discovery in the past, typically has been very, very difficult for microservices. And now with ECS It’s just, you just have to check a toggle box for the most part, and it just, it’s all set up and works for you with registration, and de-registration, with health checks. And all you have to do is just decide that you want to use those name spaces in your applications as opposed to going through a load balance or whatnot.

Jon: It’s like the matrix, it’s like in AWS, they have this infinitely tall building full of little pods. And each pod has a developer in it, a faceless developer, just deploying every five minutes, a new feature to AWS just infinitely.

Chris: So I mean, I’ve been looking at this stuff pretty, pretty deep. And that’s the one thing that I keep coming back to is just the appreciation for how much software they’ve built. When you started thinking about like, all these, even just like even the little features, sometimes you start thinking about, well, how would you implement that? What’s behind it? And you just start to really get this just really immense appreciation for, “There is a lot of software behind this stuff.” And I mean, this is millions of lines of code.

Jon: Yeah.

Chris: This is why Amazon is hiring, like mad and why the other cloud providers are like mad as well. I mean, it’s just, these are very complicated, sophisticated systems that there’s just so much work behind them. So, and we get to benefit, right? Because it’s, again, it’s the undifferentiated, heavy lifting that we don’t have to do, and it allows us to focus on our value add and that core competence that we’re bringing to the table, whatever business we may be in.

Jon: Right. Alright. So let’s continue. I guess we’re going to go into running container or… Yeah, we’re going to talk about running containers now. Yeah?

Chris: Yeah. So I mean, we talked about there’s so there’s all this new stuff that that’s been launched with ECS again, in the past year and a half. There’s areas that we can go much deeper into that are kind of like, they’re the less well understood areas, and it’s beneficial to really understand what those means instead of just kind of glossing over.
So things like, you know, networking mode. Things like placement, like how does placement work with placement constraints, and placement strategies? And how does the scheduler work and whatnot? So, we’ve got to pick something to talk about and keep it relatively confined. So today, let’s talk about Fargate.

Jon: Okay.

Chris: We can focus on [inaudible 00:24:23]. And I think it’s going to provide a good base for us to start branching out into some of those other areas that we’ve mentioned, as well. Well, I guess maybe before we talk about Fargate, specifically, and we’ve kind of already touched on this, so we’re using ECS to run our containers. There’s really three different ways to run a container, right? You can run it as a long running task, which essentially, that’s the ECS service that we talked about. And that involves the service scheduler, usually you’re integrating that in with a load balancer as well to front the traffic.

Jon: Usually Yeah, like web traffic or something like that.

Chris: Right.

Jon: Yeah.

Chris: Exactly. You can also use ECS to run a single task, one off. And then you can also use ECS to have a scheduled task that runs periodically. And so today and for the rest, we’re going to be focusing again on that, that typical use, the more typical use case of just an ECS service.

Jon: Long running [inaudible 00:25:20], just quick example, single task might be what a migration? And a scheduled task might be like an ETL?

Chris: Sure, yep.

Jon: Cool.

Chris: Yeah. So we’re going to focus on services, these long running tasks. And so, one of the first things that we’re gonna have to do when we create one of these tasks, is decide what type. What’s our launch type? And so there’s two launch types supported by ECS. One is EC2, and again, this is what we’ve talked about in the past, where you’re running your containers on a cluster of EC2 machines that you manage. Or the other new launch, the newer launch type is Fargate, and Fargate basically is, you’re now going to be running your containers on machines that are managed for you by AWS.
So this is essentially the serverless version of ECS. And so, we’re going to dive into Fargate, and just talk about what that is, and how it works, and benefits. And we’ll also talk a little bit about just pricing, because this is one of the things that was a bit off putting when it first came out. And then we can also talk a little bit about how do you actually migrate a container that’s running on an EC2 launch type, to the Fargate lunch type, and what’s involved there?

Jon: You know, I don’t really think about AWS names that often but Fargate is cool. Is it the coolest name of anything in AWS? Might be, because it sounds like Star Gate, which is kind of cool.

Chris: Mm-hmm (affirmative).

Jon: I don’t know. It’s a cool one.

Chris: Yeah, it’s I mean, I think-

Jon: Unusual.

Chris: It is. There’s definitely the typical formula is to have an acronym. And we have lots and lots of acronyms out there with this stuff. So, that’s just one of the differences right there. Right? It’s not an acronym.

Jon: Yeah.

Chris: And so that kind of makes it stand out alone. But yeah, I mean, it definitely, definitely, one of the ones that is a cooler name, Athena. I think that’s a that’s a cool one as well.

Jon: Yeah, Cloud Nine too, but that was purchased. That was not AWS choice.

Chris: Yeah.

Jon: Yeah.

Chris: Yeah. All right. So yeah, so Fargate. It was announced at Reinvent in 2017. I remember being there at Reinvent, and this was one of the 18 major product announcements they had, that they rolled out. They’re like, “Oh, yeah, by the way, Fargate. You can now run your containers, and serverless.” And then, could only spend like 30 seconds on that, and then it’s, “Oh, EKS, and-”

Jon: Right.

Chris: Yeah, it was just kind of crazy, but it’s been generally available since 2018. And so what is it? So again, this is the serverless version of ECS, where it’s allowing you to run your containers without you having to manage the servers or clusters of EC2s yourselves. So you don’t need to choose server types. You don’t need to decide when to scale your clusters. You don’t have to think about how to optimize cluster packing.
This is something we might be talking about more in the future, is the concept of bin packing, where you’re trying to get the best utilization of resources off your machines by finding spots of… it’s kind of like you’re just trying to pack as densely as you can on a particular machine instead of like, so if you can use two machines instead of three by intelligently packing your tasks onto those machines, then you’re going to be better off right? You’re going to get your costs by 50% right? So you don’t have to you don’t have to worry about that stuff with Fargate right? Because it’s no longer your problem. That’s that’s AWS’s problem.

Jon: I guess a quick question. One of the things you mentioned is that you don’t have to worry about the instance type that it chooses. But it’s going to use some flavor of AWS Linux, I assume? So that if you need to install something like FFmpeg, or Image Magic, or whatever, you can pre-compile that into your Docker image and know that it’s going to run?

Chris: Whatever you need has to be inside your image, your container image, inside your Docker file, right? So they don’t officially even tell you necessarily what operating system they’re running. It’s probably Amazon Linux One, I believe.

Jon: Uh huh.

Chris: But they do identify, it’s its own platform. So you can think of it as like Fargate OS. And you get to choose which platform version you want to be running on and they, as they make improvements, they’ll say oh, now in version 1.3 of Fargate, you have this capability, or now integrates in with Secrets Manager, or whatnot. There’s no guarantees or assumption necessarily on what the specific operating system is.

Jon: Maybe I’m missing something from our previous series. I’m sorry to sidetrack us, but I just mentioned two packages of Ffmpeg, and Image Magic that need to be compiled. I guess you could wait and compile them as part of spinning up your container. But that would be awfully slow.
You’d probably be better off in most cases with things like that, to actually pre-compile them into your image. But those two packages I mentioned do depend on the operating system that they run on. So you know, you can’t necessarily compile them for Red Hat and then have them run on Ubuntu, right? Like you got to compile them on the right thing. So, right? So I guess what I was just saying is, can you trust and do that? Can you put a pre compiled piece of binary in your image and know that it’s going to work?

Chris: Well, I mean, the one thing that you know for sure is it is Linux, right? So you have the Linux kernel. But after that all bets are off, right? And so, I kind of touched on this a little bit with containers, and when we think about we’re running different operating systems and containers, when we say like Red Hat versus CentOS, versus Fedora or whatnot. I mean, all of those share the common Linux kernel. That’s there, that’s the same core piece of code that’s there. And so when you compile something, and you’re relying on things like sys calls and whatnot, and the core libraries for the Linux kernel, that’s going to be there, right? because that’s, it actually gets a little bit more complicated with Fargate, because Fargate is actually, your containers are running by themselves inside a VM. Which is a little bit different. As well, although, for the most part, everything still applies with what we’re talking about, of how the container actually is using, it’s sharing a kernel. But in the Fargate world, it’s not sharing it with anyone else.

Jon: Okay, got it.

Chris: It has its own. It has its own, right? So, I mean you should be fine compiling that, because when you compile it, you’re gonna be doing it for a particular architecture, right? You’re going to be compiling it against like Linux and arm. So that will be something that needs to take into account, what’s the processor architecture? So I don’t know the specifics there, if there’s a guarantee like it’s always going to be on Intel or AMD or, or whatever, right?

Jon: Right.

Chris: You don’t get control over that with Fargate, right? So with Fargate you basically, the way that you specify what hardware you’re running on is by just saying, “This is how many virtual CPU units I want and this is how much memory I want.”

Jon: Okay.

Chris: And then it’s up to, it’s up to Fargate to figure out what you actually get put on.

Jon: Okay. That’s really interesting. I’m guessing there may be some limitations. There might be certain packages that are particularly sensitive to what they run on, that you might have a hard time getting running on Fargate.

Chris: Yeah.

Jon: Yeah.

Chris: Yeah, absolutely. Kind of similar to with Lambda, right?

Jon: Cool.

Chris: Cool. So, and another thing with Fargate, to keep in mind is you do have complete control over your task placement, to be within your own VPC. And we’ll get into this a little bit more, but at the end of the day, your Fargate containers are going to be running inside the VPC that you choose, and so it’s literally running inside your VPC, but all that underlying infrastructure, that’s gonna be managed by Fargate, not by you.

Jon: Okay.

Chris: So kind of interesting. So just again, something a lot of times these services, they’re running outside of a VPC. And so we have, there’s considerations there. Fargate itself is going to be running inside our VPC.

Jon: Okay.
Sorry for interrupting. I just wanted to ask you a favor, would you mind contributing to our boat fund? It’s that we’ve always wanted a boat, and we need money from you in order to get one. But seriously, it takes quite a bit of time and effort to produce Mobycast every week, and we do have some small sponsors, but nothing really speaks to us like getting support from our actual listeners. So if you wouldn’t mind, go to glow.fm/Mobycast real quick, and it only takes 30 seconds, and just contribute. You can subscribe for $5 a month or $10 a month or whatever floats your boat, or hopefully soon to be our boat. Anyway, if you give us five bucks a month, we’ll send you a sticker. And if you give us 10 bucks a month, we’ll send you a tee shirt. So, looking forward to seeing those subscriptions cruise on in. And now back to the show.

Chris: So benefits, you know, what are the… Why would you only use Fargate?

Jon: Why wouldn’t you? [crosstalk 00:35:29]

Chris: So there’s good reasons for using it. And then there’s reasons why you really, maybe you shouldn’t use it.

Jon: Sure.

Chris: And so, so some, you have no clusters to manage, right? So that that overhead is gone. And we’ve touched on this in the past as well though, it’s like well, is how much of the features that really, how much time do you actually spend dealing with your clusters?

Jon: If you spend as much time keeping on top of AWS as Chris, then you don’t spend very much time updating those clusters. You’re like, it’s just like muscle memory.

Chris: Sure. Yeah. I mean, it’s literally like, it’s an hour every quarter probably spent on cluster maintenance, if that. But for some of you, I mean for some people, for mere mortals, this may be a benefit.

Jon: Yes, definitely. Yes.

Chris: Yeah. So it’s, seamless scaling is another benefit, so again, you don’t have to manage these clusters and wait for EC2s spin up or whatnot. That’s now Fargate’s problem, and you can let them deal with that.

Jon: And they’re super optimized to be able to do that. Maybe faster than like, if you were to if you were to bang your head as hard as possible against scaling up and down EC2s as fast as you could, you probably couldn’t do it faster than Fargate.

Chris: No, no. Yes, because they’re not actually spinning up EC2s. So, but we’ll get into that.

Jon: Secret of the pros.

Chris: Yeah. And then you know, another real, a really big strong benefit of Fargate is that you only pay for you when you’re actually running task. And this is definitely one of the big benefits of serverless in general, right? Is you’re not paying for idle resources.

Jon: So Chris, the system is down, and your response is, “Hey, but it’s free right now.”

Chris: Exactly. Look at the bright side.

Jon: Yes.

Chris: No charge. Yes. So I mean this is perfect for things like batch jobs, cron jobs, other on and off workloads. Where you don’t need it 24 seven. Like if you have work that really doesn’t have that kind of, “Hey, it needs to be up 24 seven.” It just runs sporadically. Then something like Fargate makes so much sense, right? As opposed to maintaining a cluster of EC2a that are running 24, seven.

Jon: Right.

Chris: And so you’re paying for them for each hour that they’re, for each minute or each second that they’re up, you’re paying for that. But with Fargate, you’re only paying when your tasks are running. So that’s definitely, can depending on your workload, that can be a really huge benefit.

Jon: Cool.

Chris: So let’s get into some of the specifics about Fargate and just how it works. One thing to keep in mind is that each Fargate task, it has its own isolation boundary. And so we talked about this a little bit before, but it is not sharing the underlying kernel CPU resources, and memory resources, or networking with other containers. So it has its own, and so this is, this is done, basically, essentially, what’s happening is that your container is being spun up inside of its own VM.

Jon: Right, right. Yeah, and this is definitely different than using EC2. When you’re using an EC2 to cluster, you definitely have multiple containers in the same VM. Yeah.

Chris: Yeah. I mean, especially if you’re running more than one container.

Jon: Yeah.

Chris: You absolutely are. Right? So each one of those instances in your cluster is a VM. And then you’re typically running more than one container on each one of those, those EC2 instances.

Jon: This could cut down on the types of errors, one of the types of errors that we talked about in a previous episode of using up space on an instance, and being surprised about it because it was some, it’s like the EC2 agent or something that was running, running up disk space and you weren’t paying attention. Fargate may not suffer from that same type of issue.

Chris: It doesn’t because you basically don’t. That’s something that kind of goes away with Fargate.

Jon: Yeah, yeah, exactly.

Chris: Right? Is disk. So you do have access to, there is some access to volumes and disks but nothing that’s shared.

Jon: Right.

Chris: On the host itself, right? Because that’s something you just don’t have access to with Fargate.

Jon: Right.

Chris: So it would really be only, so the very, very limited cases, right? And so definitely, this gets into some special considerations for what it means to run your workload in something like Fargate. And this is one of those key differences that you have to think about, especially like logs, right? If you’re writing your logs to files, well, that’s not going to work so well. And Fargate-

Jon: Point for target, though, on that.

Chris: Yeah, I mean, otherwise, it would be, you know, be kind of, [inaudible 00:40:46] think that all those headaches would be on them, right? Of managing things like, we could just write into disk space, and so each one of these containers is going to be running inside its own VM. Turns out, it’s using the Firecracker technology from Amazon. So Firecracker is Amazon’s lightweight virtual machine technology. So it is a virtual machine. But it’s this new category called micro VMs, where they’re extremely lightweight.
They can launch very, very quickly, like in the, I believe it’s in the sub second timeframe. And they can run thousands of these things on a single bare metal server. So very optimized. We, again, in the future we may go in to, have a whole episode just to go into this whole, because this is an emerging trend. This is like the-

Jon: Yeah, we were just asked about this on LinkedIn today, Chris. Somebody was asking how are we going to, did we go into micro VMs on our containers versus VMs series? And I was like, no, sorry, we didn’t have time to get into it in just four episodes. And here it is coming up again today. We will get into it. But we know we only have about 10 minutes left. We can’t do it today. So we’ll just leave it at a lightweight VM.

Chris: Yeah. And that also actually, because I think Fargate didn’t start off necessarily with Firecracker. They introduced this in early 2019. So earlier, back in January of this year, and it led to much better efficiency. So just that alone, that allowed them to have an approximately 50% price cut on the Fargate service.

Jon: Ah, that is a price cut.

Chris: It was huge.

Jon: Yeah.

Chris: Yeah. Which was huge, right? So that’s, you can expect that trend, I think, to continue because out of the gate Fargate was pretty interesting. Everyone was like, “Wow, this is really…” You know, serverless. It’s, you know, the buzzwords are there. It’s like, “Oh, I can run my containers and serverless, like this sounds great.” But then if you look at the pricing, it’s like, “Wow, it’s, this is quite a bit more expensive than if I just run it on my own EC2s.”
So, price cut definitely helps in that direction. It’s still more expensive. And we’ll talk about maybe that a little bit and go through an example, for pricing, but something to keep in mind. Something, so another aspect of Fargate is that your tasks, they must be launched into a cluster, right? So you’re going to need to define a cluster inside ECS, and that cluster is going to be basically the logical infrastructure and permissions boundary for isolating your groups of tasks.

Jon: They should have switched the word from cluster to namespace, because that’s essentially what it becomes then, right? It’s a namespace, more or less.

Chris: I mean, that’s, for me, yeah, I mean, this is you know, when they first launched it, cluster made a lot of sense because it was backed by a cluster of EC2 instances. And then when they roll out Fargate, it’s like, “Okay, how does that fit into it?” And so this is the way they did it.

Jon: Yeah.

Chris: Yeah. The interesting thing though, is that clusters, you can, they support running both EC2 and Fargate launch types. So you can mix and match and a particular cluster. Which again, kind of goes to the whole like, this is a namespace, right?

Jon: Sure. Yeah.

Chris: Yeah. So you do have that choice. And maybe just quickly talking about the types of choices that you have with clusters. So there’s basically three types of ECS cluster that you can choose. One is going to be the one that we typically use, which is the EC2 Linux plus networking. And so really, what this is, is this, it’s the networking support that you need. So VPC sub nets, then also is going to be creating an auto scaling group with the ECS optimized Linux AMI for that so, you have your cluster of EC2 instances, it’s running, and it’s in, and you’ve specified the networking. So that’s one cluster type.
And other one would be analogous to that, but for the Windows world, so it’s using a Windows AMI, for that auto scaling group. And then the third one is just a networking only cluster. So you can actually create a cluster to you don’t have to define any, there’s not going to be an auto scaling group associated with it whatsoever. It’s just the VPC and the sub nets, that’s just the networking only. And that would be a cluster that obviously, you could only run the Fargate launch type in.

Jon: Oh, okay.

Chris: You wouldn’t be able to run the EC2 launch type, right? So those are gonna be the three choices when you create your cluster. So you can run those Fargate tasks in either one of those, those types either inside a cluster that’s backed by EC2 instances, or one that’s networking only.

Jon: Okay.

Chris: Another Biggie with Fargate is that the tasks require AWS VPC network mode. And so this is, it’s also called task and networking. But this is, so when you have a task, a container running in ECS, you’re going to specify what networking mode it should use. And so the default choice is going to be bridge. And that’s usually I think, what most people choose, just because it works. They don’t really need to, you don’t need to know too much about it. And it’s the default, things are just going to work the way that you expect.
There’s a couple other choices though the ECS offers, and one of them is this AWS VPC network mode. Fargate requires this, and so, the reason why, is because AWS VPC network mode, what that does is it attaches a network interface to the task itself.

Jon: Okay.

Chris: And this is the mechanism by which Fargate is wiring up your task into your VPC. Right? Is through this ENI. So you’re going to be, so each of your tasks that are running in Fargate are going to have their own ENI, so Elastic Network Interface.

Jon: Yeah.

Chris: And so with that, you’re going to have to specify one or more sub nets. And then also one or more security groups, there’s a lot of benefits that go with this networking mode. You’re going to get, and so the much better performance, essentially is one of the big benefits, and then you’re also getting much more granular level of access. So there’s a lot of great security benefits to this. And then you also have a lot of more visibility into just audit capabilities as well.
So you can have things like cloud trail logs associated to the ENI, and get really clearly fine level access to it. So a lot of really great things about this, you can use this same networking mode with the EC2 launch type. But it gets to be a little bit trickier because there’s there’s considerations with how many ENIs you can actually attach to a single EC2.

Jon: Yeah, let me see if I can say what I think is the case, as simply as I can. And then you can tell me if I’m just way off base. It would seem to me that with this AWS VPC networking mode, that every single task starts to look like its own computer on your little network, your little private network that you’re running. And so it’s easy to see what everything is doing and talk to everything, every single tasks is its own computer basically.
And using that in EC2 launch type, you know, same benefit. But then if you use bridge networking, well, everything does have a network address. I guess the thing, the tasks behind the EC2s, so if you have several containers running, they’re essentially, they look like they’re in their own private subnet that you can’t really address or see through. Like you can only talk to the EC2, and the tasks behind that are kind of dependent on that one network interface. So you have less visibility to the individual tasks. Is that close?

Chris: Yeah, I mean, anything you can, maybe another way to think about is just in that bridge mode example, you can think of it as, all your networking is being proxied. And so something is doing the multiplexing, right? Because at the end of it, it’s just one, it’s basically a single network interface.

Jon: A network interface, yeah.

Chris: For however many containers you have running and all of their traffic has to go over through that one interface. Right? And so something has to kind of manage that. And there’s going to be some, there’s performance implications to that. Right?

Jon: Mm-hmm (affirmative).

Chris: Versus, and also some just design considerations, right? There’s no way to, I mean, how do you actually connect to one of those things behind the multiplexer? You can’t do it just based upon the IP address of the network interface card right? You have to use that plus some other additional information like port.

Jon: Right.

Chris: And this gets into like, you know, why we have issues with port naming, and things like-

Jon: Collisions and stuff.

Chris: Yeah, and why we have features like dynamic ports and whatnot, right? So versus with TAS networking, with AWS VPC networking mode, you can think of it as, you no longer have to share that network interface, right? It’s just each one of those containers is essentially getting its own network interface card. They don’t need to share with anyone else.
It’s going to have, they each have their own IP address. So it becomes much easier to communicate to those things, you don’t have to worry so much more about ports either, right? Because your ports now only have to be unique per task. They don’t have to be unique on the entire instance on which they’re running. Right?
So, and remember a task, the way that AWS ECS defines a task is a collection of one or more containers.

Jon: Hmm. Mm-hmm (affirmative).

Chris: All right, so we can actually run multiple containers inside a single task. And so if we do that, then each one of those, they will all be sharing that same ENI, right? If we have more than one container defined in our task, but typically we usually have, it’s a one to one mapping, where a task has a single container in it.

Jon: Interesting. So Chris, we’re kind of running up against the time that we have, and we have, I think more than we can get through. And just I want to think about our listeners and what will be the best way to end this because I know from for example, I know that we want to get more into what we were just talking about in a future episode. More into like how this ECS networking works and how container networking in general works.
So we, that little side trip, is like going to blow out into its own whole thing. But yeah, is there anything else you want to try to squeeze in one or two minutes? Or should we kind of round it out? And maybe give everybody a preview of what’s coming in the next episode?

Chris: Let’s round it out with a few more details and then quickly run through some pricing information because it’s kind of interesting. And then after that we can, we’ll wrap up and, and then maybe save the discussion of, “Okay, how do you actually, what’s the practical steps of migrating a container from an EC2 launch type to Fargate launch type?”

Jon: Okay.

Chris: Yeah. So with that, some of the things with Fargate task is you’re going to be specifying the CPU and memory at the task level, not at the container level. So again, this is something we typically do it at the container level. Now we’re going to pop up again to that task level, because that’s how Fargate is looking at things, right? It’s dealing with things at the task level, not at the container level.
Whatever we specify as a CPU and memory, considerations for that at the task level, that’s how we’re going to be charged as well. That’s what’s going to determine how much it costs. AWS is going to charge us for running our containers. Logging is something to really think about here as well. So before I get supports to log drivers, one is AWS Logs. And the other one is Splunk, which is rather new. So it’s it supports one third party vendor in Splunk. Otherwise, AWS Logs, which basically is sending your logs to Cloud Watch Logs.

Jon: Okay.

Chris: If we want to integrate with other log services, we’ve got some work to do. And so that’s something definitely to consider.

Jon: With Cloud Watch Logs, can we use that like, isn’t there a Cloud Watch thing that lets us basically grab standard out and then fork it off to some other thing?

Chris: You will that’s how that’s how Cloud Watch Logs work right? It’s capturing stuff that’s been sent to standard out standard air, and then that, from your container and then that goes over to Cloud Watch. So now that your Cloud Watch logs and then now if you need to get your Cloud Watch log somewhere else, you have to have some other solution for how do they get from there to somewhere else, whether it be Logly or Sumo Logic or-

Jon: Right, right.

Chris: Data [inaudible 00:54:30], or whatever it may be, right?

Jon: Cool.

Chris: And then there’s going to be also some IM role considerations that you’re gonna have to do with Fargate. So it’s going to need, because it’s doing, it needs to do a bunch of actions on your behalf. Things like it has to attach an ENI to an instance right? So it needs the permissions. So it’s going to need some IM roles and permissions and, you’re also going to have to take advantage and start using a task execution role.
And the ECS task execution role, which we typically haven’t had a need to use when we’re running on the EC2 launch type. But with Fargate we will, if we’re going to be pushing our logs to Cloud Watch, or if we’re going to be pulling images from [inaudible 00:55:19], since both those are our credentialed, AWS services, it’s going to need the credentials to do that right? And so, the way you do that is by specifying an IM role that is the task execution role that it can use.
So those are some of the specifics of Fargate that make it different from the EC2 launch type. And that will dictate what changes you would need to make, or take into account, or what kind of workloads that you can use. From a pricing standpoint, like we talked about, pricing is going to be based upon the amount of CPU and memory that you use.
You’re charged by the second and with a minimum charge of one minute. So very, very granular. You know, just like with everything else pricing is based on, a per region and whatnot. But, I went through and did a quick calculation of like I have my own personal blog server. It’s currently running on a T2 micro EC2, using ECS running in the US West two region and so, T2 micro has one VCPU and has a gig of memory, and that ends up costing me about eight dollars and 35 cents a month to run.
On Fargate, if I go for the smallest CPU and memory choices, which is a quarter VCPU and a half a gig of memory, that ends up costing about $8.89 cents a month for that. So running my personal blog on Fargate, as is, it’s going to be a little bit more expensive per month. But also keep in mind that that’s on my, the T2 micro that I’m paying for, I could I’m getting basically at least twice as much capacity. So I can run two websites for the same price, right?
So, in that way you can think of this, is like Fargate’s probably about twice as much as straight EC2. And then if I were to go with, convert my side over to like a T3 nano, which is two VCPUs, but only half gig of memory, kind of getting closer to like, what the Fargate is from memory standpoint, although it’s still much greater CPU power, that ends up costing me $3.74 cents a month. So now it’s even more extreme.
So again, the net net is that for 24 by seven task, Fargate’s probably going to be quite a bit more expensive than running your own EC2s, and that’s why, you know, but if you have a workload that’s not running constantly, that’s not running 24 by seven, where you do have idle CPU that’s just sitting there, then that’s, those are the kind of jobs that makes sense to run on Fargate.

Jon: Right, and then also, what’s the cost plus opportunity cost of having somebody that’s able to quickly go and maintain an EC2 cluster? Like make sure that they’ve got the latest MI and update the lunch configuration with that and all that other stuff that you might have to do? Some people can do that in half an hour and some people that might take them all day.

Chris: Yeah.

Jon: There’s an opportunity cost for that.

Chris: Yeah, absolutely. I mean, like, in this case, it’s like, “Hey, it’s like $8.89 a month for Fargate, versus $8.35 a month for EC2, but I had to go create the cluster and figure all that stuff out. I mean, for most people it’s going to be a no brainer, it’s like 60 cents more a month.

Jon: Right.

Chris: You know, and I don’t have to deal with that? And it saves me, even if it just saves me eight hours of work, you may say, “That’s a no brainer.”

Jon: Yeah, that’s a year’s worth of runtime.

Chris: Yeah.

Jon: I guess I did have another question. You were saying that that things are, you size things at the task level, and I just want to make sure that I understand that. So even if you size up pretty big so, let’s say you had an EC2 backed UCS cluster, and you wanted to, you knew that it was going to take like 10 machines, right? You need 10 mid sized machines in order, and all the traffic. You would specify that, you might specify something that that would tell you the number of EC2 instances, you would specify that I guess in your auto scale group. What about Fargate? Do you just say a number of the VCPUs that kind of equals 10 for your task? Or how does that work?

Chris: So, you’re telling it in your task definition what your requirements are. “This is how much CPU I need. And this is how much memory I need.”

Jon: So you have to account for the full scale, like the full scaling of your task and that, like fully loaded scale, not just, “This is how much VCPU it takes per.” What I think of as a computer, but, “How much I completely need for my task or service.” All the way, like, full load. So if I have a million users, I gotta tell you how many VCPUs that’s going to be?

Chris: No, so remember, you’ll be telling. So, when you run, Fargate, and you have it as an ECS service, one of the things within ECS service, you’re going to tell it is like how many tasks to run, right? So you’re going to say, “Oh, this is my minimum, this is my maximum.”

Jon: Oh, okay.

Chris: Right? You can also tie in auto-scaling to both, regardless of whether using EC2 launch type or Fargate launch type. So, and again, we’ll get into this much deeper and-

Jon: Okay, I think you just [crosstalk 01:01:02] helped fix my problem. So it really is just to run your little process, your program, how much the VCPU does that take? Not like fully loaded, in production?

Chris: Yes.

Jon: Got it. Okay, cool.

Chris: Yeah, yeah. Yeah.

Jon: I guess the place I got confused is when you threw in that task is not equivalent to container, whereas I always sort of thought of them as equivalent, like that sort of threw me off.

Chris: Yeah. And again, I mean, for most people, it’s going to be equivalent, because again, your task definitions only going to have a single container. It’s just, if you’ve ever looked at in detail at a task definition file, you may have noticed that there’s two memory, and if you have a single container that you’ll see that there’s two memory parameters, and there’s two CPU parameters, ones in the inner array for the container itself, and then ones on the outer, and that’s for the task and so it’s just for Fargate, it doesn’t care about the inner on the [inaudible 01:01:57]. I mean, the inner on the container, it will look at, but it’s required to have it on the outer container for the task, right? Because that’s how it’s doing its scheduling. Right? It’s doing at the task level.

Jon: Yeah, I think we’ll have to revisit that. And it just goes to show whenever there’s something that’s a little, “Hmm, that doesn’t suit my mental model of how this all works.” It’s usually because oh, yeah, because it’s way more complicated.

Chris: Yes.

Jon: So once again, that’s way more complicated than the mental model was. All right. Well, let’s wrap it up.

Chris: I think so. Yes. So you know, we can say for next time, kind of like walking through what, how you actually use Fargate in practice and what are some of the steps involved. I actually as an exercise took my personal blog, which has been running on an EC2 launch type, and converted it to run on Fargate.

Jon: Cool.

Chris: And it was kind of a fun process and took a few hours. And so, just kind of run through that process. Like just what does that entail? Like what do you have to do? What are some of the gotchas.

Jon: Awesome, can’t wait.

Chris: Yeah.

Jon: So much for this week.

Chris: All right. Thanks, Jon. See you later.

Jon: Bye.

Voiceover: Nobody listens to podcast outros. Why are you still here? Oh, that’s right. It’s the outro song. Come talk to us at Moby cast.fm or on Reddit at r/Mobycast.