45. Serverless Made More Modular – Lambda Layers and Runtime API

Rich: In episode 45 of Mobycast, we discuss Lambda Layers, the Runtime API, and how to make serverless more modular. Welcome to Mobycast, a weekly conversation about containerization, Docker, and modern software deployment. Let’s jump right in.

Jon: Hey, welcome Chris and Rich. It’s another episode of Mobycast.

Chris: Hey.

Rich: Hey guys. Good to be back.

Jon: Yeah. Great to have you back. Rich, what have you been up to this week?

Rich: Oh, feels a little bit like the quiet before the storm. I have a lot of travel coming up in the next couple of weeks and just the way that everything found itself into my calendar, all of the stuff that needs to happen is happening while I’m gone. I’m trying to get things done but when you’re on that space where it’s like you wish you could do something but you can’t because you’re sort of blocked and you know that by the time that everything’s going to be ready for you, you’re going to be blocked again by travel. Trying to get stuff done but I feel really challenged.

Jon: Are you at least going somewhere cool? Are you going to the Isle of Naiaz or something?

Rich: I don’t know what that is and probably not that cool but I’m going to Mount Baker to shred. It will be my first two days on the mountain, too, with some buddies that I knew from back in the days at Vale. That’s going to be great, then a couple of days later I go to a conference in Philly and a couple of days later I have a Mastermind conference in Beaver Creek. A lot of stuff happening, and then of course a lot of deadlines right around there, too.

Jon: Of course. What about you, Chris? What are you up to?

Chris: I’m just trying to get back into so many things after all the holidays and what not. There’s a fresh year, lots of new things to look at, and kind of doing some goal setting or what not, and then it’s also when everyone else gets back. The plate’s pretty full this past week and dealing with a lot, busy with just day-to-day getting back into so many things.

Jon: Right. Kelsus in general tends to have busy Januaries because we have clients that like to show-off stuff at big trade shows and conferences. Just a shameless self-plug, we just built something for Spiro. That’s the app for their Spectrum’s new offering that’s kind of making a splash at CES. I just couldn’t be more prouder at the team for putting that together and seeing TechCrunch, Engadget, and CNET are covering it and talking about as one of the most interesting things they’re seeing at CES, is super gratifying. I love it.

Today we are going to talk about serverless made more modular. Chris names these and I love how he cited more modular. He didn’t just come out and say, “Serverless made modular.” It’s, “More modular.”

We’re going to talk about Lambda Layers and the Runtime API. Basically this is kind of digging back into the pool of content that we picked up from re:Invent. One of the things we love to do on this is just to really talk through some of the more complex or interesting sessions that we go through at conferences and bring them to you in a way that’s suggestible, in a way that gives us a chance to Mystery Science Theater 3000 a little bit. Let’s start and give us a little bit of an introduction, tell us about how you came to this particular session and what’s it all about, Chris?

Chris: Definitely at this past re:Invent, it was loud and clear that serverless is really important, obviously, and it’s something that AWS continues to make big investments on. We’ve talked about serverless in the past on Mobycast and have done some episodes on it. For me, I always like to come back to what does serverless mean. I think a lot of people, serverless means just Lambda and it’s really a larger term than that. If you’re not managing the actual infrastructure, they don’t call that serverless. Things like DynamoDB fall into this space, Aurora Serverless, even things like SQS can be considered to be serverless.

It was really interesting at re:Invent just to see AWS already has this huge catalog in pseudo services for allowing folks to do serverless but yet they continue to innovate in that space and they still have a lot of work to do. On the Thursday keynote session with Werner Vogels, that’s what they did a live announcements around serverless. It turns out a lot of it was around Lambda this year.

They’ve made quite a few improvements there, ranging from announcing Firecracker which is their new micro VM technology for spinning up essentially containers or small VMs, whatever you want to call it, but just very, very quickly and being able to handle a lot of these things. They use that now as the core platform for Lambda, for spinning up Lambda functions.

There were other improvements announced related to capabilities with Lambda like increasing timeouts from 5 minutes to 15 minutes and some other things that people had some issues in the past. It’s getting more mature. Two of the more interesting ones that they announced were Lambda Layers and the Lambda Runtime API. These are kind of two new foundational pieces to it that really open up the programming model and make it much more flexible. They announced that at Werner’s keynote on Thursday.

Whenever they do make announcements for new products or services, obviously they want to keep those hidden and keep them secret until the announcement’s made. They may have sessions planned for it but they are not going to put those sessions into the catalog until after they’ve been announced.

Me, personally, I’m looking at my Friday calendar. There’s not a lot of content on Friday’s calendar. It’s the last day of the conference and they’ve kind of winding down. They’ve gone from five venues to two venues and just a reduced number of sessions. I wasn’t able to reserve anything, so I was kind of wondering what am I going to do?

Jon: Vegas.

Chris: After five days of Vegas… it’s more like, “Ah, it is Vegas.” But I was very, very happy after Werner’s keynote, checked the app for updates to the catalog and of course, there’s new sessions added based on the announcements. One of them was a session specifically on Layers and the Runtime API. I immediately snagged that, reserved that, and was super happy, also reserved one for the cloud map service that was announced as well. I’ve got two really cool sessions on Friday.

It’s the role we’ll talk about today, just kind of breaking down what was discussed during that session about the new Layers feature and the Runtime API for Lambda. Before we talk about that, another anecdote to that was again, given that Friday was the last day of conference, they went from five venues to two. There’s probably a fourth of the content available now, maybe a fifth, so a lot less sessions available on Friday. I think the expectation is that there’s going to be less people, people are going home early, there’s also the big party the night before Thursday night replay, people partying late so they’re probably not going to want to get up and go to a session.

It turned out that that didn’t happen at all. I was really surprised to see bright and early, 8:30, 9:00, people were in the waitlist lines and it was jammed with people trying to get into these sessions. I was really lucky to have gotten reservations for both of these two sessions that I was but there are a lot of unhappy people grumbling especially for this Layers one because the Layers one was over at the Bellagio and it was actually held in one of the […] rooms and so it’s this tiny, tiny room, 70 chairs.

Jon: AWS Twitter has been non-stop talking about Lambda Layers since it was announced, so you would have thought that AWS would have had more sense, like this is going to be a big interesting session that everyone is going to want to know more about so I’m surprised, too.

Chris: Yeah, it’s no doubt. If you would put together a list of what are the most interesting sessions at re:Invent, this is going to make the top 25 list out of the thousand sessions, like very broad spectrum appeal, lots of people are going to be interested in this, like this could have accommodated easily a thousand people. Instead, it was this tiny room with 70 chairs.

I was breathing easy because I had my reservations, so I’m getting in, but the line of people waiting on the waitlist was easily 500 people, maybe more, and then you had a few people there that were kind of just screaming up and down the line saying, “Beware, warning. That room is only big enough for 70 people,” and you could tell if people in the waitlist are like, “Do I believe them or not? What do I do here?”

They were right. It was a very, very tiny room. It even slowed down the start of the session because the presenter wanted as many people as he can get. He saw all these folks that were interested in his talk and he wanted them in there, so he’s trying to negotiate with folks that were running the room there, like, “Hey, can we just put them against the wall or something?” and they’re like, “Nope, not going to do it.” It was interesting to see this back-and-forth negotiation for about five minutes before it’s like, “Okay enough. It’s not going to happen. If there’s not a chair, there’s not a person.” So, get going.

Jon: Yeah, you’re thinking about the power of those people.

Chris: Oh yeah. It was kind of interesting to see the negotiation back-and-forth but it wasn’t going to happen. Long story short, super fortunate that I got into it and it turned out it was a great session. It was jam-packed with content. The presenter—I will try to get his name pronounced, [Danaro Putcha], and he’s a principal evangelist in the serverless group there at AWS. He did a wonderful job, lots of good content but also gave a lot of really great practical tips. We’ll dive into that a little bit here and just break down that session.

Jon: Cool. This was specifically SRB375, in case people are trying to find it on the internet. Some of them are on there.

Chris: Yeah, and if you are interested and want to get the full session, the video recording of the session is on the AWS YouTube channel. They’ve also published the slides for this as well on the SlideShare. I’m sure we can camp links in the show notes here for both of us, they’re available.

Jon: Great, let’s get into it.

Chris: [Danaro] started off first just talking about two main pieces here, Lambda Layers and Runtime API. First went into Lambda Layers. You talk a little bit about what they are and why they’re useful. Most people who know Lambda, it’s a function implication platform. You basically have some code, it acts as a function, you upload it as a zip file to the Lambda service, and then you have something that triggers the execution of that so that when it gets executed, it’s now reading in your code that you’ve uploaded, that it’s executing that function entry point, which again it can call other functions and what not. But as just as you’ve packaged up your code as a zip file, you’ve uploaded to Lambda, it’s now invoking that on-demand, executing it, and then the results are returned back to it.

Some of the challenges that come with that are if you have a very simple function like in JavaScript with no dependencies, it’s pretty simple to go ahead and do that. But what if you want to run code that uses five other packages or it has these other dependencies that have to come along with it? It could be that instead of just one file, you need 500 files. They’re all been zipped up and sent up to Lambda. That’s the runtime support that you need for it.

It gets even more complicated if you actually have binary dependencies. If the code that you are running, maybe it has some packages that for various reasons, maybe performance or other reasons…

Jon: My favorite one is ImageMagick for this example.

Chris: Sure, ImageMagick is a great one. ImageMagick, C++ code, there’s going to be bindings there to go from your dynamic language to ImageMagick, so what are you calling it from JavaScript, Python, or whatnot, there’s going to be some compilation that needs to happen in order for those bindings to work. There’s some additional work you have to do as a developer to get that compilation done and then zipped up and then that becomes part of your zip package that you upload to Lambda.

Up until now, it’s been a very isolated process. We have to kind of do the exact same thing every time across all your functions and across all your teams are kind of having to do the same thing. There’s no way they’re really kind of share or to build on maybe these dependencies that you have.

That’s exactly what Layers allows you to do. It really is just a layer of files that you can stack on top of whatever runtime you’re using. You can have up to five of these layers, so however many layers makes sense to you. But you can kind of think of it as if it really is for sharing dependencies or if you’re doing some more complicated packaging, again doing with things like binaries or what not, you can do that once and store the results of that in essentially a layer. Then you just have to do it once and now you can have that as a reference. Whatever is now different about the particular function or the particular code that you’re doing, you can just reference that like a reusable component, if you will.

Jon: I just wanted to make that more explicit with an example. I don’t know for sure if this is a correct example but I’m going to try it and we’ll talk about it. Going back to my ImageMagick example, I can imagine you having a function that lets you upload an image and store it somewhere, but on the way to storing it, it’s got to do something with that image, maybe crop it or maybe downsize it or whatever. That’s something that ImageMagick can do and then it goes and stores it somewhere, S3 maybe.

Then, another function that goes and gets images custom-fit to a certain size, like maybe they all have to be square, and they’ve got to be 600×600 pixels or something. That function is going to grab the image out of S3, run an ImageMagick on it to crop it just right and then return it to the caller. I’m thinking that that would be an opportunity where you could create a layer that has some code in your language of choice, maybe it’s Node, maybe it’s Ruby, that talks to ImageMagick and uses it to do the job that needs to do. That code would be used across both of those functions for both the Create of the image and the Get of the image. If you could put that code into a layer, you wouldn’t have to put it into both of those packages. Is that a good example?

Chris: Yeah. In this particular case, having a layer just with ImageMagick—

Jon: Maybe a little bit of interface code between…

Chris: Sure, yeah. I mean, you may have a common library that kind of has an abstraction that’s built on top of ImageMagick. Invokes it, executes a certain API and gets back the results or what not. That whole thing might be a good candidate as a layer because it’s probably pretty complicated to put. There’s a build process associated with it. You have to go make sure that you’re building that code essentially on an A&M line that is bit the same that Lambda uses. You can’t compile your code on your Mac and then upload it to Lambda. Not going to work very well.

There’s work associated with that. If you do that as a layer, it’s now reusable component that you don’t have to worry about anymore. Now you can just focus on what’s different and now you have that one layer with this shared library that provides the interface into your ImageMagick functions. Now you don’t need to worry about it anymore. It’s going to be there for that function so the function can just make the call into that shared library. It doesn’t have to be part of its layer. It’s part of the previous layer that was put in there. That’s a great example for when this really shines.

Jon: I think it’s fair to say just talking that through. I think it’s fair to say that without some additional help, there’s more heavy lifting involved in doing that than there would be in, say making ImageMagick a dependency in maybe your express app and just having a whole app that provides both the create and the get features, you know, APIs.

So, even though we get layers now, and that layers help us not have to do all the dependencies for each Lambda function, it’s still technically less work to build an application than it is to build a whole system one function at a time inside Lambda. You see what I’m saying?

Chris: I think we’ve touched on this in the past, too, when we talked about serverless is that it’s definitely a different model and you have to structure your code differently. You have to think about it differently. Everything changes. CI and CD change the way that you test these things, the way that you build these things, the way that you package them, the way that you deploy them, the way that you invoke them. It is quite different, which makes it actually a lot more difficult to adopt something like this than even something like containers.

To go from VMs to containers, your whole programming model doesn’t have to change. Some of your process stuff is going to change but the mental shift there is something that you can probably assimilate within a few days and then refine it from there versus serverless. Kind of really makes you think about things quite a bit different. Even though it’s been out like Lambda itself was launched in 2014, I believe, it’s been around now for 4+ years. It’s not like this robust, mature platform. There’s still a lot of work to be done, we’re just now getting layers. This actually opens up a bunch of other holes that AWS has not yet addressed and they’ll need to at some point.

Jon: I bring that up largely because I’m just imagining there maybe some people listening, you may be out there listening and saying, “Our CTO, our MVP of Technology, or whatever it is, have the plan to build our application on serverless, I’m going to go for it, and Layers is going to make it easier for me.” I will say that there’s more things that have to happen.

One thing we haven’t talked about is just that as part of doing this, there are some tools out there. The one that I’m familiar right now is called the Serverless Framework. What it does is basically lets you treat your local systems—wherever you’re doing your development—it lets you treat where the files are on that as kind of your framework for application development, and then it’s able to translate that into, “Okay, these files are here, those files are there, I’ve got a little […] sort of tell me where everything is supposed to go.” I can pretend this is an application, like a classic application, like an express application or something, but in reality I’m going to do a ton of cloud formation where to go deploy these behind the scenes that you don’t have to really deal with.

The development process can actually feel a lot like it does currently for regular applications. I do think it’s worth taking a look at those. I think that they’ve done a great job of making Lambda development feel like Express Node.js development, so worth taking a look at. But know that underneath all of that, a lot of stuff is happening and whenever there’s a lot of stuff happening, there’s a lot of places where things can go wrong and you have to learn and be able to troubleshoot, et cetera.

Chris: Yeah, absolutely, and like you point out, there’s definitely some great tools for developers for developing. It used to be called SAM Local, it’s now SAM CLI. It might be the same thing that you’re referencing with the framework. Definitely and you can run Lambda locally, right?

Jon: Right. Essentially, if you’re developing a lot of serverless stuff and you’re in the console, it’s like clicking around in there, you’re probably doing it wrong. There’s better tools than that.

Chris: Yes, absolutely. I think going back to your original point, layers don’t actually make serverless easier. They make it better and more modular and allows you to use some of the engineering techniques that you’re used to. But it doesn’t actually make anything easier per se.

Jon: Right. I think the best example of making it more modular is just, now I’ve got to go change something that’s sort of a dependency, “Oh and look, these 11teen functions use that dependency, you now have to go change those 11teen functions. You can change your one layer.”

Chris: Yes, and maybe we should talk a little bit about just layers themselves and how they’re created, how you use them, some things to keep in mind when using them and what not. Layers are really kind of no different than the way that you kind of currently use Lambda functions. You’re basically packaging up a set of files into a zip file. That zip file is your layer and you upload it to Lambda, and now it’s available to be referenced by a Lambda function.

Lambda functions can reference up to five layers. You specify the order in which those layers are installed. Something to keep among these layers is that layers have no knowledge of other layers. You can’t build on top of a layer.

I think a really good analogy to this would be Docker images. Docker has a concept of a base image and then you can extend from that base image.

Jon: Keep adding packages in the future.

Chris: Or even delete from it, and then just going from there.

With layers, you could do that but you have to it all manually. You have to manage it yourself because again, I think that all it really is is a zip file and what Lambda’s going to do is Lambda’s going to unzip that zip file into a very specific path. Every layer that you use is going to be extracted to the same path. You need to make sure you understand what these layers are doing and the coordination between them because they can override each other. If you have a conflict between two different layers where one has this version of file and the other one has that version of a file, it’s going to come down to order of extraction based upon how you listed the order when you reference these layers in your Lambda function on last write wins.

It’s something to be aware of and something you’re just going to have to keep track of that yourself. You probably can come up with your own internal process where, to create the layers themselves, you can do it in an inheritance-type approach where you have a base layer, if you will, and then you can then add to it. But you have to provide the tooling and the process for that’s how you build your zip files.

Jon: You said something that made me picture something that I want to try to say and see if it feels right. If I’m developing a regular old app, not a Lambda function but just like say Express, or Ruby on Rails, or Sinatra, or something, I may have a folder in there that’s database objects and there are other stuff that talks of it that translates between the database and the code.

That stuff may be used by lots of different controllers, let say, and those things would reference those database objects via just the path that they’re in. But if you import them, you just have to be like ./database-objects and then the name of the file. That’s how you talk to those other files from your code. The way it all works is just by the virtue of the fact that the files are where they’re supposed to be on the file system.

It sounded like you are saying to me is that layers essentially works the same way. I’m just going to unzip this and dump the files here and that sort of an agreed-upon convention that I know where I can access those files. Maybe there’s a folder called layers or something, so I know I can say layers.whatever or import-layer/whatever. Is it kind of that way where I just pretend the stuff in my layer is just stuff in a folder that I can import for my code?

The other way of doing this would be like, “Oh, it creates a shared library and that shared library is a binary and then you can talk to it.” That’s true with some other code mechanism but to me it’s sounding like you’re saying, “It’s just code that you can call and import into your current file or class or whatever you’re working on, module.”

Chris: Yeah, in a way. It’s not too terribly advanced.

Jon: Right. That’s why I’m trying to help people understand. It’s like pouring the files into somewhere where you know where they are and it’s doing that for every function.

Chris: Another way of thinking of this is that, like in the past, it you have a Lambda function, you upload a zip file. It’s one and only one zip file. Now with layers, they’re the same. You can do this is in up to six zip files.

Jon: Other functions can see up to five of those zip files without you having to upload them again.

Chris: And again, all it’s really doing is Lambda just says, “Okay, go pull in the zip file, extract it to this directory. Lambda doesn’t care and you can do whatever you want with it. You could have like each zip file is mutually exclusive and it’s talking to different parts of the file system, it could be they’re all talking to the same path and it’s overriding. It gives you a way of packaging things into reusable components. But the underlying technology, it really is just like zip file extract.

Jon: Talking to the layer is the same way of talking to code that you’re already used to. I think that’s the part that I’m getting at that’s maybe sort of subtle is like when you learn a programming language and you want to go access some functionality from a class or module or something that’s in another file, you typically have to do something like import it into the current file that you’re working in.

What I’m trying to help people see is that’s all Lambda layers is providing for you. You can picture that as though it’s your actual computer and you can access those other files just the same way you normally would. It’s also kind of a question. There’s no magic you have to do or those things are available…

Chris: When you have your Lambda function, you specify how it uses these layers. That’s the only linking between it. Your Lambda code is just going to expect that those files are there. You won’t necessarily see specific, like, “I’m going to inherit, I’m going to import or what not.”

I guess a good example would be like if you ran a program in Node.js and maybe you’re using MTM to manage some dependencies, and let’s just say for whatever reasons, you have just the common set of dependencies that you use for your Node apps. Maybe it’s a list of 20 dependencies that’s part of your base, your underlying layer that you create. You have a package.json file with 20 dependencies in it, MTM install, you package that as a layer, you upload it.

Now you have your Lambda function and maybe it has to use Lodash or something like that but Lodash is in the common layer you just created. In that code, you may have a package.json to list some other two or three dependencies that it uses but you won’t see Lodash. Yet, the code itself will be able to basically just require Lodash automagically because it’s going to be there on the file system as if it had been installed by MTM, because you did that in a previous zip file layer.

Again, it requires you to keep track of what are actually in your layers, like what are you expecting to be there. In that sense, it’s a little brittle because you have to manage that. Something else along these lines of the brittleness is that you can change layers, you can version this. You can say, “Okay, this particular layer, maybe some of the dependencies have been updated. I’m going to go ahead and create a new version.” That’s now have version 2 of the layer.

But maybe you have 500 Lambda functions and those 500 Lambda functions use the V1 version of it. So, what do you do now? You can delete the old version but as long as there are other Lambda functions that are referencing it, it’s still going to be kept around. You just won’t be able to use it for any new functions. It won’t show up in the list of available layers but it’s still being held in the background.

You start running into these management issues of some questions like how do I know what Lambda functions are using the old layer? There’s no easy way to figure that out right now. That’s a missing piece. Again, this is something that you’re just going to have to just deal with yourselves and figure it out.

Jon: That’s a really interesting thing, though. I was just thinking about what happens when you update a layer. I think in my first mental model that I created of layers when I was trying to think of it as super, super easy, I was imagining that even though you have the shared code that whatever behind the scenes in Lambda is managing this, is literally pouring out the layers into each and every one of your functions. You just get to treat it though as one thing.

But now, when I think about updates, I realized that whoever implemented Lambda is probably not wanting to go do 500 updates on 500 different Lambda instances. Maybe they do something more like using EFS or something to basically say, “Hey, this is where your layer lives, Firecracker VM,” so now I can go change that under the covers and voila, you got a new version without having to do it 500 times.

Chris: Yeah. Layers are just a file. They’re probably stored in S3, almost guarantee it, they’re stored in S3. They get an ARN and this is how reference them by their ARN. From that sense, there’s no correlation between the running of your functions and that what Lambda’s doing is just saying, “Okay, I need to go and create a sandbox for this function. Does it have layers? Yes, it does. Okay, what are those ARNs? Look up the ARNs. Figure out where they are. Go retrieve the zip file, extract it to the directory.”

Now I can go do the same thing for the Lambda function itself and then go and execute it. As long as it can find that ARN, then do that. That’s what it’s going to do. In that errand, it’s tied to whatever version it was, whatever you specified for that function. When you go and create a new version of your layer, as far as Lambda is concerned, it’s just another ARN,  it’s just another resource, and it’s not tied together to the other one. That’s what this problem is. That’s why you can’t. It’s not going to automatically update your functions and you probably don’t want that either, right?

Jon: Yeah. If you have a long-running function and it’s in the middle of running, you don’t want the world to change underneath it in the middle of running.

Chris: Yeah and then you also may not want to push out for various reasons, like some functions may very well be able to handle a new version but other ones, maybe they would break. Without a richer ecosystem, it’s the right thing to do. But it puts more the burden than you and it’s something that you’re just going to have to manage.

Jon: Got it. Makes sense.

Chris: Yeah. Maybe another limit just to be aware is that layer, just storage base of that is applied towards the same per regional limit for Lambda functions storage space. That’s 75 gigabytes. That means that the total storage space for your zip files that you’ve uploaded for all your Lambda functions and all the layers themselves can’t exceed 75 gigabytes per region. Something to keep in mind there.

Jon: Did you have to just create a Lambda function that creates a new AWS account and then moves the storage into the new account?

Chris: Yeah, and that’s really the work-around. If 75 gigs ends up being something you need more than that, then you’re going to have to create layers in either other regions or other accounts.

Maybe some other things we can talk about with layers are permissions. Again, layers get an ARN, they use IAM for permissions and security. You can use the layers within an AWS account. You can share layers between accounts, or you can also have layers that are publicly shared, so they’re open to the world. It’s the kind of the beginnings of the potential for a layer marketplace, if you will, right.

Jon: Yup.

Chris: This is another area that’s kind of missing. There’s no infrastructure to support that. If you do have a layer that you want to make public and you want to share it with the community, you have to publish the ARN for it and people have to know what the ARN is. That’s the only way that they can see that. There’s no repository for layers. There’s no marketplace.

Jon: I’ve got my VC on the phone.

Chris: Absolutely. It’s one of those things where it’s going to happen. I’m sure AWS will build something but who knows when that will show up but this is definitely something that is sorely needed right now. AWS did and now state they’ve kickstarted that. They have an unofficial prebuilt optimized public layer that they’re publishing that will be available to the community. That layer includes the NumPy and the SciPy packages. Popular packages usually with Python to do machine learning data sign type stuff. It’s the usually difficult packages, not difficult, they require work to setup and install, compile and what not. There’s a layer that has that so you don’t have to deal with that. If you’re using NumPy or SciPy in your Lambda, it’s now super simple to get going with that. You just run […] that public air and for that layer in your Lambda code.

Jon: I love the fact that they’re optimized, too. You just know it’s going to scream on AWS because they optimized it for you.

Chris: Yup, indeed. We kind of already talked about just the mechanics of building the layer. It’s really the same process as what it is to make a Lambda function. You have your files, you do whatever you need to build those files and artifacts and then when you’re done, you zip it up and then you go out to the Lambda console. There’s new AWS APIs for doing this for creating that layer registering it with Lambda and uploading that zip file. Pretty simple, pretty straightforward, pretty expected. That is layers.

After that, the second half of that talk was all about the Runtime API. The Runtime API, this is what opens up the flexibility for Lambda, it gives you the ability to have any language supported in Lambda. Up until this point, before this was announced, there were certain runtimes that were supported by Lambda and you had to use one of those. Python, Java, Node.js.

Jon: The […] that everybody was complaining about was Ruby, which is kind of funny because of all the developer communities out there, I think, the Ruby one was most interested in Lambda and serverless with Lambda.

Chris: Yup. That was one of the announcements that they made, and it […] a keynote is that now Ruby is officially supported as Lambda execution environment. They actually did that via the Runtime API. It’s just baked in, so now you’re able to choose Ruby as your runtime environment for your Lambda function, you don’t have to build a custom runtime for it, it’s in the platform.

This Runtime API, it’s a very simple, straightforward interface that gives you the capability for building whatever runtime you want and running Lambda. If you want to run Cobalt code, you can do that. If you want to run rust, or if you want to do […], whatever, or PHP, all these just opens up any kind of language that you want to do.

Jon: It sounds complicated, it’s like there’s something about the way they named it, or just the fact that it’s AWS, or something that just makes you feel like ooh, that’s hardcore. The reality is you already know how to install those languages on your computer or on your virtual machine. Now, we also know that Lambda is running its stuff on firecracker which is a little mini VM. If you imagine how can I make it so this little micro mini VM can run a new programming language? If you were able to just log in to it and wanted to do that, you would know exactly what to do. From that perspective, it’s like oh, this isn’t rocket science, this is just installing a language as interpreter, or compiler, or whatever. Runtime system onto that micro VM.

Chris: Yeah, that’s definitely a big part of it. Just whatever runtime support that particular environment needs.

And then the other part of this is okay, how do you integrate in with Lambda? That’s the whole Runtime API. There needs to be this interface between Lambda and your custom run time so that they can basically do the back and forth handshaking. There’s a certain sequence of flow events that happen. The first step is an event comes in, have to do things like get the tracing ID, if there’s context associated with that. Then you actually invoke the function, and then after that you have to handle the response and handle any errors to the cleanup and hand that back to Lambda.

There’s this back and forth flow coordination between the Lambda platform, and the runtime environment. That’s the bulk of the work here. It’s one thing to go install PHP, you got to do that and make sure that it’s optimized and it’s installed correctly and whatnot. But now you have to provide that interface so that Lambda knows how to call you and you’re implementing that interface that it’s expecting.

I think the first half of that loop is probably the easier part of handing off a request and having the function run. Probably the more difficult part, depending on whatever runtime you’re supporting, is making sure that you’re actually handling all the error cases. How do you trap those errors, the response, and making sure that that all gets caught correctly, packaged up, and then sent back into Lambda. That can get trickier.

This thing about languages like Node.js and not just the way it handles errors, how do you catch exceptions and everything else. It’s not rocket science, but it’s also not…

Jon: A great little example of Node.js, and one of the things that’s kind of weird about Node.js and Lambda is you can return an error or you can throw an error, and Lambda knows how to handle that. I think whoever implemented Lambda’s Node capabilities had to make a decision… “Okay, what am I going to do with a thrown exception versus a returned error?” For any language you implement with a runtime API, you got to think about that stuff.

Chris: Yeah, absolutely.

Jon: Nobody’s going to do that, right? Any language you want is probably already there, already there was Cobalt, Ruby, PHP. And then since then, I’ve seen announcements around Rust, Pascal, I think I’ve seen Lisp. Unless you’re implementing your own–that can be the case… Maybe SalesForce has its own proprietary language, maybe they want to put some stuff on Lambda. They can do that for their own proprietary language APEX, or if you’re a big company with your own language and you want to use Lambda for it, you can do it.

Chris: Yeah, there’s been a lot of good work done by other folks in this space. If you want to run C++ in Lambda or C, that exists. That custom runtime has been built. Same with Rust, Cobalt, PHP, it’s just growing. The important takeaway is that the runtime API has enabled that. Now, it’s just whatever language you want to do Lambda in, you can do that.

Jon: Cool.

Chris: Yup, and then maybe just to finish up. Mentioned that […] great session, did a good job presenting a lot of content pretty quickly, and precisely. And then also kind of sprinkled it with some tips and things. Just wanted to follow up with a few pro tips that came to light during the talk, that was kind of interesting. They’re all really around Lambda, having Lambda functions, and whatnot. Maybe things that people are not necessarily aware of.

One thing that was kind of interesting, when Lambda runs, your functions run, it has a sandbox that runs and writes. Let’s just keep it super simple so I can just pretend that it’s a VN that spun up, that your code is running. Lambda then gets the option of deciding on whether or not to keep that Sandbox, or to tear it down, or to assign your function to another one. There are a lot of scenarios where your function, repeated calls, are going to be running on that same sandbox. That means that if you’ve written to Slashtmp, whatever you wrote to that on the previous function call, that’s going to still be there assuming that you’re still running on the same sandbox.

This is a technique that you can use. If you wanted to have a very quick, very easy way of caching stuff, you can use Slashtmp in the file system for your Lambda functions as a cache. But just realize, it could be pulled out underneath of you at any time. You definitely want to have a way of rebuilding, or not relying on that being there. You can take advantage of the fact when it is there.

Jon: That’s cool.

Chris: Similarly, same thing goes for global object. If you have global objects in your Lambda function, they will persist between indications as long as your Lambda function is running on that same sandbox.

Jon: Oh my god, I hope somebody is in the middle of a two-week bug tracking down and is like, “Oh my god! That’s what it is.”

Chris: Yeah. A few more issues to keep in mind about just how it actually works, the covers. And then a practical tip that we’ll suggest is two to three layers at most, don’t use the full five. There is a performance cost. This is sequential. When your function is invoked and you have a list of layers, it has to do that. It’s going to do that sequentially, and it’s going to do it serially. It’s not going to be done parallel, it’s going to be done serially. It has to extract the first layer, wait for that to be done, then and only then will it go and extract the next one. And then until it finally gets to your function, and it extracts that one, and then it’s done and ready to go.

Jon: Wait, so your function gets invoked and it’s got to unzip as many as five files in order to even invoke your function?

Chris: If you have five layers associated with that function, absolutely.

Jon: That would take a long time.

Chris: It’s really just how long does it take… how big your zip file is.

Jon: It could take in the order of seconds, I’m imagining.

Chris: It could, perhaps. Definitely. You definitely want to keep that in mind. It is true that usually once that sandbox is initiated, as long as it’s being regularly invoked, the odds are good that you’re not going to have that startup time again because it’s not going to be able to use sandboxes, but again, no guarantee.

Jon: I literally thought today on a Slack that I am on, somebody was asking that exact question. He was like, “Yeah, I’m doing this function running Lambda and I’m trying to get rid of the spin up time that’s going on. Does anyone have a low mental overhead suggestion?” The answers were kind of like, “Yeah, you can kind of do some stuff but if that’s a problem for you, it would be better to not use Lambda.”

Chris: Or just make your Lambda function simpler. This is why a lot of times if that’s the concern or issue, usually Java as your Lambda function and environment, it’s susceptible to this versus other languages. And it’s not necessarily because Java is slower, but it’s just typically Java, the way that it plays out its code, and with classpass and whatnot. There’s just lots of files. It’s a bigger zip file.

Jon: Yeah, basically we’re saying if you’re on Node, we’ve seen Python a little faster.

Chris: If you’re really concerned for performance, something like Go is probably a really great choice. It’s compiled and it’s definitely lower level, much smaller in file size as well.

Jon: And then some people were also saying, and I love reading this, some people were saying I’m the next simplest thing. If you’ve got one function that you need to get done, we’re likely moving these to ECS tasks when they’re not fast enough for us on Lambda.

Chris: Yeah, it just depends on what it is that you’re doing and how often it’s being invoked. We’ve talked about ECS quite a bit in the past on this podcast. You can have things that are constantly running, you can have things that you can easily invoke based upon events and have a task do one and done, you can have scheduled tasks.

Jon: I’m guessing it would be quicker to spin up an image and run a function than it would be to unzip a bunch of files and then run a function on it.

Chris: I think theoretically, Lambda has the potential to be faster if you package it right and if you do things like… you may have to do some warming.

Jon: Yeah, you should get into warming though. It’s like oh my god, I’m working against the tool.

Chris: That’s one of those things where I don’t think there’s a clear cut answer to say it’s Lambda versus ECS. Lambda has Firecracker now, ECS doesn’t, and it’s going to be a while before ECS gets that. Although fargate will probably be the first one that gets it.

Jon: The reason I was thinking a cold start on ECS might be faster than on Lambda is because today I learned that you get the Firecracker thing spun up and then you have to wait for the any layers and other things to get unzipped and laid out into the Firecracker, and then now I can execute it. Whereas I think with the docker image, you just spin it up and go. Everything you need in the docker image is there and ready to be used.

Chris: Yeah, in both cases you have to have the VM, and then you’re now bringing up your executional arm. On ECS side, that’s your image and create a container within that VM on ECS. It’s already prebuilt, which is kind of what you’re getting at, versus on the Lambda side it’s not one image prebuilt. It’s basically a file system. It’s a collection of files, it’s a file system. Again, if you have lots and lots of files and it’s going to take time to unzip those and to write them to disk, and if you have multiple files, then of course it’s going to take even longer.

Again, it depends on how big your zip files are, how many of them you’re using, how long does that take, does the two platforms like what’s the tradeoff between them. Just looking into the specific stuff and see if you’re on Lambda and it’s not working as well for you as you want, are there easy things that you can do, maybe you’re doing some things that you really shouldn’t be doing and you can make that run better. Or it could just be like what you’re trying to do, like you know what, you really should package this up into Docker image.

Jon: Sorry for diving so deep on that. If we have any listeners left, that for me was the biggest thing I learned today, for some reason I thought that Lambda functions were like images. Turn on the image and go. I learned they’ve got a little work to do before they’re ready to rock.

Chris: It’s just a collection of files, it’s a file system. It’s like a USB stick.

Jon: Well, thank you, Chris. Thanks for putting this together, Rich.

Chris: Thank you.

Rich: Thanks, guys. Well, dear listener, you made it to the end. We appreciate your time and invite you to continue the conversation with us online. This episode, along with show notes and other valuable resources, is available at mobycast.fm/45. If you have any questions or additional insights, we encourage you to leave us a comment there. Thank you and we’ll see you again next week.