48. Microservice RESTful APIs – Tips & Guidelines for Crafting Beautiful, Simple APIs

Rich: In Episode 48 of Mobycast, we discuss tips and guidelines for creating beautiful simple APIs. Welcome to Mobycast, a weekly conversation about containerization, Docker and modern software deployment. Let’s jump right in.

Jon: Welcome, Chris. It’s another episode of Mobycast.

Chris: Hey, Jon. Good to be back.

Jon: Yeah, good to have you and Rich. Unfortunately, he’s gone today. He’s closer to me than usual. He’s at a conference or something in Beaver Creek, which is just down the road here instead of three hours away in Denver.

Chris: Conference. Yeah, it’s a ski trip.

Jon: The ski trip was a couple of hours of mastermind planning or something every night. Since he’s not here, we can say that. Maybe just one hour. Today, we’re going to talk about RESTful APIs. We’re going to talk specifically about microservice RESTful APIs and we’re giving out some tips and guidelines for creating beautiful simple APIs. Before we jump in, I just want to know. What have you been up to this week, Chris?

Chris: I have been busy wielding a snow shovel. Here in Seattle, we don’t get snow often. When we do, though, it’s definitely an event. It is back. We get dumped on about six to eight inches of snow, and it’s very cold down into the mid teens at night. At Seattle, we just don’t know what to do.

We’re stymied by this and there’s no one in sight. We’re looking at another snowstorm tomorrow and it looks like the weather’s going to be like this for about two weeks so I’m just like, “What is going on, February? This is not supposed to happen.”

Jon: It’s snow shovels all the way down. We have snow here, too. Just yesterday, I had a conversation with my neighbor about how angry he was that the plough keeps plowing his cars in on the side of the street. I told my wife about it when she got home from work and she said, “I had the same conversation with him. He was so mad.”

Chris: Yeah. What are you going to do, though, right? You’ve got to plow the streets. Where is the snow going to go, right?

Jon: Right. All right, so let’s jump into talking about these microservice RESTful APIs. Why don’t we just start out by doing our usual thing? Let’s define it. What do we have? What’s REST?

Chris: Sure. REST is an acronym. It stands for representational state transfer, which is definitely a mouthful, and it comes from a paper or dissertation done by Roy Fielding, a computer science candidate, I think, obviously working on probably a PhD and whatnot back in 2000. He wrote this paper about what are the architectural styles and designs for network-based software architectures.

A big chunk of this was devoted to this concept of representational state transfer, which essentially is just–the concept is you have resources and collections of resources. You can apply operations on them and get back results. Everything that you need is encompassed in those pasts and whatnot. It turns out that something like HCP is perfectly suited for this and so, in a way, we had REST.

REST was kind of like a description of just how things worked in the web world. You had collections of resources and resources meaning, basically, websites and the pages that belong to them, and you could do operations on those pages.

A very simple example is the browser being a client of a REST API in that you’re telling it to go to some beginning location via URL. That request then gets sent over to a web server to say, “Give me back this particular page,” the page comes back with all its data and included in that page are a bunch of links to other things that users can do. When you click on it on your browser, then your browser knows, “Now, I’m going to go fetch this particular resource for this particular collection.”

Jon: I think there’s a subtlety that you’re getting at that didn’t really dawn on me until–it might have been 5 or 10 years ago, but I was reading the HTTP specification and sort of realizing what was going on in the minds of the people that wrote it and how they were thinking of webpages as these abstract things that are on servers that you can go get, or you can go change them, or you can go make a new one, and I was like, “That’s not really how I had thought about all this before.”

Once I realized that, it was just sort of an epiphany for me that they had imagines these webpages or these web objects living somewhere and needing to be updated, and deleted, and gotten enlisted and things like that. I think that’s kind of the nuance you were hitting on, is that that webpages, or web objects, or web resources were the original. Now, we think of REST resources as these things you might have in a database or a data store but, originally, they were things that the HTTP folks thought of as living in a web server.

Chris: Yeah, absolutely, and that’s it. When this paper came out in 2000, it was just kind of codifying the idea and maybe extending it a bit, but we already had REST. The web was REST and we had clients that were called browsers. We had the servers or web servers, but the API or the microservice, if you will, just ended up being–it was serving web traffic, webpages. It wasn’t a particular application.

It’s a pretty easy jump to say, “Well, now, my application’s not a website. It’s now a message board,” and the same kind of things of apply on going and fetching messages, on creating new messages, on updating new messages and deleting messages, all of the same kind of basic principles that you would have with a website.

I think that’s definitely that particular facet, that way of looking at this, is that the web really was and continues to be the original RESTful application and RESTful API. It’s a very useful way of thinking about things and very simple from the API standpoint, from an implementation standpoint, from, “How do I consume an API going forward for further apps?”

That’s why it’s become so popular for microservices because it is simple, it’s something that everyone understands and gets, and you don’t have to reinvent the wheel. The plumbing exists for all this stuff.

Jon: Right. Yeah. Beforehand, everyone was writing socket code and everyone was doing it differently.

Chris: I was just having a talk with my son about this last night and it was like–because he’s in an advanced placement class, computer science class, and they are–one of his projects that he’s doing is he is doing a client that goes and checks with a game server. There’s this game server service, GoG. I forget what the acronym is but it’s a popular game service and it hosts a bunch of different games, updates and whatnot.

He’s built this client in REST that basically interfaces with their API over REST to go fetch what games out there and then which ones have been updated and has paths to the binaries and whatnot. He’s built this client that goes and interfaces with it, provides a really nice way of doing that, but it’s all REST. His client is making these REST calls and, of course, it’s GOG going over HTTP and HTTPS and so he’s doing it in REST.

He’s become pretty popular. He’s actually got 50 stars on his GitHub repo and there’s been thousands of downloads of this particular thing so it’s kind of cool to see that. I said, “Is there anything else out there that’s like that for this particular game server that you’re going against?” He said, “Well, there’s this other one and it’s written in C.” I’m like, man, all that web code in C just sounds really much more challenging now versus doing it a higher level like REST or JavaScript or […] and whatnot.

I’m going to point out as I go. Wait and let me give you my story about walking to school and snow five miles each way. Back in 1999 or 2000,we weren’t doing REST. It was the wire-level protocol. This is what we had to do. I was writing low-level C code. I was writing completion port I/O, socket-level code. I was doing protocol buffers, packing bytes and DWORD aligning. It’s very performative, of course, but it was also just really difficult, challenging and it took a lot time to do versus–now, it’s just so much easier and simpler so thank goodness.

Jon: Let’s get into a little bit of RESTful basics. Let’s talk about those a bit so that we can eventually make a picture of what an elegant microservice API would look like.

Chris: Sure. Yeah, and I think some of the high-level techniques here or principles is you’re thinking of your system and your API is a collection of resources and individual resources within those collections. These are nouns.

This is what’s basically been served up by your API, been managed, again, whether it be things like webpages, comments for a message board, orders in an ecommerce site, files, videos, whatever it may be. It’s these entities that are the base-level thing that’s being represented, and then you can have collections of them.

That’s one of the key principles there and your path at these things are essentially your URLs. Those are describing those with a path and those paths inside those URLs. You should see nothing but nouns in there, really no verbs because your RESTful API is operating on these entities, which are nouns.

Where the verbs come in are on the actual HTP method that you use. There’s many different types of HTP methods defined in the HTP protocol, things like get, post, put, patch, delete, head, options, and there’s others as well. Typically with a RESTful API, we’re going to use a core sub-set of those to do the kind of basic things you want to do. Normally with an API, you’re talking about CRUD, create, read, update and delete, and those match nicely with these HTP methods.

For me, it’d be easier to be doing an HBCAT and that becomes your verb, if you will, on your URL. To do the deletes, it’s the delete HTP method. To do creates, you’re going to be using post. To do updates, you’re going to be doing either a put or a patch, probably. It’s a really nice alignment with the types of operations that you want to do on these resources.

Again, that’s the first big key principle here, is that your URL should be pretty simple. They really are modeling and representing collections of resources and the resources themselves. Then, for verbs on those resources, you’re using your HTP method, and that leads to a really simple API.

Jon: Right. I think just to put in some examples, if you were going to create a new order, you might have a ‘/orders’ and you would post to that. It wouldn’t be like an API that was like ‘/orders/createnew’. That’s not what you would do.

Chris: No, please don’t do that. That’s a great example because that’s something that I do see happens quite often, people making that kind of mistake. It just complicates it. It’s definitely non-standard, non-commence. It’s not what people are expecting when they look at a RESTful API.

If you’re creating an order, absolutely, you’re going to be posting to the orders collection. That post says, “I’m creating a new entity inside that particular collection of resources and that will return me now a new order with its own ID that I can then reference individually inside the collection.”

Jon: Exactly. Cool. It looks like you wanted to talk a little bit more specifically about GETs.

Chris: Yeah. When you’re talking about your RESTful API and the types of operations that you can do against them, you do run into these concepts like, “What are safe operations and what are Idempotent operations?”

Jon: Interview questions.

Chris: Yeah, Idempotent is definitely a really good one. Idempotent means is you can call that particular API as many times as you want and it’s safe. You’re always going to get back the same representation back at that resource. It’s going to be very consistent. That’s Idempotent.

The things that are Idempotent are, obviously, things that are not mutable so not mutable operations. If it’s a ‘get’ or a ‘head’, I definitely would expect that to be Idempotent but also things like ‘delete’ should be Idempotent. If you call ‘delete’ multiple times, that should be safe. It’s shouldn’t cause problems on the backend and it’s not going to change the representation of the resource that you’re calling that delete operation on.

The same thing goes for ‘put’. Put is for your updating a particular resource in its entirety. You’re sending the complete representation of the resource as part of the update so if you send it multiple times, it should be the same result every time. Those are some of the things or operations that are Idempotent.

Examples that would not be Idempotent are things like ‘post’ and ‘patch’. Those are not guaranteed to be Idempotent. Post is a bit wide-ranging on how you can use that but a common scenario is for creating new objects. If you call ‘post’ multiple times with the same parameters each time, it’s going to be creating a brand-new object so definitely a different representation there.

Jon: This is the first time I really thought about why this Idempotent thing is so important. I think now that you’ve described it the way that you just did, which is one of the clearer descriptions I’ve heard of it, I’m realizing it’s just because of the realities of the web world. It’s just because oftentimes people do double-click on links, triple-click or something happens and there’s some jitter and the request gets sent multiple times.

If I send a ‘delete’ and I delete this person out of the database and then accidentally send that again, I expect the world is going to stay the same. The person is still deleted. Nothing changes. I can send that a hundred times. The first time I send that, it deletes the person but after that, every time I send it, I’m not deleting another person or changing anything beyond what I would have changed.

Chris: Yeah, absolutely. This is a very common scenario when you’re dealing with distributed systems and you have, potentially, lots of users. You also have users that are using, potentially, multiple devices. We all have apps that you can use like in a mobile phone app, whether it be iOS or Android. You have web versions of it. You can have multiple windows up, different browsers.

Just name the users. I may do something in Chrome and then open it up in iOS or maybe it’s already opened on iOS and I try to do the same operation. Dealing with this–and this is kind of just like table stakes. It’s going to happen, and having a safe way to do this is pretty important to think it through so absolutely. That’s why you’ll hear things like Idempotency discussed quite a bit with APIs.

Jon: Now that we’re clear on what Idempotent means, I think there’s still some more discussion that we should have about classifications of operations. Can you tell me a little bit about what safe means in terms of RESTful APIs?

Chris: Sure. Yeah. With Idempotent, we said that’s where you can call it multiple times and the representation that’s returned back won’t have different outcomes. With safe, that means the operation that you’re calling will not modify the representation of the resource that you’re calling that on. Essentially, this is a mutable operation.

Examples of safe operations would be HTP methods like ‘get’, ‘head’. Those are going to be your safe methods that no matter how many times you call that, the representation coming back is going to be the same and it’s not going to change.

That kind of gives rise to kind of an important thing about cacheability. Because this is all based upon HTP, RESTful APIs leverage themselves well for thing like caching, realizing your safe methods are definitely cacheable that can lead to API considerations.

You also have sometimes where you don’t want things to be cached. If it’s something that could be changed on the backend by other entities, you’re going to want to either set a directive like, “Hey, don’t cache this,” or maybe you’re going to set an expiration on it, just things to be aware of there.

Jon: Right, and I think something that you’re kind of alluding to, though, is that because it’s RESTful, because we’re using the HTP methods, our backend can kind of be aware and know that if no other methods have been called other than ‘get’ for the last 5 to 10 minutes, then we can be sure that the next ‘get’ is going to return the same exact thing as the last ‘get’.

Chris: Yeah, it gets a little bit complicated because you have perhaps multiple users, many, many, many users and if they’re accessing the same resources, then you run into those issues where you’re like, “Hey, it really is changing so what do you there?”

Jon: You could have somebody in the database changing things but it creates the possibility–I guess, this is what I’m getting at–of having a fairly straightforward way of knowing when to clear cache.

Chris: Yeah, it’s kind of like one of those things that just comes along with the protocol and the infrastructure. It’s there for you so you can take advantage of caching. You just need to think it through and understand when do you want things to be cached and when do you want.

The truth is that this actually probably causes a lot of bugs for people because some things are being cached but they don’t want it to be cached and so it’s giving unpredictable, unexpected results. It is a double-edged sword but, again, just something to be aware of, something to think through. Understanding when do you want these things to be cached and when do you not want them to be cached can make for a much better API, performance and everything else that goes along with it.

Jon: It turns out that talking about building a beautiful, simple API takes more than our target 20 minutes of what a Mobycast’s length should be. We still do have to talk about authentication, air-handling, and some other helpful tools. What do you say we talk about that next week, Chris?

Chris: I think that sounds like a great plan.

Jon: Okay, thanks for the discussion today and looking forward to continuing.

Chris: All right. See you next week.

Rich: Well, dear listener. You made it to the end. We appreciate your time and invite you to continue the conversation with us online. This episode along with show notes and other valuable resources is available at mobycast.fm/48. If you have any questions or additional insights, we encourage you to leave us a comment there. Thank you and we’ll see you again next week.