68. Microservices Boot Camp Part 1

Rich: In episode 68 of Mobycast, Jon and Chris kick off part one of our Microservices bootcamp, answering the essential questions of what, why, and how. Welcome to Mobycast, a weekly conversation about Cloud native development, AWS, and building distributed systems. Let’s jump right in.

Jon: Welcome, Chris. It’s another episode of Mobycast.

Chris: Hey, Jon. Good to be back.

Jon: Yeah, good to have you back. It’s been a few weeks, not for our listeners, because we try to keep on coming out every week.

Chris: Don’t tell the secrets.

Jon: Oh my goodness. Yes, but we haven’t done this in a few weeks. I’ve been on vacation for the last few weeks. It’s kind of fun to be back and have something new to talk about. This week, we’re going to talk about Microservices. We’re doing this little series, Microservice and Bootcamp, and this is part one, it’s designing and sizing in Microservices. Before we get into it, what have you been up to lately, Chris?

Chris: Boy, that’s a good question. It’s summertime now, so kind of like doing that transition every year of going from folks in school, kids in school, my wife’s a school teacher, and transitioning to the, “Hey, it’s our vacation.” A part of me is a little sad everyone’s out on vacation except for me, but that’s okay. My vacations come later in the summer and then also it’s kind of been interesting, my son’s got a full time job. Actually, he’s working too, he had no break. He graduated high school and immediately went to a full time job, and he’s got to be up early, out of the house every morning by like 7:15 AM. He’s not getting home until after 5:00 PM, and dealing with traffic, and all that good stuff. It’s pretty interesting to hear him come home and be like, “I’m so tired.” I’m like, “Welcome to my world.”

Jon: That’s such a brutal transition. I just remember being young and suddenly coming to the realization that I was only going to have two, maybe three weeks off a year.

Chris: I was like, “Why was I so excited to get here so quickly?”

Jon: Yeah, and I also was on vacation. We went to the east coast and I visited some old friends from college. I didn’t go to our 20-year reunion, but that’s what it would have been for me. I went to visit a pretty incredible little event called Aspen Ideas Festival, which is a bit hoity-toity, but super interesting. It has a lot of really cool speakers and a lot of really cool people there. I’ll try to capture some of the energy that that gave me and take it forward for the next few months.

Chris: Yeah. I’m kind of jealous. It sounded like it was quite the soiree, rubbing elbows with all the folks and the communities there in a really beautiful part of the world as well.

Jon: It’s so beautiful there. I mean, I live in a really beautiful part [Eagle 00:03:02] here, but I was like, “Ugh, I got to get over to Aspen; we’ve got to move here.”

Chris: We talked about this the other day. It’s like the Dumb and Dumber movie, “We got to go to Aspen.”

Jon: Let’s start here. We’re just going to go through what is a Microservice, we’re going to talk about their business value, we’re going to talk about what functionality we should put into a Microservice, and then we’re going to try to determine the size—how to size them. You can’t do any of that without just giving a brief overview. We’ve talked about Microservices before but let’s give our Kelsus definition, of our Mobycast definition of what a Microservice is.

Chris: Yeah. Maybe just to preface this just a little bit of why talk about this. Looking back, we talk about Microservices all the time. Everyone does, it’s a buzz word, it’s really common, and everyone’s heard of them, everyone’s using them, but we never really specifically addressed it. We have received comments from listeners saying, “Hey, it would be really helpful to have a more nuanced discussion around what are the practicalities of Microservices. It’s really easier to talk about it in theory, but I need help, tips, guidelines for how to actually make this work in the real world.”

It gets a little hairy; it’s not straightforward. I’m hoping that’s what this multi-episode series becomes. There will be a little bit of definition in theory, but for the most part, it really just focuses on just the practical how-tos, the gotchas, what’s worked and what hasn’t worked, and hopefully, that ends up becoming really helpful for folks.

Jon: Great. Yes.

Chris: With that, let’s have a definition of a Microservice because that’s going to help set the framework here for we then go on and design these things. What do they do? What’s their sphere of responsibility, if you will and whatnot? In general, Microservices is the process of factoring your business capabilities or logic into well-defined domains and subdomains. It’s a mouthful, but in general, it just means it’s divide and conquer. You’re looking at your business at the problem space that you’re dealing with and you’re just identifying the chunks.

What are the natural sized chunks? What are those domains? And then for the bigger domains, breaking them down into subdomains. You’re doing that modularization. Looking at your system at your problem space, your business logic, and identifying, “What are the key pieces? What are the components?” That becomes that first process, that part of the process.

Jon: I was going to say that hopefully, you’re doing that process if you’re writing a monolith too, that’s useful inside monolith. Microservice must be a bit more than that.

Chris: Yes, absolutely. Basically, those domains or subdomains, those become candidates for those Microservices. As you said, if you have a monolith, hopefully you’re doing this too, because if you don’t, then it’s just going to become a big ball of mud. If there’s no organization there— to that monolith—if there’s no modularization to it, it’s quickly going to get out of hand. If you do have a successful monolith, then you have been doing this out of necessity.

Microservices, first and foremost, they’ve been factored down into some reasonable sized chunk of doing something. Basically, they’re doing that one thing of that particular domain or subdomain and think of them at the module level of packaging; they stand alone on their own. Then another thing that I think is pretty useful here for a definition is to say that they have their own data model which is completely decoupled from the rest of the system.

Jon: That’s what I was waiting for. For at least in my mind, that really separates it. I think a lot of incorrectly put together Microservices don’t do that right. They’re still pretty coupled from the point of view of the data model.

Chris: You can definitely have Microservices that don’t have any state in which case they don’t have a database, but for those with state, that do have databases, I think it is pretty important that in order to take advantage of all the benefits that Microservices give us, that there is one and only one owner of that state, so that it is fully encapsulated. If you have multiple services all hitting the same database table, that’s a huge coupling between them. You just don’t have Microservices anymore at that point. You’ve done some work to have the modularization at the logic level, but the state is completely coupled. I think for us, definitely through this series of talks will definitely stick to that, that Microservices with state, they own it, and only they are able to directly access that.

Anything else that does need access to that state, they’re going to get access to that via the only Microservice. The only Microservice is going to provide APIs. Everything here is driven by APIs and making those API calls. Whenever consumers want to get access to that, they can make those API calls to the Microservice, and it is responsible for touching that database only.

Jon: Yeah. I mean just concretely, if you have say, a user Microservice, and maybe a whole separate other Microservice for user profiles, the user profile database cannot be changed by the user Microservice. You have to call user profile Microservice to change that profile. I just wanted to kind of make it concrete.

Chris: Absolutely. That’s kind of a description that we can use for what is a Microservice, something small, of reasonable size, it’s modular, it’s kind of independent, if it has state, it’s the only one that is directly managing that state. “Okay. Great. That’s a Microservice.” So, why? Why are Microservices so popular? Why is it such a buzz? Why are people adopting them?  There’s a lot of benefits to Microservices; it’s the same benefits that you get at the code level. As we’re writing code, we know that it’s probably not a good idea to have a single file with 1000 lines of code in it, with three functions that are all 300 lines each or something like that—that’s not a good design, right? It’s very hard to maintain. It’s hard to understand. It’s very complicated.We know that by breaking that up into smaller pieces, that that’s going to be a lot easier to deal with in every regard.

This is really what Microservices boil down to. The benefits of it is just it’s that divide and conquer. It’s breaking things down into smaller sizes that are independent, reducing the coupling between them, so that you get a lot of these specific benefits. There’s many benefits, some of the ones that we can specifically talk about would be development flexibility. It becomes much easier for you to onboard new developers to your code base. If it’s a smaller code base. If you have a big monolith system that even if it is kind of organized in its structure, to break it down into modules, you still have to seal that stuff and […] way through it. Versus, if you have a Microservice just for the user profile store, that’s all you have to look, that’s all you have to come up to speed on.

Jon: Right, yeah. The new developer doesn’t need to know. If the new developer is going to work on that geolocation service, they don’t need to know anything about the payment service at all. It can’t break the payment service by updating the wrong code or forgetting to put up certain configuration, you might be able to—I’ve definitely even seen it happen in a more monolithic architecture.

Chris: Yeah, absolutely. Not only is it easier to onboard new developers, it’s also easier to scale teams, this makes it much easier just from a staffing standpoint where each one of these Microservices can have their own team. Maybe a team of one, bigger ones, maybe multiple people. That as you do that decomposition, it’s basically making them much easier to fill out your engineering team, and how to staff that, and to grow it, and to not have the problems that come with getting bigger, and bigger and the interdependencies. Because again, these are, for the most part, very loosely coupled; really, only three interfaces. You can scale people faster, and easier, you can onboard new people, they can become more effective quicker with these.

There is also the benefits with deployment. You now have quicker, and faster deploys because the unit of code that you’re deploying is smaller. Again, it’s not coupled to other things. You don’t have to wait for, maybe someone else to check and code from another brand. I mean, those kinds of issues just become much less of an issue because you’re dealing with a smaller piece of code. It’s possible to have quicker, faster deploys. You’re also reducing the risks of those deploys because the amount of code that’s changing is less than if it is potentially, with a big monolith. I mean, I remember in the past, doing an actual build and deploy, that was an event.

Jon: Yeah.

Chris: It’s like a once-a-month type thing. For some teams, it’s once a quarter, and it’s a big thing, it’s all hands on deck. You know there’s going to be some downtime, you know things are going to go wrong. It’s just a big deal. With Microservices, that’s much less. If that is the case with Microservices, then you’re doing it wrong, something’s gone really wrong, or you’re just calling a monolith a Microservice, maybe.

Scalability is another really great benefit of Microservices because you now have much more refined control over exactly what parts of your system that need to be scaled, maybe there are two or three particular API calls that are the most popular parts of your system, and maybe it’s user profile. Fetching the user profile is something that’s just called quite a bit. Maybe it has the most performance overhead, or that’s the most frequent call.

Well, if you lift that out and make that its own Microservices, now you can just scale up the machines that are responsible for providing that piece of functionality. Versus maybe the other 90% of the code base is just fine on less hardware or smaller system. Instead of having to scale everything up for the least common denominator, you can now be much more strategic in how you scale. It also makes scaling a lot easier just from an engineering standpoint as well. Going smaller makes things just a lot easier across the board. Those are some of the benefits of Microservices. Is there anything else comes at the top-of-mind, John, that you can think of?

Jon: No, I don’t think so. But I always have a tendency to think about things from the beginning of a project, or the beginning of a company, and what sort of happens then, versus what happens later. I think that you’re about to get into some of those tradeoffs because, at the beginning of a company, some of this stuff actually works against you. That scalability, for example, when your scale is small, having to have a different database for every single Microservice is actually more machines that you probably need; maybe you can run the whole thing on less than one machine, and suddenly, you’ve got to have multiple databases in order to even run […].

Another one, the other tradeoff that you’re about to get into around some complexity that you add to the system by having things decoupled, and knowing how to communicate across interfaces that are not just code interfaces, but rather like protocol interfaces, like HGP—that’s a little trickier. It could be more work if you create a lot of Microservices at the beginning of a project, and there’s more non-business oriented heavy lifting that teams might have to do at the beginning of a project that’s really spread into many Microservices that later might pay dividends, but in the beginning, it might feel you’re doing a lot of work that you could have done so quickly with the monolith.

Chris: Let’s talk about some of the tradeoffs. It’s not all roses with Microservices. There are some tradeoffs. Probably, leading the way with those tradeoffs is just the fact that Microservices result in more moving parts. You’re building a much more distributed system with Microservices. Instead of having one or two elements, maybe you now have five or six elements. At the system level, definitely, more complicated. As you pointed out, as you’re starting out, “Why do I need these six things?” Really, […], it would be faster, and it’s more than enough.

These get into some of just the practical considerations as well. You have to just look at every situation for what it is, and know what the benefits are, what the tradeoffs are, and make those decisions on what’s best for you and your system. If you’re making a prototype application, there’s probably no reason to go full-blown Microservice if you know this will never get bigger than a certain size, or if it’s going to be something that’s going to be retired, in the relatively near future, then that has to factor into it. But if do have hopes of growing, you do expect this to be long term service, if you do expect to have a growth in user base and just folks that are…

Jon: Especially in team size.

Chris: …absolutely in team size. All of those things. If growth is in your future, if longevity is in your future with it, then you’re going to have to put some investment, pay down some investment at the beginning. It’s going to require some process, culture changes to deal with for folks that are not used to doing these things. You’re going to get pushed back from developers and maybe DevOps folks and whatnot because it’s kind of a crazy idea. It’s like, “What? Instead of having one database, we’re going to have four? It’s kind of weird and seems like a lot of extra effort and whatnot.” But again, that’s just part of that initial investment to pay upfront that again, if you do grow both in team size, and software size, and overall load on the system, it’s going to pay off in spades.

Jon: A few things that are coming to mind before we get into how to do this that I think might help people that are having those fights. One side is like, “We need to break this down into more service.” The other side is, “No, this is fine the way it is. Let’s keep it all together in a larger app.” I think the thing that I like to keep in mind is that, as long as you’re organized in how you write your code, and how you modularize it inside of a larger monolith, or as long as you’re organizing how you produce your Microservice, so either side that wins, what I like to keep in mind is that when a problem happens down the road, guess what you’ll still know how to do, you’ll still know how to code.

I’ve been thinking that way for 15 years. I’ll still know how to code when we run into the problem. I’ve never seen a company with capable people that know how to work together that hasn’t been able to get themselves out of a code issue. It’s more when that people don’t know how to work together that really fall over. Just keep in mind that you’ll still know how to code even if you don’t make a perfect decision is a great way to move forward.

Another kind of quote is from Martin Fowler, that says the same thing, but in a different way, “When you run down the road into something that seems hard, make the change easy, and then make the easy change.” That might mean breaking into Microservices, and then write the Microservices. Okay let’s get into it, a little bit more about what functionality we should put into Microservice.

Chris: Yes, so this gets into more of the nuts and bolts. It’s like, “Okay, great. We’ve got the definition of a Microservice and what it is. We talked about the pros and cons and we kind of decided that, yeah, this is something that we want to embrace and use. How do we go about deciding what is a Microservice versus what isn’t?” I think one of the things that’s helpful here is to kind of think of it is there’s two broad categories of Microservices: there’s the vertical style, and there’s the horizontal rate.

I think the ones that are pretty straight forward and much easier to grasp are the vertical Microservices. These would be going after very specific application, business logic type space. They’re performing a specific feature or task of the system that you’re building. It could be that if you’re doing a….

Jon: Well, I mean, I’ll just use examples from when we’ve actually done this. We had a lawyer app that lets you connect with lawyers and schedule meetings with them and pay for those meetings. A big part of the app was a monolith that did all of that stuff. But then they wanted the ability to add in a scheduling component so you could literally look at a calendar and schedule. That scheduling component, that whole part of the application was new and wasn’t there before, instead of just adding it to the existing monolith, we made a Microservice out of it. It had its own everything. That would be an example of this whole feature—a feature that became a Microservice.

Chris: Right, yeah. Absolutely, it’s a great example of one of those vertical style Microservices. Another one might be, if you’re doing like a software development tool, and then maybe have something for doing that code reviews. There could be a Microservice just for handling the code review process of that as well. It’s a specific business…

Jon: I’m thinking about GitHub right now. GitHub literally still has the rails application, but I bet the code review stuff that they put on top of it is a separate service.

Chris: Well, they recently acquired Pull Panda, so that will be considered a Microservice, or it’s a separate service. These vertical Microservices, again, they’re the natural breakdown of applications, and features. They align closely with those domains and subdomains of the business capabilities that we talked about earlier, and kind of defining what a Microservice is. It’s just going to be up to you and your particular problems based on what you’re building, to identify what those vertical units are, but it should be a pretty natural consequence.

Jon: Right. I want to add another thing to this, as you’re starting an application, and your thinking about how to architect it, and what Microservices to build. A lot of times, because Microservices are so useful for scaling teams, a lot of times this will naturally fall out based on your initial team. If you have three people working on something—three people working on the back end of something—you might just decide, Julie works stuff related users, Steve works on stuff related feature A, and Sharon works on stuff related to feature B, and they’ll work on them separately as Microservices. Then from there, they may split again as the team grows, but initially you just have three people not creating any dependencies between each other. It’s kind of a nice way to start a project.

Chris: I think what’s going on is like, “Implicitly, you’re doing that” that design thinking up front like, “Okay, what are the different domains?” and that’s what these people are working on, which ends up being kind of a circumnavigation way of actually doing that process and say, “Yeah, these are Microservices, you just identify what people are working on.” The same kind of thought process is happening in the background. Those are the on the vertical Microservices.

The other kind of broad category would be the horizontal ones. These are core foundational Microservices. They expose functionality that would be common to other Microservices kind of need to consume. Maybe a good way of thinking about this is a lot of the cloud providers, so whether it be Azure, or AWS, or Google, they have a bunch of just platform services that you can consume to go build your systems.

If you need to store files, so Amazon S3, and so this is a service that exposes an API. You can do your inputs, you can retrieve files from it. Similarly, identity would be another type of a core common platform service, where you need to get the identity of a user, understand what their permissions are, and again, this is going to be something that maybe all your vertical Microservices, or many of your universal Microservices, would need that kind of functionality. Instead of putting that functionality inside each one of those Microservices, instead you can just carve that out, and say, “I’m going to packages of as its own Microservice,”  Let these other services become clients, become callers to it, and consume that functionality via an API.

There’s lots of candidates for these kinds of common horizontal Microservices. Most of the time, you’ll probably be consuming them from other providers just because there’s so many out there, and there’s so many…

Jon: Other providers like AWS.

Chris: Yeah, right. We talked about this before, I mean there’s 150 services just from the AWS console that you can get access to, and a lot of those would be candidates for this. So, there’s SQS for queues, and things like Reddit for caching, Cognito and IAM for permissions and identity, S3 file storage, systems manager parameter store for configuration. Lots of those core foundational services from other providers.

But sometimes you will discover that it makes sense for you to have your own as well. You may have some very specific functionality that’s just part of your business logic, or business rules, but it’s so core and foundational, that it’s something that all of your other Microservices will need as well. You may you may find yourself identifying that you do have some candidates for horizontal Microservices.

I think a common one here is it some of the crosscutting concerns. Whether it be things like metrics, or logging, or authorization, […] base access control, those kinds of things, end up being good candidates for doing this. You kind of get a hub and spoke architecture. Those common core services end up becoming the hub if you will and then your verticals are the spokes that are making calls into it. It’s going to be one of those things that’s going to be very specific to your situation but I would really strongly recommend that if you find yourself duplicating logic across multiple Microservices, then definitely take a step back and think about, “Does this make sense to have it as its own?”

Jon: Can I modify what you just said and say, it’s not so much duplicating logic but duplicating state, if you know what I mean. If you have to have the same state in your databases and keep in sync across multiple Microservices, then you may have a problem. If it’s just code, you can just make a library, and have your Microservices all use the same library. But it’s when every Microservice has to know about the same thing, it’s that every Microservices database has to know about whether the user is logged in. It’s kind of keeping track of that state in some sort of persistent state. They’re not saying like, “That’s maybe not okay. I’ll keep track of this all over the place.”

Chris: Yeah, and it’s definitely both scenarios. I mean for sure, it becomes even more important when state is involved and you’re trying to duplicate that. Even the logic ends up becoming problematic because what if you want to update that logic. If it’s shared library that’s actually inside eight other Microservices, that means you have to redeploy eight different Microservices. A version in problems potentially, “What version of library they’re using?” If you take that core functionality and you encapsulate it as a service on its own, you basically freed yourself up from some of those problems.

As you pointed out, state becomes even more important to do this, because trying to keep data in sync across multiple different sources, that’s one of the hardest problems out there along with naming things. The two hardest problems in computer science are naming things and caching things.

Jon: I guess what I want to say though, I agree, and I think we’re on a real sensitive area here, because this is kind of where some of the religious debate comes around Microservices. Many people listening might have been involved in a project where people try to break down their core services in the sort of horizontal services too much and then just ended up with a soup of services that are all dependent on each other. And then, “Oh my god, how do I even find where the problems are because I’m six layers deep every time I need to do one thing.” That, I think is real. It definitely has happened to people, and people definitely quit jobs over it.

Here’s me always sort of thinking about things from the beginning of an application, maybe a good rule of thumb to start out with is that if you are going to do both horizontal Microservices and vertical services, to try to keep your horizontal services one layer at first, and you may decide you need to break them into multiple layers, but just try not to do that unless you have a really good reason or previous experience that tells you it’s a good idea.

Chris: The number of horizontal Microservices you have is going to be very few compared to the number of verticals that you have. If you find yourself having lots of horizontal, there’s probably something wrong there. Just looking back over my experience in the systems and the various projects that I’ve worked on, the times that we built our own horizontal Microservices, they’re a few and far between. They’re very considered decisions. They are more difficult to deal with and they do lead in some of those problems that you’re talking about. You can get into this problem where it’s like Service A is dependent on Service B, and B is dependent on C, but C is dependent upon A or something.

You have this thing where it’s like, “Wait, we can’t even start up anymore; we’ve got a loop in the system.” You can get in those situations where it gets ugly really quick. From just a practical standpoint, you definitely want to be thinking long and hard about how you’re dividing those stuff up, what really is a core service that’s going to be part of your hub in the architecture, and just realize that that’s a considered decision. Again, these are definitely more difficult to deal with because of these dependency issues, and like, “What are your deployment dependencies?” And whatnot.

One of the things to kind of keep in mind is that normally when you’re working on software and doing deployments, you can probably have multiple different environments. You might have a development environment where it’s relatively unstable, and that’s where you’re doing your system testing, and integration and things are breaking, and this is where your bleeding-edge code is going. You might have another environment that’s much more stable where you can have other people do functional testing, and user testing, and UI testing, whatnot and then you’ll have an environment for production, “This is what your end users are using,” so multiple different environments.

Your vertical Microservices definitely will go through that flow of being in each one of those environments. With these horizontal Microservices though, you probably don’t want to have the same approach to it because this is part of your hub because it is depended on by these other Microservices as a key dependency, it needs to be stable. If you have a development environment where you have an unstable vertical Microservice, if the dependent core Microservice that it depends upon to do its basic step, if that’s also unstable, well then, it’s just complete chaos; you can’t do anything there.

I think the recommendation here would be with your core horizontal Microservices, have a single stable environment for those, and that’s what all your other environments will use. Dev stage, prod, will all refer to that one stable environment for your horizontal core Microservice. That way, they don’t have to worry about that being unstable. You have to work around, “Okay, what’s my process going to be for testing new versions of the core Microservice? How do I integrate that?” But you need to hide that from your consumers.

Jon: Right. I was thinking, there may still be dev environment there, but it’s just not something that dependencies would make use of.

Chris: Exactly.

Jon: I think we can finish off there. Next week, we’re going to talk more about Microservices, we’re going to talk about how to move from monolith to Microservices, and we’ll talk a little bit more about how to size your Microservices. I think we’ve done it. We’ve really gotten into more practical things to do with Microservices, and how to think about it from the point of view of a team. Do you have anything else you wanted to add before we close off here, Chris?

Chris: No, I think this is definitely a good point to finish up this week, and then leave this rolling into the next one where we, again, get more into just the practical, the nitty-gritty of, “Okay, how do I make this real given just what goes on in the real world? How do I size these things? If I do have a monolith, how do I start becoming more Microservice-oriented? Not everyone can be […], so how do I do that?” It’s not an easy path.

Jon: Right. Just as a teaser, I’m really excited about a future episode that we’re going to do on Microservices applied to UIs, this idea of microfinance, but we’re not going to get there until we kind of lay all the groundwork. I’m excited for that too.

Chris: Yeah.

Jon: Thanks, Chris. Have a great rest of your week.

Chris: Alright. Thanks, Jon. See you.

Jon: Bye.

Rich: Well, dear listener, you made it to the end. We appreciate your time and invite you to continue the conversation with us online. This episode, along with the show notes and other valuable resources is available at mobycast.fm/68. If you have any questions or additional insights, we encourage you to leave us a comment there. Thank you, and we’ll see you again next week.