69. Microservices Part 2: Sizing, Decomposition, and Dismantling the Monolith

Rich: In Episode 69 of Mobycast, we get hands-on with part two of our Microservices bootcamp and discuss sizing, decomposition, and dismantling monolith. Welcome to Mobycast, a weekly conversation about cloud-native development, AWS, and building distributed systems. Let’s jump right in.

Jon: Welcome, Chris. It’s another episode of Mobycast.

Chris: Hey, Jon. It’s good to be back.

Jon: All right. Let’s talk about what we did. We are missing Rich, as you can tell by my intro. He’s on vacation. He’ll be back, I think, next week. What have you been up to?

Chris: I’ve been transitioning with responsibility. I had a client that I’ve been working with for the past two and a half years. Just recently, I’ve completed my transition plan to roll off that so I can go to work on some new initiatives here in Kelsus. I’m really excited about that.

Jon: Yeah, we’re very excited about it too. We’re going to do some big new things at Kelsus. It might involve this here, in MobyCast, stay tuned. I also have some personal news. Yesterday, I got an email, and I was like, “What? Does that really say what I think it says?” It said, “You won a raft.” I won a white water raft in a contest. I’m kind of excited about it. This is a small raft but it’s big enough for my wife and I, and our two kids. It’s like a white water raft to go down the Eagle River or Colorado River or whatever. It’s not just a raffle, there was some skill involved in winning this raft, Chris.

There was a contest put together by the Eagle Value Watershed Council or the Eagle Watershed Council where you could guess when the peak runoff would be, how many cubic feet per second it would be at the gate at Gypsum, Colorado. I guessed the closest.

Chris: Wow.

Jon: Yes.

Chris: Good one. Go look for gambles now. Let me guess, you won the raft but you just have to take care of the shipping and handling. That’s like $400.

Jon: No. Eagle River Watershed Council is one block away. I drove my truck over to the back and drove it home. It was great.

Chris: Cool. Congratulations.

Jon: Thank you, pretty exciting. Maybe some of these new initiatives that are coming out of Mobycast will take a little longer than we expected. 

Chris: “Where’s Jon?” “He’s on the river.” You’ve seen the movie A Christmas Story, right?

Jon: Yeah.

Chris: He wins the major award. He’s like, “It’s a major award.” He won it with skill because he was doing crossword puzzles or something. This is your major award. Go ahead and put that raft in the front window there so that every one who walks can see it.

Jon: Now that we’ve got that out of our system, today, we’re going to talk about Microservices. We’re on part two of our bootcamp. It’s hands-on Microservices. Today, we’re going to talk about sizing, decomposition, and dismantling the monolith. I’ve read the outline and talked to you about it before this show. I’m really excited for what we have to say today. Go ahead and get us started, Chris.

Chris: Yes. We’ve kind of covered the theory, the definitions. This portion of the series is really just about rolling up the sleeves and how to use these in practice, and what are some of the trickier aspects of implementing Microservices in the real world. Hopefully, some of the folks will find that pretty interesting and useful.

To start off with, we can talk about how do you determine what the size of your Microservices should be? What is that unit, that incremental unit, that you should be breaking things up into.

Jon: It feels like naturally the size of the Microservices whenever you want it to be plus all the dependencies that start working their way into it. Then, pretty soon it’s huge.

Chris: Then, we’re back to the monolith.

Jon: Right, exactly.

Chris: Yeah. It’s one of those things where it’s determining the right size. It’s going to be different for everyone. It’s really going to be based upon just your familiarity with some of the various topics, operational readiness, your team size, and abilities there. Especially if you haven’t really been working too closely with Microservices. Maybe you’re CI/CD pipeline is not as robust as you’d like it to be. It’s going to go all in all the Microservices. Say now we have instead of one thing, we have eight things that are individually deployable. That might be too much. It’s like you’re buying enough way more than you can handle. 

Jon: Sorry to interrupt because that’s really interesting. It’s just so important that Microservices have this extra load that comes with them around just operational stuff that we have to do to keep them happy. That stuff gets built over time. Just something to keep in mind, if you’ve been doing Microservices a lot, you have all that stuff because you’ve been doing it for a long time. You can more quickly spin up a new Microservices and it’s just easypeasy. As you’re doing it for the first time, you’ve got a lot of extra nonbusiness logic stuff to think about. Yeah, okay. Go ahead.

Chris: Absolutely. That’s definitely something you need to consider. It’s just from an operational standpoint, from a culture standpoint, from a process standpoint―how comfortable am I with just dealing with multiple things that I need to work with. You also want to keep in mind your team size and what people have been working on. It’s probably a good idea to think of sizing your Microservices so that they’re something that you can break up and have a team own it. How big that size the team may be. 

You can define a team of size one. You can be a team of eight people. That is one of the advantages of going to Microservices, it allows you to break things up. They standalone, they’re independent. You can go give it to a particular team and that team can own it. Other teams really don’t have to know about that―that service. You just have to have a contract in place for how they operate with it―what’s the API and what not. As far as the specifics of that service, they don’t need to know.

You want to take that into account when you’re trying to decide, “Hey, how big should this Microservice be?” Make sure it’s something that the team can handle. This is where we can use the term cognitive load. Cognitive load, it refers to the used amount of working memory resources that you’re using. Basically, it’s like, what can you keep in your head without having to go to refer to something else. It’s like RAM. It’s like what can you keep in contact without having to page the disk, swap it out, and go look something up, or go refer the documentation or whatnot. You want to keep that in mind. How much do I have to know? How much do I have to keep in my head? The bigger that amount of demand on my working memory resources, the more likely that this can be too big for the team. 

There’s a whole bunch of research and writings that go into this like cognitive load. It’s a very common, well researched way of measuring things. There’s actually three different types of cognitive load related to it. Is it loads, is it demands on the specific topic or task that you’re working on? Versus is it some of the extraneous information that, like the environment, you have to know about? Versus germane cognitive load is basically the tribal knowledge, the synthesis, how does this thing play within a more complicated system. Those three things all go into cognitive load.

I think that the real takeaway here is just make sure that it’s sized so that knowing all those three types of things, like the specific code base, it’s environment out of the ploy, and how it operates on things on the system. That is, again, small. It’s sized sufficiently so that people on that team all can deal with that without not knowing what to do, without consulting documentation, “I have to go talk to someone else on the team.” That’s probably a good guideline for how big your services should be.

Jon: Right. I guess that would be something that you would learn how to do through experience, and also through not just past experience, but current experience. There’s not a thing that says if Microservice has 35 Trello cards, then that’s the perfect cognitive load for Microservices. It’s not that. You can’t just know, like, “Oh my god, I just can’t keep this thing in my head anymore. Maybe we should break it up and put it into different pieces.” That’s like the current experience―a part of it. “This is getting too much to handle. I feel like we’re getting slower as a team because every time we have to go touch this nightmare product, we have to go recode for an hour before we can even do anything.” That’s probably too high of a cognitive load system.

I guess the past experience would just be like, you’re working within a team, the product manager’s like, “Hey, we’re going to do this, this, and this. It’s going to be great.” Then, the architect’s just like, “Oh, yeah. It’s going to be in this type of system.” You might raise your hand and say, “Oh, I built something like that before. The cognitive load got too high so we broke it down like this. That would be how you would use past experience to suggest breaking things into multiple Microservices to keep your cognitive load overhead manageable.

Chris: Yeah, it’s a process. Experience really counts here. It’s going to be something you’re going to get better and better at. It’s one of those things when you do exceed the cognitive load, it’s going to be painfully apparent, not when you’ve gone too far. That’s part of that learning curve―that experience. 

Part of this is to document. Think about documenting. What is this operating with? How many dependencies does it have? How difficult is deploying or monitoring or dealing with issues that come up? If you’re making […], how big your […] is? All that stuff will be clues into how big this thing really is and how complicated it is. 

Jon: Right. I kind of touched on this last week and I want to hit it again because it’s one of my favorite things to talk about. I think teams can get to a place where they feel like breaking something down. We’ll talk a little bit more about breaking down monolith but even splitting up too larger than a Microservice. They get into this place where, oh my god, it would be so much work to fix this and make it better. Let’s just keep adding on to it and dealing with the little pain each day instead of the big pain of fixing it. 

Then, tell me Chris if this has been your experience too. So often I see that when finally, “All right, time to swallow the pill, break this down, and do the right thing.” It’s like a week later, “Oh yeah, it’s starting to come together.” You just lived through three months or six months of pain to avoid a week of work? That happens all the time. It’s not so hard to address some of these big problems. Sometimes, sure, the one area where it might be more than a week of work is when there’s just a massive migration and that could take three to four weeks. That could really slow you down. That’s painful. Still, it’s doable. You can do it. We’ll get into migration in a little bit. Let’s keep going.

Chris: Yeah, cool. That’s sizing. Again, really specific to each situation but definitely take into account your team, also your operational maturity, just dealing with services in general. Those should be two of the big things there. 

With that, let’s talk a little bit about moving from monolith to Microservices. We talked about this last week about what are the benefits of Microservices and what are some of the advantages versus monoliths. As we change that shift and thinking of going from monolith to Microservices, what do we need to consider? What are some of the strategies there? A big part of this ends up becoming the service decomposition. This goes along with sizing to some respect. What are appropriate candidates to become Microservices? Especially, how do you do this when you have an existing monolith and you decided you wanted to go to Microservices?

This is a really common scenario. This is the one that just about every single software team runs into at some point. They have some legacy codebase. It’s a big monolith. They do want to go to Microservices. How do you do that? The analogy is like you’re trying to change out all the parts of a race car while it’s going 150 miles down the road.

Jon: And the race car is made up of a solid brick aluminum.

Chris: Yeah. If only, right? Because then it’s actually pretty easy in one part. Yeah. Again, this is the norm. This is the problem that every single software team faces. There is that common problem of, “Where do you begin? How do you do this?” It is going to be painful. 

Spoiler alert, this is not an easy process at all. Before you embark on it, just make sure that this is the right path for you and your situation. You may look at the pros and cons of going from monolith to Microservices. You need to look at your operational maturity. Do you have CI/CD pipeline? Is the code changing frequently? Are new features needed to be added? All that asks the factor into it. It could be that you have a monolith, it doesn’t need to be changed that often. Maybe it doesn’t. You’re not deploying all that often. It may be best just to leave it alone.

Jon: Absolutely.

Chris: We have these things that still exist. There’s plenty of examples of the equivalent of monoliths that continue to live there. 

Jon: These are major awesome services that we all use that have part of their systems are monolithic.

Chris: All right. I’m thinking even older. Think about all the COBOL code that’s still out there. In the financial sector, in the government sector, there’s still mainframes from the 70s and the 80s running. Probably not adding new features. They’re not using agile development. They’re more maintenance mode.

Just make sure this is the right path for you and your situation. As far as approach, once you decide that you do want to go from monolith to Microservices and you need to do your service decomposition. I can think of two major approaches here.

One is this concept of a brown field approach. Brown field, it’s kind of a newish term. We all heard of green field. Green field’s pretty straightforward where it’s like, “Oh, we’re going to start from scratch. Clean piece of paper.” That’s green field. You don’t have to take into account any legacy issues.

Brown field is when your new software that you’re building has to take into account and coexist with software that’s already in place. I think, actually, the term comes from the architecture building industry where they’re going in. They’re doing remodelling or adding new buildings inside existing developments and whatnot. 

The brown field approach for the monolith is, you basically leave the monolith as it is. You’re not really going to touch it. Instead, you’re going to build new features that you want to add to your system, get built up as many Microservices. They’re kind of vaulted around the perimeter, if you will, the monolith. You can use things like reverseproxies to wrap traffic. The clients still see it as one system. You used those reverseproxies to then route it to whether it goes to the monolith or it goes to one of these new Microservices.

Jon: This, I guess, makes the most sense when the existing monolith is something that really doesn’t need to change very often. It’s pretty stable, it does what it’s supposed to do, everybody’s happy with it, but they want new features. They don’t want lots of changes to existing features which may… Laws changes to existing features and having a hard time having any velocity with that. It’s one of the things that might leave people wanting to implement Microservices. If that’s the problem that you’re facing, maybe brown field is not your solution.

Chris: Yup, exactly. That’s one. Again, if you can keep your monolith as it is. It’s really not changing much to the new stuff, Microservices approach, build around it, and do your proxying. That’s brown field.

The other big approach would be hey, it’s changing. We have to break it up. We’re going to break up this monolith into multiple Microservices because this is important business logic. We want to add new features. There’s some core capabilities inside that monolith. We actually do need to do that. Have it lifted, break it up, and now, splitting off into multiple Microservices. This is definitely the harder problem, I think it’s probably also the more common problem. I think a lot of folks with monoliths out there, they’re being actively developed. 

That would be this next portion. How do you dismantle the monolith? How do you break it up?

Jon: Just before we get there, I want to say that there is a third option, I think that is really important for people to consider. It might not be obvious and it may get a lot of pushback from the business. It’s worth thinking about which is just starting over from nothing. The reason to keep in mind is it’s just business. 

Sometimes, it’s faster to move without the legacy stuff in your way. If there’s valuable data in there in the legacy stuff, you might be able to move that data to some other place where you can still access it, some sort of reporting or analytic system wherein you can still see what people did in the old system but just build something new.

Just an example that comes to my mind, it’s not exactly this, but PayPal did the person to person payment system. It just didn’t get anywhere. Maybe there were engineers working on it, having trouble with it, and doing all kinds of things with it. Then, PayPal bought Venmo which is a person to person payment system. It’s going gangbusters still to this day. PayPal still is solving the same business problem with completely different software. I guess, that’s not the same as prewriting your monolith as Microservices but it is the same as replacing one system with another for the business purpose.

Just keep the business in mind. The business in the software related, every once in a while, I get all started over from scratch, if it makes sense.

Chris: Yeah. It’s analogous to the build versus buy. In that particular case, PayPal bought instead of buying. It’s a good point of is what you have still relevant? How complicated is it? How much cognitive load does it require? If it’s really complicated and there’s a lot of tribal knowledge associated with it, then, yeah, you may think twice about how you’re going to move away from it. Maybe you just spin up a new team with years of requirements and go build something. Cool.

Let’s talk about dismantling the monolith. We decided we want to do this divide and conquer approach. We need to break up our monolith and Microservices. How do we go about doing that? There’s some key challenges here, some key decisions that need to be figured out. That is, you need to decide what you’re going to decouple and when to do that. Then, how are you going to do this in an incremental fashion? 

One of the key things that you need to keep in mind, you want to have a stable system after every change as you go through this. You don’t want to have a plan of return―a lot of risk with that. It may work out but if it doesn’t workout, you may find yourself looking for a job. You want to do this incrementally and minimize risk as much as possible. 

I think, preferentially, you’ll want to start with larger services, groups by business logic domain. You want to minimize the number of new Microservices. Especially if this is a newer operational environment for you going to having many deployable things versus you really don’t have one. You don’t want to go from 1 to 16 right out of the gate. You may decide hey, what are some of the key business domains that this monolith is doing? What are those vertical domains, and break it up by that? Here on this side, making them a bit bigger, then making them a bit smaller. You want to keep those fewer as you work through it and as you become more operational and mature, then you have opportunities to further break those down. You can decide now, “Okay, this is too big. It is requiring too much cloud, it’s more than one team can handle, let’s break it up.”

This is lather, rinse, repeat. You can do staff-wise refinement on it. That’s a suggestion there, start with larger services, and then you can refine that as you mature. Again, definitely make sure you’re taking atomic incremental steps. You want to be working towards a state of, essentially, less entropy. That’s what your goal should be along the way. Every time that you’re doing one of these incremental steps, the system’s in a better state than the previous step. Also, you should be thinking along the possibility that you may not take anymore steps after that. 

Jon: How do I understand that? I read that in Outliner earlier today and I was like, “What do you mean?” Give me an example of an incremental step and then an example of one that might be the last one.

Chris: You think of this as like you look at your existing monolith. There’s probably four different vertical business domains here that we’re going to spin those out. We’re going to do those. Incrementally, there’s some shared data that’s involved here. We’re going to be doing some database migrations and whatnot. You may break that into phases. You’re going to say, first, we’re going to spin out one of those vertical domains and then update clients accordingly and call that phase one. Maybe there’s no state associated with that.

The next phase, we’re going to spin another one. This one does have state. We do need to split its data. There’s data migration involved. Now, we’re going to have two Microservices. What was the monolith is now a smaller monolith. We have two databases. Then, the next phase would be another one. Now, maybe you’re adding some other features that you’ve decided that we need better feasibility into this. There’s some distributed tracing or something or monitoring going on or whatnot.

You’re doing this in phases. The important point is every one of these phases, you have a complete system that’s functioning, that’s in a good state. If you had to put your pencils down now and walk away, just make sure that you can do that. That’s the suggestion here. You never know.

Jon: Bosses are like, “No more splitting stuff into Microservices. You got to do feature work for the next seven months.”

Chris: Yeah, some new priorities come up. Exactly, yes. Again, it just goes to you want to reduce risk, you want to make sure that every time you’re making a change, you’re leaving the system in a better state. You have that plan but a lot of possibility that you may not execute on the entire plan.

Jon: Got it.

Chris: All right, cool.

Jon: Yeah, that’s a good idea.

Chris: Another suggestion for how to do this is definitely you want to minimize the dependencies of your Microservices. These new Microservices that you’re creating, minimize their dependencies back to the monolith. You want to go the other direction. If you’re creating these Microservices that all have the dependencies back to the monolith, you can imagine them all making  API calls back to the monolith in order for them to be updated, they have to wait for updates on the monolith. You’re kind of shooting yourself in the foot. Instead, you want it to be the other way around. You want to make it so you can change your monolith so it can depend on the Microservices. Keep that in mind as you’re doing your design and move into this process. 

Something else that you’re going to be faced with is just determining what you should decouple, what gets pulled out of that monolith into its own Microservices? We’ve touched on this a little bit but remember, the primary reason for moving capabilities out of a monolith is the ability to release that independently because either they’re very important to the business or they need to be changed frequently. That is one of your guide posts for your heuristics. Identifying what is it that’s in that monolith that should be moved out into its own thing.

Jon: I’ve got to apologize, Chris. I kind of lost the thread of what you’re talking about. It’s because I’m still caught up in the thing that we just finished talking about which is not depending on the monolith. I’m going to ask this one question about that and then, I, maybe want you to clarify what you’re just talking about again. I think this is really important about dependencies back on the monolith thing. 

In my mind it was just, what would cause you to do something like that? Then, all of a sudden, I imagined this Microservice that does five things. It has some sort of workflow, that has five different steps in it―go get some files. Then, it goes to the database and get some information. Then, it does something to the files without that information stored in that somewhere. Maybe that four steps.

If each of those four steps requires doing some RBAC checking, some rule based access control and all the rule based access control is in the monolith, that can be ugly. That might be an example of the Microservices going, “Hey monolith, can I do this? Hey monolith, can I do that?” If that were the case, it would sort of suggest what can you do to make sure before any of the Microservices, with all these stuff that it’s going to take, are authorized in advance so it doesn’t call back or maybe you’re going to play or backout as your first Microservice―all kinds of stuff to do that.

Chris: Yeah. Our back system is pretty entwined in your monolith. You know, it’s going to be pretty hairy to lift that out or to reduce the reliance on it. Pick something else that doesn’t use RBAC. 

Jon: Exactly.

Chris: This is really common. You’re always going to be running in this situation. That monolith, it has everything in it. There’s going to be some core things, crosscutting concerns, it’s going to be challenging. It’s going to be an individual thing. You have to decide, “Hey, is one of those things is going to be a target to this own Microservices? Or are there other things that don’t need those common things?” That’s what gets blown off first.

Jon: Right, okay. Back to where you were, catch me up a little bit.

Chris: This is just pretty similar to what we’re just talking about now. You need to decide what is it that you’re going to lift out of that monolith into Microservices. One of these heuristics that you should be using to help you identify what should be pulled out is the code that you want to be able to release independently. You want it to be standalone. That’s because either it needs to be updated frequently or it’s just a really important piece of business logic that’s so important that stand alone makes sense for.

Jon: Needs its own systems running it.

Chris: Right. That’s just something to definitely keep in top of mind when you’re asking yourself, “Okay, you’re looking at your monolith. What should be pulled out? What are the things that I really want to be able to release independently?” That’s going to be a really good way in helping you with that judgment call.

We talked about this before. Vertical domains, business domains, and their data are really good candidates here. Moving away from a shared database is going to be something that’s really important to be thinking about here. We talked about these in the past, in the previous show, that Microservices in general only have one reader writer to a database―one service that does that. It’s an owning service for that particular state. Anyone else that needs access to that stage is going to do it through an interface, through an API call, and not directly touch that database. 

You’re looking for these vertical domains in your monolith that you can split out if they are stateful―if they do have state. Then, you also need to move their data as well. You can’t keep that data together inside that shared database. That’s just going to be the…

Jon: That’s going to be the most painful part of your system.

Chris: Yes, yes. Those are some of the tips and guidelines for how you determine what you should decouple. Then, beyond that, you’re going to think about how do you do this? How do you do them―the migration―as you lift up this code? This is where you can get to decide, it kind of goes back to your comment of, “Are you going to reuse the code or do you rewrite the code?” This is a very real decision that you should take seriously. Don’t fall victim to the sunk cost bias. We’ve gone and we’ve spent money on something. Because we’ve already spent a certain amount of money on it, we’re going to keep spending money on it because we have this implicit bias that says it’s now worth, we don’t want to lose that investment so we’re going to continue putting money into it. 

Same thing with the code. Just because the code has long lived and there’s lots of lines of code, you spent months and months, and years and years working on it. You can’t take that into account on whether or not you should be reusing it. You really need to step back and ask yourself, “If I were to make this investment today, is that the right call?”

Jon: Yeah. That is part of it. I feel like both sides of this are equally bad. Don’t fall victim to the sunk cost bias. Also, don’t listen to the person who’s like, “I can write that in three weeks.” Do not listen to that person either. Real systems take real time to build. Just because they took real time doesn’t mean that you need to save the cost. 

Chris: Yeah. This is where that term pragmatic comes in. You really have to take into account what is the cost of doing this work and really what’s the most pragmatic approach. What’s going to provide the best solution at the appropriate cost? It’s like maybe you can’t afford $1000 solution but maybe you can afford $250 worth. Make sure that the spend is commensurate with the quality or the level of solution that you’re working towards.

Jon: I guess another thing though that I would consider is this, again, it’s sort of stepping outside the development domain and into other job roles, whether or not the system is well understood outside the code, sometimes it’s not. Sometimes the code is the one that understands the system the best. When that happens and you think you can rewrite it, a lot of times, you’re wrong. A lot of times. 

The only thing that understood everything the system did is that literally the lines of code that made the system do what it did. You can’t rely on a product manager or other business users to tell you everything it did. It’s too much work. When that happens, it’s really unfortunate. It’s probably worth your team taking the time to do some sort of discovery but outside of the ability to do that, discovery may just need to keep the code.

Chris: Yeah. That’s unfortunate. It’s a common thing especially for projects that have been in development for some time whether it be months, years, or how many team members come and go and whatnot. When you are in that situation where it’s like, “Man, we’re really not sure how this thing works. We’re really afraid to touch anything.” That’s not a good place to be. You do have to make the decision.

Jon: I get it, yeah. It goes back to our other episode we did on feature flags. Maybe the thing to do there is to start making changes or building that new Microservice with feature flags and have people tell you, “Wait, this thing used to do all these other five things when I used it. Now it only does this one thing.” At least you sort of limited your ideas. Doing things to limit your blast ideas when you’re in that situation is probably more important than just breaking it up into Microservices.

Chris: Yup, absolutely. Take advantage. There are static code analysis tools and even runtime code analysis tools to kind of show. That can help out a lot here. Go back. Can we define what are the requirements here? What does it need to do? What are the interfaces? Techniques like that may help a lot. You’re absolutely right. When we start using whatever’s been rewritten or lifted up, feature flag, yeah, to minimize the risk of doing that, especially if it was something that involved so much treble knowledge and just mystery around it.

Jon: We’ll put a link to that, our feature flag episode, in the show notes. 

Now, the last piece we’re running along here, I’m sorry everyone. I don’t want to save this for next week because it’s so practical. You just have a list of six things here. I think you should essentially read out verbatim, Chris. Then, we’ll also put them in the show notes because it’s literally, “Here’s what you need to do with your database.” Just do these steps and you’ll be all good. Want to go for it?

Chris: Yeah. This is what everyone’s been running to. You have the monolith, there’s that one big database as you breakup your Microservices that each have to have their own backend stores. You’re going to have this migration and how do you do it. It’s a safe and responsible manner where you don’t need to have down time. How do you do this?

It’s not easy. It is doable. The technique here is do it in small incremental updates with very little danger of breaking the system so that you can revert and go back. Here’s a pattern that you can use. Personally, I have used this myself in systems at Kelsus on projects. We’ve done these kind of systems. Other companies have done this system as well. It’s been a transition. This will work.

Step one, you’re going to implement double writing. Double Writing just means you have two sources of the same data. You have the existing system and you have the new database that you want to move to. Whenever you’re doing updates to the data, it’s now going to write to both places. You’re now having two copies of the data. That’s the first step.

The second step would then be to run a migration. You’re going to now migrate all of these same data to new database. You don’t know how much data you have. That could take a long time but we also have the double writing going on. Any updates that are made while migration’s going on are going to ensure that both tables are staying in sync during that migration. Now, we’re now on the system where we have both. We have a complete duplicate copy of the data. One in the existing system, one in the new one. 

Now, the next step―step three―would be we want to start proxying the reads and writes from the existing system to now go to the new one. You’re not updating your calling code necessarily. You’re just now basically changing the implementation if you’re using an ORM or whatever your database layer is. Instead of reading and writing to the original database, your data layer is actually going to go to the new one. 

Jon: That’s the old code doing that.

Chris: Yup. Then, after that, you want to basically stop allowing access to the original database table completely. Now, it’s all coming from the new database.

Jon: That’s just for the tables or data that is part of the Microservice, right?

Chris: Yeah. If you started of with a monolith and a big database with many tables, some Microservices that you spun out, it’s some subset of that. It could be three tables. Sometimes, it could be your breaking up table into maybe more than one table. This same approach works for that. It works really, really well for that. 

Then, after that, the next couple of steps are really just about the cleanup and how you tear down the old stuff. You’re going to stop doing the doublewrite. You want to make sure that your callers are now going through only to be talking to the new data since they’re no longer going to the old code. Then, finally, you’re going to remove those proxies. 

Then, along the way, you definitely want to have things like loggin so that you can make sure you have caught all these colors that were going against the original interfaces. You can also use feature flags here as well to safeguard along the way each one of these different steps. You know that you’re not going to have a big surprise in the next step when you go removing things.

Jon: Very cool. The six steps are going to be hard for listeners in their cars to keep on their mind. Feel free to refer back to the transcription and show notes for those steps. I guess we’ll probably get this exact steps into these show notes at the top in the summary area so that it’s easy to access.

Chris: Yeah, the key thing here is to understand the technique―double writing. You’re going to duplicate the data and gradually you’re switching folks from instead of going to point A for the data, they’re going to go to point B. It’s kind of a railroad switch, almost. You can’t do the switch on the railroad until you’re sure that the data is now in the right place.

Jon: The tracks on both sides of the switch.

Chris: Yeah, exactly. That’s exactly right.

Jon: Great. Well, super fun episode. I better wrap it up quick here. Anything else you want to say before we say goodbye to our listeners?

Chris: No. I hope that was useful and practical because it gives folks those ideas and takes some of the hairiness of going from monolith to Microservices.

Jon: Alright. Thanks a lot, Chris. I’ll talk to you next week.

Chris: All right. Thanks, John. See you.

Jon: Bye.

Chris: Bye.

Rich: Well dear listener, you made it to the end. We appreciate your time and invite you to continue the conversation with us online. This episode, along with show notes and other valuable resources is available at mobycast.fm/69. If you have any questions or additional insights, we encourage you to leave us a comment there. Thank you and we’ll see you again next week.