Macro-ifying Microservices: A Way to Think about Microservices

What now seems like ages ago, microservices blew up as an architecture model. It was probably Martin Fowler who put everything that was being discussed before into one single article. And I think that's when the information became accessible and easier to spread. Nowadays, everybody uses them, extensively, but I feel like at some point along the way we stopped thinking about why we use them, and just that we needed them, and they had to be small. But in the places that use microservices most effectively in my experience, like Amazon, Google, and Meta, people rarely talk to them as microservices. They just are.

As such I've seen a lot of places where microservices were created for the sake of it. Places where an application or service would grow a little, and suddenly it would get split up in order to abide by the microservice ethos. These are situations which I personally call putting the solution before the problem. The mistake I think that causes this solution to pop into people's heads, is the name itself. Micro in itself means small. So we couldn't possibly have a big microservice, surely?

What are microservices?

Technically, microservices are an architectural pattern where an application is designed as a collection of smaller services that are interconnected through some type of protocols. They tend to be self-contained, and individually managed. Wikipedia does a great job of explaining the basics.

Conceptually, my favourite way to think of a microservice is a function. What makes a microservice micro is that takes one type of input and returns one type of output. As such I like to say that my favourite microservice is Meta's web-service. There's little specific information about its size publicly, and it changes all the time. But people who've worked in software for a while know it as one gigantic monolith that tackles a very large part of what Facebook does, and in some part the other parts of the other Meta companies. The reason it's a microservice is that what it does is it takes an HTTP request and returns an HTTP response. From a function perspective it is actually very simple.

Now imagine you had two functions that had the same input and the same output. It would take a bit of decision-making to figure out which one to use. The extreme example that I like to use of how that can become a challenge, is one particular case where a company had this relatively big website where every different page in its menu was a different microservice.

Why do people use microservices?

Microservices come with a lot of benefits. And I feel that even when people do not refer to their microservices as microservices, the concept is simple and base in our understanding of software that is ever-present. When it first started being discussed it was an architectural decision of itself. Nowadays, it is less of a question of whether we have microservices, just how we break them down. When it's a small codebase, there is no need to think about them, because the benefits they offer are covered by the single service/application. But nowadays, software has grown bigger, and they are inescapable.

I like to talk about them from the benefits they offer.

1. Compatramentalisation: This is by far the biggest benefit of microservices, and one can argue every other single benefit that they offer is a result of this. But talking about them from Amazon's strategy for microservices, we have roughly one microservice - one team. If one team only focuses on one service, they can be experts on what that service does, how to maintain it, and how to fix it. They become the best people to think about how to improve it and how to develop it. And most importantly they don't need to worry about the other services. They assume they work as advertised. This is a beauty in people management more so than software management. It reduces stress and increases focus because all you need to focus on is the problem space your team deals with. And you may ask: Wouldn't it be weird to shoehorn all of one team's responsibilities into one microservice? Well, technically the teams are split like a microservice and that is why it works. They worry about their part of the problem, and only their part of the problem.
2. Plug and play: microservices work in a somewhat similar way to SaaS products where if something fulfills the same purpose, in theory (this is important), one can just swap different services for their new more improved alternatives. Ultimately if we think of a microservice as a thing that does one thing alone, and the mathematical way of portraying that is a function that should in theory be easy. This plays into rewriting certain services and deprecating them (but that is a completely different story for a different time)
3. Focus: similar to compartamentalisation, having one service serve one purpose and one purpose alone makes it much easier to reason about it. This helps direct one's focus to solving a smaller set of problems thus making things more solvable. Of course, this requires some overarching architect to identify the way to split the problems in such a way that people can actually focus on their piece of the pie, and not have to worry about other parts in order to achieve the overall goal.

What are the challenges with microservices?

Like with any architecture model, there is no one ring to rule them all. Ultimately it is just that a model. If someone is telling you microservices will solve all your engineering problems, they're probably selling you something. Ultimately, it is just a term of software architecture, and like any architecture it also has its pitfalls. Here are the ones I've seen most commonly, even in well architectured microservices.

1. Push Safety: this is what I would call the greatest gotcha of microservice design. In and of itself not the scariest thing. The solution technically is to just write good push safe code, right? Well, yes. But at times that sounds a lot easier than it is. If your code is being used (and why would you have it if it wasn't), the people using it will have certain expectations on how that code is used. Sometimes you have control over the code that uses your code too, so when you need to change something you can change that to. And then is when you need to be very aware about the deployment schedule of each interactive piece. The better designed the microservices the easiest it is to argue about that push safety. In a poorly designed system of microservices, the push safety can grind the development to a halt.
2. Maintenance: as much as we try and automate away releases, capacity management and everything related to our services, there is still overhead that comes with every service a team maintains. Each individual service needs individual monitoring for health, each service requires individual capacity planning. The more microservices there are, the more people need to understand and handle DevOps roles to maintain these microservices, and while these are important for everyone to know, it is expensive for both time and focus.
3. Code Duplication and Discoverability: Microservices should all do different things and those should never overlap. In theory. Well, in practice... it's not as simple. Even if we do extract the common code out into libraries, the versioning often needs to be done manually for all affected microservices (more about why this is even trickier in the next bullet point). Sometimes the easiest way is to copy and paste the same code into a few different microservices, because they all need to do this one particular thing. On a small scale, it's ok. As the architecture grows bigger, it turns trickier. With monorepos we make discoverability a bit simpler, but ultimately when we make a change, we'll need to find all the places, and sometimes, these places fall through the cracks.
4. Staleness: theoretically microservices do one job, and they do it well. So the perfect microservice is written once and then never needs to be touched again as it's running, right? Well, wrong. While the service itself does continue running, software has this terrible habit of changing all the time. It is incredibly common to fall into this pitfall where there's nothing that needs to be done, because everything functions perfectly, until it does, and then we realise that we can no longer compile our microservice. A very common case when this happens is security vulnerabilities. Where a simple version change requires hours of getting the code to compile again and deploy. And sometimes the only person who knew the code already left the company.

When do I add a new microservice?

Never if you can help it. If you do, think first if there's already a microservice that solves your problem. Even partially. Can you make that one work for you?

If this sounds restrictive I'll use an analogy. Your team uses Asana for task tracking, and you are frustrated that there is no ability to do time tracking in Asana. Now you really want time tracking for your team. One solution you have noticed is that monday.com has time tracking. So you try and duplicate the tasks into monday.com so as to have time tracking to them. But people are still used to using Asana as the main task tracker. Leading to a lot of confusion. Monday.com is that new microservice you want to build.

Now if your team does not have a task tracker at all, introducing a new task tracker to your team is not as controversial and difficult action. But when you do create it, do your research. What do your team use for tracking tasks? What is important to do so? And then build a new microservice will all the information. Or maybe, just maybe you find out there was a completely different team far away in the opposite side of the company that had this problem, and solved it.

But isn't this a problem with any new solution? Yes, it is. It is a problem with new tools, new anything people want to build. It's just that microservices make it so much more inviting.

When should I split my microservice?

In my entire engineering career I have only ever seen one case where a microservice was in need of splitting. That has probably less to do with services not needing splitting and more to do with the observation that often when microservices are in use people usually over-split than under-split.

This service was a Frankenstein's monster of different bits and pieces that were being called in increasingly inventive ways. It probably didn't help that somebody had started splitting the service and then stopped before they could finish.

Why are you trying to convince me to unite my microservices into one monoservice?

Now I don't want you to get the wrong idea here - I'm not trying to convince you to smash all your carefully split microservices back into one giant monolith. But I am wondering if some of these challenges brought up memories of headaches you've had wrangling your own microservices. Did you nod along to the maintenance troubles or discoverability issues because you've battled them yourself? Or maybe as you read about the common pitfalls, you started thinking about cases where you work that could actually be simplified by consolidating certain services. If so, you're not alone! The goal here isn't to undo all microservice architectures, but to rethink how we approach dividing them in the first place. It's about zooming out from solutions to problems and thinking how we can split our problem spaces in a way that makes it easiest to maintain and develop in the future.

Conclusion

Microservices were a brilliant idea that changed how we do software engineering. But like any great ideas, I've seen it morph as it traveled by word of mouth. When well designed, while microservices still come with challenges of their own, they offer pros that make both people and software management a lot easier, cleaner and safer. But when we loose track of the reasons behind the architecture, it's easy to fall into pitfalls, and then their challenges grow into problems the engineers have to fight every day.