Microservices with Kubernetes

Short Summary

This was a three days training where Murat Karakaş explained;

  • what Microservices are,
  • what Spring Boot is,
  • what Kubernetes is,
  • how they combine,
  • what best practices are with them.

Long Summary

I took some notes meanwhile and I’m going to research the topics, provide notes, links to understand the training completely. I noticed that there are many new technologies out there and I believe it’s for my benefit to research these technologies and have it written here.

We can think this as a notebook for the training.

Questions

1 -) Are microservices always a viable, feasible solution?

https://www.freecodecamp.org/news/monolith-vs-microservices-which-architecture-is-right-for-your-team-bb840319d531/

https://dev.to/alex_barashkov/microservices-vs-monolith-architecture-4l1m

https://robertnorthard.com/devops-days-well-architected-monoliths-are-okay/

TLDR:

Well architected monolith is better than a hodgepodge of duct tape.

There is no silver bullet. Although generally microservices provide better functionality, monoliths are not dead and there are cases where it’s best to use monoliths. It’s also easier to transform monoliths to microservices but not the other way around.

Highlights:

When To Start With A Monolith

  • Your Team Is At Founding Stage: Your team is small, between 2–5 members, and is thus unable to tackle a broader and high-overhead microservices architecture.
  • You’re Building An Unproven Product or Proof of Concept: Are you building an unproven product in the market? If it’s a new idea, it’s likely going to pivot and evolve over time, so a monolith is ideal to allow for rapid product iteration. Same applies to a proof of concept where your goal is just to learn as much as possible as quickly as possible, even if you end up throwing it away.
  • You Have No Microservices Experience: If your team has no prior experience with microservices, unless you can justify taking the risk of learning “on the fly” at such an early stage, it’s likely another sign you should stick to a monolith to start.
  • You are required to build MVP ASAP: Sometimes you are required to build your minimum viable product as soon as possible. Since there are less operational overhead, monoliths tend to be built faster.
  • You don’t have the budget: When you didn’t get millions in investments to hire DevOps or spend extra time on complex architecture.
  • You don’t have a hot service/bottleneck: When you have a hot service, you will want to isolate and scale them without touching other services. If you don’t have any bottleneck services, there is almost no point to separate them because they go along with other parts.

When To Start With Microservices

  • You Need Quick, Independent Service Delivery: Microservices allow for fast, independent delivery of individual parts within a larger, integrated system. Note that, depending on your team size, it can take time to see service delivery gains versus starting with monolith.
  • A Piece of Your Platform Needs to Be Extremely Efficient: If your business is doing intensive processing of petabytes of log volume, you’ll likely want to build that service out in a very efficient language (i.e. C++) while your user dashboard may be built in Ruby on Rails.
  • You Plan To Grow Your Team: Starting with microservices gets your team used to developing in separate small services from the beginning. And having teams separated by service boundaries makes it much easier to scale up your team when you need to without introducing exponential complexity.
  • You don’t have a tight deadline: Microservices require you to research and architecture planning to ensure it works.
  • You have a team with knowledge of different languages: By knowing different languages’ capabilities, you can use them for specific domains. This way, you can have benefits of different languages.
  • You worry a lot about the scalability and reliability of your product: When using microservices, you can configure each service’s scaling mechanism by their own requirements.
  • You have an existing monolith app and see problems with parts of your application: Maybe the problems can be solved by separating monolith to microservices.

 

2 -) What is 12-Factor App and is it still relevant?

https://12factor.net/

TLDR:

It is a manifest which defines 12 main factors to build an ideal application. And yes, it is still relevant (Check the video above).

Highlights:

The twelve-factor app is a methodology for building software-as-a-service apps that:

  • Use declarative formats for setup automation, to minimize time and cost for new developers joining the project;
  • Have a clean contract with the underlying operating system, offering maximum portability between execution environments;
  • Are suitable for deployment on modern cloud platforms, obviating the need for servers and systems administration;
  • Minimize divergence between development and production, enabling continuous deployment for maximum agility;
  • And can scale up without significant changes to tooling, architecture, or development practices.

 

3 -) What is GitOps?

Continue reading

Snapshot Restore versus pg_restore – Time Comparison

Summary

I’ll just cut to the chase here and give detailed analysis immediately after.

The time difference between pg_restoring an 11GB sized PostgreSQL database with 159MB pg_dump file, versus restoring snapshot of its t2.medium RDS Instance:

time_table.PNG

Analysis

Important note! The time values are subject to change, because it depends on network, other running processes, intermediary EC2 Instance type etc.

pg_restore

I used a t2.medium SSM Bastion Instance to execute PostgreSQL commands. As I’ve mentioned before, provisioning a more powerful EC2 Instance might increase the performance drastically. You can find information about SSM Bastions in this page.

Imagine you have a PostgreSQL database named “XXX” with 11 GB size. Don’t get distracted with the name, I don’t know why I named it “XXX” at the first hand, but here we go anyway 🙂

Continue reading

Session Manager – Modern Bastion

Overview

Are you tired of dealing with key pairs? Maybe tired of securing them with KeyPass? Or configuring SSH daemon for ease of use? Don’t worry, Systems Manager – Session Manager is here to save you from all these.

Session Manager is announced on 11 September 2018 with this blog post. Seriously, I remember being excited like potatoes thrown into hot oil. Because the process for connecting RDS Instances via Bastion Hosts were literal pain in our workplace. All those SSH private keys, security of them, jumping one from another for again, security reasons… All I can say is, it was taking long.

At this point, you might say, “Why aren’t you using a third party UI such as Bastillion EC2 for connecting Instances?” That’s because there are intermediate EC2 Instances  involved with jump process. Complexity is already high, no need to increase it.

Now, you might also say, “But you know you can use a tool to add people’s IP to the security group of the Instances when they try to connect? Then let’s say, after 12 hours, you might remove the IPs from the security group.” For curious cats, it is explained in this Medium Story. That’s an option for sure, but I don’t like the idea of changing the state of a resource that much. Needlessly to say, we want immutable infrastructure in the end, right?

Continue reading

ECS Cluster Hibernation-Scheduled Stop/Start

But Why?

Intensely utilized ECS Clusters can cost lots of Benjamins. Needless to say, there is no way around to lower the costs in Production environments besides provisioning appropriate type of instances for the tasks, coding efficiently, architecting a well-planned infrastructure and so on since the clusters must be running all the times. But of course an Infrastructure Engineer can take actions on Development environments to lower the costs.

I’m shutting down all clusters on Development environment between 23:00-07:00 in my way by a Python Lambda script which gets deployed by Terraform. I stop clusters by setting their Auto Scaling Group to 0. This makes all Container Instances to shut down. But what about the inital ASG states? Where do minimum, maximum and desired values go? I write them to a DynamoDB Table before setting them to 0.

I start the clusters by reading and setting the initial values for Cluster specific ASG from that DynamoDB Table.

For the schedule, I’m using CW Event Rules to trigger the Lambda script.

Let’s see the scripts!

Continue reading