My First Project at Metosin!

IntroductionLink to Introduction

I started working at Metosin at the beginning of 2020 (read my previous blog post about that: My First Weeks at Metosin!). I have now been working at Metosin for about 1,5 years, and I just ended my first project. The first project at Metosin was really exciting. My responsibilities in the project were quite broad - I had a chance to do cloud infrastructure, DevOps, and full-stack Clojure/script development. In this blog post, I describe my experiences - I hope you find this story interesting!

ProjectLink to Project

The project was really interesting. It was a public sector project and also pretty important for the Finnish society. These kinds of projects are always nice to work with since you feel that you are serving the general public. The project was quite big. There was an infrastructure team, three development teams, a test team, a UX team, etc. The general feeling in the project was really nice. Everyone was kind and willing to help each other. After the project, I was happy to hear that the project director had sent a review regarding my work to the Metosin CEO. In his email, he had said that "... Kari is a team player and an ambassador of good feeling." It was easy to be happy working in the project since all my colleagues were really kind and helpful, and there were a lot of interesting technologies to learn and use in that project.

TechnologiesLink to Technologies

There were a lot of interesting technologies in the project. I spent the first six months in the infra team developing the AWS infrastructure and the next 12 months in the Clojure dev team. I describe in later chapters the technologies used in the project in more detail, but let's list the main technologies here first:

• AWS: The cloud platform.
• Pulumi: The infrastructure as code tool.
• Docker: The container runtime.
• Kubernetes: The container orchestration runtime.
• Helm: The package manager for Kubernetes.
• Drone: Continuous Integration tool.
• Clojure: The Clojure programming language.
• ClojureScript: Clojure on the browser (transpiled into Javascript).
• Reagent: Minimalistic React for ClojureScript.
• Re-frame: A web app framework built around Reagent.

InfrastructureLink to Infrastructure

I started the project in the infrastructure team. At the beginning of the project, we considered various options for developing the infrastructure in the AWS platform. The candidates were Terraform and Pulumi. After some POCs, we decided to use Pulumi. I have written a blog post regarding those tools; you might want to read it here: Terraform vs. Pulumi Experiences.

Pulumi was pretty ok to work with. I prefer Terraform myself, but our partner wanted to use Pulumi, and our partner will, later on, take care of the maintenance of the system, so we decided that we are going to use Pulumi as our infrastructure as code tool.

AWS was a requirement in the project. There was a mutual understanding that we are going to use Docker as a deployment unit, but we had some discussions about whether to use ECS or EKS as the Docker container orchestration environment. Initially, I thought it would be simpler to work with ECS if only a limited number of applications were to support. After considering the cons and pros of ECS vs. EKS, the consensus was to use EKS because our partner thought that they are going to use Kubernetes more in the future also in other projects. Therefore it might be better to choose EKS as the Docker container orchestration environment to harmonize the company environments.

Kubernetes turned out later on to be a bit challenging to work with (especially with EKS), but we finally created a nice overall infrastructure for our applications. Using Kubernetes namespaces, we isolated various environments inside the same Kubernetes cluster. We used Helm as a Kubernetes package manager.

We also used some other AWS services: S3 to store various official resolution documents, RDS as a relational database as a service, AWS Secrets Manager to store system secrets, to name a few.

Continuous IntegrationLink to Continuous Integration

While working in the infrastructure team, we also implemented the continuous integration environment for the project. We assessed various options and finally decided to use Drone. Drone is a container-based CI tool: you implement your build pipelines using various off-the-shelf and custom containers. Working with Drone was really interesting and straightforward. You just choose the Docker image for the task and then write the task using bash.

I have written a separate blog post regarding my experiences using Drone; you might want to read it here: Using Drone CI.

Clojure DevelopmentLink to Clojure Development

After the infrastructure was mostly up and running for development purposes, I moved to another team. I joined Metosin since I wanted to work with the Metosin top-class clojurians, and therefore I had asked to join the Clojure development team. In the Clojure development team, we implemented three applications using Clojure. Clojure is a superb language for implementing data-oriented full-stack applications. The main Clojure application comprised complex business logic, and the Metosin Clojure gurus (Tommi and Juho) had implemented the most challenging parts of the application (e.g., the process engine). It was really a joy to study the Clojure application and its tests and learn from the best. It was also pretty easy to continue the application development since the base work was rock solid, and it was easy to implement, e.g., new processes and hook them to the system. To support the main application and other teams' applications, we also implemented two auxiliary applications using Clojure.

Programming Clojure was so much fun! I first worked with the two backend auxiliary Clojure applications. The applications used Integrant as a state management tool, and I was soon hooked with the ability to reset my Clojure application state with one hotkey. You may want to read more about Integrant in my blog post Clojure Integrant Exercise.

After a few months, I started to work with the main Clojure application which was a full-stack Clojure/script app. The frontend used Reagent to provide a ClojureScript interface to React, and re-frame as a ClojureScript framework for building user interface - re-frame uses Reagent under the hood. I had very limited Clojurescript / frontend development experience before this project, but I must say that the Metosin support for the work was excellent: The Reagent leading developer (Deraen) works at Metosin! Juho was really kind and always willing to help in the Metosin slack if I had any Clojurescript related issues. Working with the Clojure full-stack app was a lottery win for me: I had a chance to get good Clojure/script experience - the very reason I chose Metosin a couple of years ago. I also used re-frame in one of my personal projects and wrote a blog post about it; you might want to read it as well: Clojure Re-Frame Exercise.

MentoringLink to Mentoring

I had unbelievable luck with the project. One of the world's top-class Clojure programmers - ikitommi - worked in the same Clojure team with me. I joined Metosin to learn Clojure power programming and who reviewed my every PR: one of the best Clojure programmers on the planet! Gosh! I should have paid these guys instead of getting a good salary into my bank account every month! I believe I turned out to be a much better Clojure programmer in this project. With the support of this level, the learning is really fast. I was pretty soon quite productive in the project. And it really is a superb feeling to be productive and enjoy programming. I learned quite a few Clojure productivity tricks - and naturally, I wrote blog posts regarding them:

• Use Clojure with Docker and Kubernetes! - A superb trick I learned from Tommi. Put a nRepl server inside the app, and you are able to connect to the application running in a Docker container either in your own Docker runtime or in the Kubernetes cluster. Change code, and evaluate it in the REPL and check in real-time how your app behavior changes inside a Kubernetes cluster. How cool is that!
• Clojure Power Tools Part 1 - Use various Clojure power tools like hashp, portal, and the Metosin reagent-dev-tools. All these tricks were provided to me by the Metosin Clojure gurus in various discussions or in Slack.
• Clojure Power Tools Part 2 - More Clojure power tools! Poor man's debug and a Fredericks’s debug-repl.

Programming Clojure using the REPL and these power tricks was so productive and fun. You can just reset the Integrant state with one hotkey, and you can see the new behavior in the app in the frontend in any window in the business scenario you are developing. We also implemented various fixtures that were available in the development version frontend: you can just click some fixture, and it ran through a complex business scenario arriving into the state you needed for implementing that functionality. There were also fixtures for selecting various user roles to develop and test the app's security features, etc. Gosh, I learned so much in this project!

ConclusionsLink to Conclusions

When I reflect on my experiences in that project, I realize that during the project I wrote several blog posts regarding the technologies used in the project. The blog posts that I write are a pretty good personal indicator for me whether I am excited regarding a project and the technologies used in the project or not. Considering how much I wrote blog posts during that project, I must say that this was a really good project: a lot of interesting technologies and possibilities to learn. And Metosin is just a great place to learn from the best: the programmers who implement top-notch Clojure libraries work with you in the same project!

There are many interesting Clojure & cloud projects at Metosin. If you are interested, I strongly recommend contacting Eeva Lennon, and read our Careers page. Maybe you will be our next metosinian working in our next cool Clojure & cloud project with Metosin’s top-class clojurians and infrastructure specialists! (BTW. I just started working on a new cool Clojure & cloud project - building stuff using AWS IoT, etc.)