Joel Kaasinen
Technology

Packaging Clojure for Production

Update 2022-05-04: rewrote parts contrasting shadow-cljs and Figwheel Main.

IntroductionLink to Introduction

This post is a look at the various ways of packaging a full-stack Clojure web application for production.

A full-stack Clojure web application has a backend (that is, a HTTP API) written in Clojure, and a frontend (that is, something that runs in your browser and communicates with the backend) written in Clojurescript.

Many posts & templates (on Learn ClojureScript, on this blog and on other blogs) cover setting up a new full-stack Clojure project for local development. There is less talk about how to run one in production. General guidelines are set by classic sources like The 12-Factor App, but more concrete advice is needed as well.

Back in the day the only option for full-stack tooling was Leiningen plus lein-figwheel. These days one can choose between shadow-cljs and Figwheel Main for frontend tooling and between Leiningen and tools.deps aka deps.edn aka Clojure CLI for a build system. Also, for some people, the world has changed from running uberjars on virtual machines to running docker containers on cloud providers.

This post comes with a companion repository on GitHub that contains working examples and documentation for them.

Big Question: Running The BackendLink to Big Question: Running The Backend

The main question you need to answer is:

How do you want to run your backend?

We'll cover three answers to this question: Uberjars, Docker and Git Checkouts. We won't be covering serverless/FaaS backends, which are an interesting topic of their own.

UberjarLink to Uberjar

An Uberjar (also sometimes called a Fat Jar) is a single file that contains a JVM application and all of its dependencies. Uberjars have been the standard way of deploying JVM applications for ages.

To run an Uberjar in production, you need some additional infrastructure for handling starting, restarting, monitoring and logging. Systemd is a good choice for these on Linux, but we won't cover it here.

Pros:

  • A single self-contained runnable file
  • Battle-tested standard for running JVM apps
  • Can be run anywhere with Java

Cons:

  • Need to set up a Linux Virtual Machine

DockerLink to Docker

(I'll just assume you know what Docker is. If you don't, perhaps Google it.)

If you squint a bit, a Docker container is the same thing as an Uberjar: a single artifact that contains an application and all of its runtime dependencies. However unlike Uberjars, Docker containers aren't restricted to the JVM, but can contain arbitrary native code.

Just like with Uberjars, you probably want some additional infrastructure to run your Docker containers, instead of just using docker run manually. There are lots of options for this like self-hosted Kubernetes, GCP on Google Cloud, EKS on AWS, etc. We won't go into these here.

We'll cover two different ways of packaging Clojure applications as Docker containers: wrapping an Uberjar into a Docker container image, and constructing a container image directly.

Pros:

  • Can leverage existing Docker infrastructure (e.g. existing Kubernetes cluster)
  • Language-agnostic: good for multi-language environments

Cons:

  • Need to set up Docker infrastructure
  • Can be harder to debug/profile

Git CheckoutLink to Git Checkout

The most straightforward way of running your application in production is just using a git repository like you would for development. This can be a great way to get started running a prototype, but might not be the best for large-scale use.

Pros:

  • Simple to set up
  • Can hack on code directly in production

Cons:

  • Not as robust
  • More work to keep multiple deployments in sync
  • No single deployable artifact
  • Can hack on code directly in production

Big Question: Serving The FrontendLink to Big Question: Serving The Frontend

A secondary question you need to consider is:

How do you want to serve your frontend? (That is, the .html and .js files.)

What matters here is whether you choose to serve your frontend from the backend, or externally

From The BackendLink to From The Backend

Serving the frontend files from the backend simplifies deployment: you only need to deploy the backend, and the frontend comes for free. In practice this means you need to bundle your index.html and myapp.js files into the backend, and serve them along the backend HTTP API.

Pros:

  • Backend&frontend versions stay in sync
  • Easy deployments

Cons:

  • More complex build step

ExternallyLink to Externally

Instead of bundling your frontend with your backend, you can have a separate solution for serving the frontend files, e.g. a traditional web server like Nginx or a CDN like Cloudflare. This effectively separates your frontend and backend builds and deployments, and might suit projects that have the frontend and backend in separate repositories. However, this might cause a bit of friction with a Clojure/Clojurescript full-stack app that wants to share code between the frontend and backend.

Pros:

  • Simpler build
  • Can update backend & frontend separately
  • CDNs can speed up delivering the frontend

Cons:

  • Potentially makes frontend/backend code sharing more difficult
  • Need to configure/set up the web server or CDN

Small Question: Which Dependency Resolver?Link to Small Question: Which Dependency Resolver?

Before we dive into examples, we need to cover two smaller questions about tooling.

To be able to run or build your web application, you need to fetch all the libraries you're using. The two main solutions for this for Clojure are the traditional Leiningen and the newer deps.edn aka tools.deps aka Clojure CLI.

The main difference between these is that Leiningen does lots of other things, while deps.edn is focused on just fetching the dependencies. While you can use Clojure CLI as a method for running various tools like test runners and uberjar packaging, these things are more tightly built into Leiningen.

My personal opinion is that simple project setups are better off with Leiningen, while more complex ones (a monorepo, multiple microservice, etc.) might benefit from the flexibility of deps.edn. However be prepared for some sharp edges and having to write custom code when dealing with deps.edn.

Small Question: Which Clojurescript Compiler?Link to Small Question: Which Clojurescript Compiler?

Another piece of the full-stack puzzle is compiling your Clojurescript source files into javascript that can be sent to the browser. This is further complicated by the different needs of local development and production. When developing the app on your laptop, you probably want a watcher that recompiles changed .cljs files and hot reloads the changes into your browser so that you can see your changes immediately. For production, you probably want a single as-small-as-possible .js file that can be sent to the browser once and then cached.

Just like with dependency resolvers, you have a couple of options to pick from.

Figwheel (or rather, the tool now called lein-figwheel) was the pioneer of hot code reloading. Shadow-cljs showed up a bit later as the pioneer of using npm deps. Figwheel Main followed and is on par with shadow-cljs features these days.

Both Figwheel and shadow-cljs are compatible with Leiningen and deps.edn, so you can mix and match as you will. For advanced use there are some differences, but they won't matter for this post. Both tools are improving all the time, so check the latest guides when deciding which to use.

We mainly use shadow-cljs at Metosin.

Worked examplesLink to Worked examples

Now that we've talked about what your options are, here are some examples of how the pieces go together.

Uberjar, Leiningen, FigwheelLink to Uberjar, Leiningen, Figwheel

The lein uberjar command is a tried-and-true way of building uberjars. You need an :uberjar profile in your project.clj, and you're good to go.

The only possibly tricky bit is including your built frontend code in the backend if you chose to serve your frontend from the backend. This is usually best accomplished using leiningen :prep-tasks.

You can find an example using Leiningen & Figwheel-main in the lein/ directory in the companion repo.

Check the README and the comments in the project.clj file for more information.

Uberjar, deps.edn, shadow-cljsLink to Uberjar, deps.edn, shadow-cljs

There are a number of ways to build uberjars for deps.edn projects. These include

You can find an example using tools.build and shadow-cljs in the deps-uberjar/ directory in the companion repo.

Again, check the README and the comments in the files for more info.

Docker ContainersLink to Docker Containers

If you're running a Docker container, you have one more decision to make:

Will your docker container contain an uberjar?

Some arguments for including an uberjar:

  • if you already have a working uberjar build, it's easier
  • your Dockerfile is simpler
  • you can use a very simple java base image

Some arguments against including an uberjar:

  • can't take advantage of docker layer caching
  • need a working uberjar build
  • more complex images needed, especially if you build your frontend inside your Dockerfile

With an UberjarLink to With an Uberjar

A Dockerfile that wraps an uberjar is pretty simple:

FROM openjdk:17

RUN mkdir /app
WORKDIR /app
COPY path/to/project.jar /app

ENTRYPOINT ["java", "-jar", "/app/project.jar"]

Both the lein/ directory and the deps-uberjar/ directory in the companion repo include Dockerfiles like this.

If you want, you can also build your uberjar inside your Dockerfile. Just add a layer that runs lein uberjar, or use a multi-stage Dockerfile for smaller output images. Depending on your setup, moving the uberjar building inside the Dockerfile might not be worth the trouble though. Uberjar building is very reproducible already on its own.

Without an UberjarLink to Without an Uberjar

A Dockerfile without an uberjar looks something like the following. Deps are downloaded in a separate RUN command to make use of layer caching. We're using deps.edn here but similar things are possible using Leiningen.

# Base image that includes the Clojure CLI tools
FROM clojure:openjdk-17-tools-deps-buster

RUN mkdir -p /app
WORKDIR /app

# Prepare deps
COPY deps.edn /app
RUN clojure -P

# Add sources
COPY . /app

CMD clojure -M -m my-project.main

More complexity is added by building the frontend. You can find full examples of both backend-only and backend-plus-frontend Dockerfiles in the deps-docker/ directory of the companion repo.

Running from a Git CheckoutLink to Running from a Git Checkout

Running from a git checkout is certainly simple. You probably won't need anything outside your normal dev setup. Here's an example run.sh that fetches the latest master, builds the frontend, and runs your project using clojure CLI.

#!/bin/bash
git fetch
git reset --hard origin/master
shadow-cljs release app
clojure -M -m my-project.main

For leiningen, the equivalent would be something like:

#!/bin/bash
git fetch
git reset --hard origin/master
lein cljsbuild once min
lein run

Other ConcernsLink to Other Concerns

It's almost time for a summary, but here are some additional concerns that this post doesn't really address.

  • Including git revision info in build artefacts
  • Deploying
  • Configuration

A significant topic we do need to talk about is Ahead-of-Time Compilation.

AOT (Ahead-of-Time) CompilationLink to AOT (Ahead-of-Time) Compilation

Clojure sources can be compiled into Java class files and these class files can be bundled into the uberjar instead of clojure sources. This allows for slightly faster startup time, or might be needed for certain kinds of Java interop where you need to generate an actual Java class from clojure.

The Clojure reference, the Leiningen FAQ and the tools.build guide have good material on AOT compilation.

However, AOT is not necessary for packaging a clojure app, so we've left it out to simplify the examples. We recommend not doing AOT unless you absolutely have to: it keeps everything simpler at the cost of a slightly longer startup time.

One minor advantage to AOT compilation is that you can specify a custom main class for your uberjar. Then you can run your app with

java -jar my-project.jar

instead of this more verbose command our examples use:

java -cp my-project.jar clojure.main -m my-project.main

It's up to you if this is worth the hassle of AOT compilation though.

PostscriptLink to Postscript

I hope this guide helps you pick a method of building & deploying your full-stack Clojure app that fits your organization, team and process. There are no right answers, only a (sometimes bewildering) amount of different options.

Joel Kaasinen

Related posts

More

Contact