Juho Teperi
Technology

Using Shadow-cljs with ESBuild

If you are using UIx or other modern React JS wrappers with ClojureScript, you might be looking to leverage useful JS libraries like TanStack Query. In some cases, you'll be hit by Closure errors when attempting to use these JS libraries.

More and more JS libs are being released to npm with ES6+ code without transpilation. Publishing code like this can be more efficient as library users can choose what browser features they presume to be available and what features to transpile depending on their target browsers. Closure also supports this with :output-feature-set option

Unfortunately for Cljs users, Google Closure can be somewhat slow to support some new ES features, which can be required even to parse the library JS code. In the case of TanStack Query, Closure is missing support for public and static class fields. (Some recent work has been related to this feature.)

The error when Closure can't read a problematic JS file looks something like this:

Errors encountered while trying to parse file
  .../shadow-cljs-esbuild/node_modules/@tanstack/query-core/build/modern/focusManager.cjs
  {:line 30, :column 2, :message "'}' expected"}

And the relevant JS code, where we can see the static class fields denoted by # prefix:

var FocusManager = class extends import_subscribable.Subscribable {
  #focused;
  #cleanup;
  #setup;
  constructor() {

If you are using Shadow-cljs, you can choose to use another tool to bundle your JS libs for your app. In this setup, Shadow-cljs will create an entry JS file with require-forms for all the JS files used in your project, and you will use some tool to bundle those into one JS file with transpiled code for use in browsers.

With some configuration, you can make this setup nearly transparent by making Shadow-cljs run the tool for you during both release and dev builds, so developers still only need to run one command.

JS build toolsLink to JS build tools

You could use the old and well-known Webpack. ESBuild is another extremely fast alternative. Using it instead of Closure might reduce your release build times by some amount. In our case, ESbuild can process the JS libs about 15 seconds faster than Closure, but this project uses a lot of JS libraries so the effect might be smaller in other projects.

It might also seem likely that Closure could optimize the JS lib code better due to also knowing about your Cljs code and what calls you are making to the JS libs, but in our case, the total artifact size decreased by a bit by using ESBuild compared to Closure. Some of this could be due to chosen target options: maybe ESBuild is transpiling less ES features to support older browsers, so it will output less JS code.

Shadow-cljs integrationLink to Shadow-cljs integration

In shadow-cljs.edn you need to enable the external JS-provider option and setup path where the entry file is written for ESBuild and setup the build hooks to start the ESBuild processes from within the Shadow CLJS build:

 :app {:target :browser
       :js-options {:js-provider :external
                    :external-index "target/gen/libs.js"
                    :external-index-format :esm}
       :release {:build-hooks [(dev.build/run-cmd-flush {:cmd ["yarn" "build:libs"]})]}
       :dev {:build-hooks [(dev.build/run-cmd-configure {:cmd ["yarn" "start:libs"]})]}}

NOTE: You could also replace the build hooks with regular Clojure code which starts both the ESBuild process and Shadow process: https://shadow-cljs.github.io/docs/UsersGuide.html#clj-run. This could be simpler as we don't need to start the processes in any specific stage of the shadow-cljs compilation pipeline (which is the intended use of build hooks.)

In package.json you can setup the two commands to run ESBuild relevant configuration:

 "start:libs": "mkdir -p target/gen && touch target/gen/libs.js && node watch.mjs",
 "build:libs": "node build.mjs"

And you need the implementation for those two build hooks, which can placed for example to src/dev/build.clj:

(ns dev.build)

(defn start-process [cmd]
  (let [;; Redirect output to stdout.
        ;; inheritIO also redirects stdin, some tools might not like that.
        builder (-> (ProcessBuilder. ^"[Ljava.lang.String;" (into-array String cmd))
        (.redirectError java.lang.ProcessBuilder$Redirect/INHERIT)
        (.redirectOutput java.lang.ProcessBuilder$Redirect/INHERIT))
        process (.start builder)]
    ;; Also stop processes when shadow-cljs is stopped.
    (.addShutdownHook (Runtime/getRuntime) (Thread. (fn [] (.destroy process))))
    process))

(defn run-cmd-configure
  {:shadow.build/stage :configure}
  [build-state {:keys [cmd]}]
  (let [process (start-process cmd)]
    build-state))

(defn run-cmd-flush
  {:shadow.build/stage :flush}
  [build-state {:keys [cmd]}]
  (let [process (start-process cmd)]
    (.waitFor process)
    build-state))

To configure ESBuild it is useful to have one file with common configuration for both dev and release builds, build_config.mjs:

export default {
  bundle: true,
  sourcemap: true,
  entryPoints: ["target/gen/libs.js"],
  outdir: "public/js",
  target: ["es2020", "chrome119", "firefox120", "safari17", "edge119"],
  metafile: true,
  logLevel: 'info'
}

And then two files to launch ESBuild, watch.mjs for dev build:

import * as esbuild from 'esbuild'
import shared from './build_config.mjs'

let ctx = await esbuild.context({...shared})
await ctx.watch()

And build.mjs for release:

import * as esbuild from 'esbuild'
import shared from './build_config.mjs'

await esbuild.build({
  ...shared,
  minify: true
})

You can investigate further and try this out using example project. The example is based on uix-starter repository so you can also check the changes compared to that.

Further improvements and notesLink to Further improvements and notes

  • Can this be used with Shadow-cljs :target :browser-test? There are differences in how imports work on Closure and other tools (for example, how CommonJS modules are handled), so it is important that the same tool is used for all builds.
  • Shadow-cljs places all JS libs into one file, with no module splitting. It could be added later. Issue
  • You can also integrate SVGR with ESBuild and other tools to include SVG images as React components. SVGR can probably also be used with Closure by pre-compiling SVG files into JS files beforehand and just committing those JS files into your repository.

ConclusionLink to Conclusion

Hopefully, Closure will be able to keep itself relevant by supporting enough ES features to parse modern JS libs. Meanwhile, you can leverage other JS bundlers with Shadow-cljs to handle your JS libs. Cherry is also an interesting ClojureScript compiler that drops Closure use altogether. It seems like React is also moving in a direction where some optimizations are done at compilation time (React Compiler), and for Cljs apps to benefit from these features directly, it probably means the JS output from the Cljs compiler would have to be processed by JS bundlers (or maybe the Cljs React wrappers can implement the same optimizations themselves.)

Juho Teperi

Related posts

More

Contact