For programmers, Open Source just makes sense. We love reusing code and powerful abstractions. Building shared libraries and modules out in the open with other people gives us powerful tools we can use again and again – even if we change projects or employers.

From the business perspective, giving out code for free sounds a bit daft, but using code given by others for free sounds like a good idea. This is the inherent tension between business and open source.

People have been trying to come up with business models that are compatible with releasing open source software for decades at this point. An early example was Netscape releasing their browser as open source in 1998. Red Hat (nowadays a subsidiary of IBM), founded in 1993, built their business by distributing and supporting existing open source software, as well as building their own.

In more recent years, people have been trying to combine the software-as-a-service business model with open source. This has resulted in ideas like [the Open-core model](https://en.wikipedia.org/wiki/Open-core_model) and relicensing kerfuffles in projects like [Elastic](https://www.coss.community/cossc/elastic-license-2-0-and-the-evolution-of-open-source-licensing-3jb3)
and
[Redis](https://arstechnica.com/information-technology/2024/04/redis-license-change-and-forking-are-a-mess-that-everybody-can-feel-bad-about/).

For us at Metosin, as consultants, releasing non-customer-specific tools as open source has been an obvious thing to do. We want to make every project we work on better, and our open source libraries let us distill and reuse our learnings. We've justified the expense of maintaining libraries, [arranging conferences](https://www.metosin.fi/blog/clojutre-update) and meetups as marketing – after all, many of our customers have gotten in contact with us due to our open source work! Discussions at [Heart of Clojure 2024](https://www.youtube.com/watch?v=GEqzw6V31tk) reminded us how highly our open source efforts are appreciated in the Clojure community.

Even though open source has been one of the drivers of new business for us, actual consulting and open source have felt like separate sides of the company instead of a cohesive whole. The first step in integrating the two was taken when we [worked with Solita to bring OpenAPI 3 support to Reitit](https://www.metosin.fi/blog/openapi3) – here was a project that combined business and open source directly! We wanted more work like this.

We have worked on formulating  this into a product, and are now excited to launch [Metosin Open Source Commercial Support](https://www.metosin.fi/en/open-source-support)! If you're looking for guidance on how to better use our (or others') open source libraries, optimization or debugging help, or perhaps you really need a new feature and would be willing to put some money towards it, we're here for you.

PS. Don't worry, we'll still keep developing and maintaining our libraries, for you, for free. Commercial support is for when you need something more targeted.


Announcing Open Source Commercial Support

There have been two hard pills for me to swallow as I've matured in my
career, moving to more senior positions. By maturing I mean widening my impact from my individual contribution to the team level
and beyond – being a multiplier, as they say. These hard pills
are:

1. You see more problems than you can fix.
2. You can't expect a team to produce better quality than it knows how to.

To keep growing, I have needed to accept these new truths and adapt my approach.
First, let's unpack these truths a bit before talking more of adapting.

(Terminology note: what I'm calling a senior engineer/developer in
this post might be what other people call a _staff engineer_.
Regardless of titles, I think a developer on a career path like mine
will hit these topics some time along their career.)

## Problems

Let me draw a caricature of a software engineer's relationship with
problems along their career.

- Junior: I can see a problem, but I don't know how to fix it
- Mid-level: I can see a problem, and I know how to fix it
- Senior: I can see a lot of problems I know how to fix, but I can't fix all of them myself

The growth of an engineer in the beginning of their career has been
all about getting better at solving problems. To become an independent
contributor, one must independently solve most of the problems one
encounters in daily work. Juniors are expected to ask for help, but
also expected to learn and develop and eventually stop needing help.

A mid-level engineer can competently solve problems. However, their
visibility is often limited to the task immediately at hand. They're
thinking mostly about their own feature, not of their teammates'
features, or the totality of the work of the team. As the engineer
matures, they start to look more broadly at the work of the team. This
means they'll start seeing more problems, but also new types of
problems.

Firstly, there are all the immediate problems their teammates are
solving, and the senior engineer can help with these. However, they
can't help all of their teammates with all of their problems.

In addition, there are new types of problems. These can include things
like inconsistent assumptions between teammates, unclear APIs,
duplicate work, or quality problems. As the engineer matures, they'll
start seeing even more of these, subtler and more nuanced problems.
These kinds of problems are usually harder to fix, take more time, and
can't usually be solved once-and-for-all, but need constant
maintenance.

Thus: _You see more problems than you can fix._

## Quality and taste

Some of the new problems our developing senior sees are related to
quality. Things like constant regressions in some part of the code, a
flaky integration test suite, a crumbling separation of concerns
between two components, etc. These problems can weigh heavily on
someone who is used to excellence in their own work.

Also, as our senior has grown, they have developed a _taste_. They
have certain ways of doing things, and certain things they like to
avoid because they have been bitten by them in the past. Their
teammates probably don't share the same taste – they have developed
their own, or are in the process of developing it.

One way of trying to solve both of these issues is adding control and
quality gates. Perhaps insisting on code review, a code style, a
process or just watching over teammates' shoulders. I think we've all
seen team dynamics like this. The team has grown, or a new team has
been created, and the most senior developer wants everything done like
they would've done it themselves. Code review takes many iterations.
Nothing is ever good enough. They are trying to forcefully improve the
quality. The results are clear: velocity and team morale suffer.

An engineer who is trying to be the arbiter of quality for their team
becomes a bottleneck. If they want everything to be done up to a
standard that the team just isn't naturally producing, it's the same
as if there was no team, just one engineer. Also, focusing on one
person's view on quality makes the team share all of that person's
blind spots. Other, unseen, problems will crop up, and teammates won't
react to them because they've been passivized by the control applied
to them.

The hard truth here is that there are no quick wins for quality.
_You can't expect a team to produce better quality than it knows how to._

## Improving the team

What can be done? If there are more problems than can be fixed and
better quality can't be found, is the project doomed to mediocrity?

Surely not, our senior engineer just needs new tools!

The key to working around both of the hard truths I've discussed is
_improving the team_. Let me highlight the key parts:

1. You see more problems than _you_ can fix.
2. You can't expect a team to produce better quality than it _knows how to_.

If we as engineers can inspire our teams to work with us to fix some
of these systemic problems, it doesn't matter that we can't do
everything ourselves. If we can improve the team's more junior members
by mentoring, guiding, and giving them space to grow (by _not_
standing behind their shoulder!), the quality output by the team will
improve. It's not easy: giving the right mixture of support and
freedom is difficult. But if we ask people, they might tell us what
they need.

Improving the team is slow and uncertain work. We can't try to solve
all of the problems we see at once. We need to really think about
which problems are the most important, and which problems can't be
worked on yet. Also, when it comes to more nuanced problems like
quality and architecture, it can be hard to quantify the actual impact
of the problem. Is the misleading API actually hurting the business?
How much will a faster test suite improve work? Thus it's important to
stay humble, and to work on these things in dialogue with the team,
not as a dictator. Dialogue is a also a great way to get the team
invested in solving the problems.

Even though improving the team is slow and uncertain work, the effort
is worth it. Eventually, as the team as a whole matures, there will be
multiple seniors to share the burden of the two hard pills. This will
make dialogue within the team even more valuable, letting the team
prioritize which problems to fix even more accurately. Also, it will
be a lot less lonely than being the sole senior in a team.

We can also look at the topic of improving the team in terms of
building _culture_ in the team. But that is a topic for another day.

## Acceptance and letting go of control

With dialogue and involving others comes a final hard pill to swallow:
letting go of control. As a developer grows from junior to mid-level, they
have started relishing their feeling of control. They can solve the
problems, they can make the decisions on what code to write. They are
an independent contributor. However, as a senior, they must start to
accept the loss of control: they can't control the team's quality, and
they can't solve all the problems. They need to help the team, and let
the team help them. The result is never exactly what our senior
envisioned, but it might be something with even better outcomes.

Bringing in more perspectives from team members might reveal that our
senior had been too focused on something (e.g. test coverage) while
ignoring something more important (e.g. number of regressions).
Having to think about the actual impact of things can let our senior
_analyze their taste_. This means trying to move from subjective,
intuitive judgments to recommendations grounded in actual risks and
trade-offs. Whereas previously they just had some intuitions on how
things should be done, now they can think about what actually matters,
and what is just a matter of preference.

By the way, mentoring is a great way to both grow your team, and to
sharpen your judgment on matters of taste. Really walking through the
pros and cons of different approaches with your mentee and staying
open to their suggestions will make both of you better developers.

Something that resonated for me was a comment on the internet along
the lines of "don't break your back delivering something nobody asked
for". I know I've often tried to make a test suite better than it
actually needs to be, and pressed my teammates to do the same.

If letting go of control sounds weird, another way to think about it is
_prioritization_. As always in software, we can't work on everything,
so we must pick some valuable things to work on. To be able to
accurately prioritize which problems need solving, our senior needs to
move beyond the team level and understand what produces value in the
organization. That might mean dipping their toes into the business
side of things, interviewing stakeholders and users, and
de-emphasizing technology for a while.

## Conclusion

I'm still working on adapting to these hard truths as a senior
engineer, but I think I've managed to accept them. This has allowed me
to be satisfied with non-technical work, and to better understand my
value, even when I'm not shipping features. One of the best ways to
grow for me has been to mentor people, both teammates and co-workers
in other projects.

I hope this post gives you some food for thought on your journey,
especially if you don't agree with everything I wrote. There are many
different careers in software, and I only know mine. I hope you have
fun with yours!

On Seniority and Acceptance

If you are using [UIx] or other modern React JS wrappers with ClojureScript,
you might be looking to leverage useful JS libraries like [TanStack Query].
In some cases, you'll be hit by Closure errors when attempting to use these
JS libraries.

More and more JS libs are being released to npm with ES6+ code without transpilation. Publishing code like this can be more efficient as library users can choose what browser features they presume to be available and what features to transpile depending on their target browsers.
Closure also supports this with [`:output-feature-set` option](https://shadow-cljs.github.io/docs/UsersGuide.html#_output_language_options)

Unfortunately for Cljs users, Google Closure can be somewhat slow to support
some new ES features, which can be required even to parse the library JS code. In the case of TanStack Query, Closure is missing support for [public and static class fields](https://github.com/google/closure-compiler/issues/2731). (Some recent work has been related to this feature.)

The error when Closure can't read a problematic JS file looks something like this:

```
Errors encountered while trying to parse file
  .../shadow-cljs-esbuild/node_modules/@tanstack/query-core/build/modern/focusManager.cjs
  {:line 30, :column 2, :message "'}' expected"}
```

And the relevant JS code, where we can see the static class fields denoted by `#` prefix:

```js
var FocusManager = class extends import_subscribable.Subscribable {
  #focused;
  #cleanup;
  #setup;
  constructor() {
```

If you are using Shadow-cljs, you can choose to use another tool to 
[bundle your JS libs for your app](https://shadow-cljs.github.io/docs/UsersGuide.html#js-provider-external). In this setup, Shadow-cljs will create an entry JS file with require-forms for all the JS files used in your project, and you will use some tool to bundle those into one JS file with transpiled code for use in browsers.

With some configuration, you can make this setup nearly transparent by making Shadow-cljs run the tool for you during both release and dev builds, so developers still only need to run one command.

## JS build tools

You could use the old and well-known Webpack. ESBuild is another extremely fast alternative. Using it instead of Closure might reduce your release build times by some amount. In our case, ESbuild can process the JS libs about 15 seconds faster than Closure, but this project uses a lot of JS libraries so the effect might be smaller in other projects.

It might also seem likely that Closure could optimize the JS lib code better due to also knowing about your Cljs code and what calls you are making to the JS libs, but in our case, the total artifact size decreased by a bit by using ESBuild compared to Closure. Some of this could be due to chosen target options: maybe ESBuild is transpiling less ES features to support older browsers, so it will output less JS code.

## Shadow-cljs integration

In `shadow-cljs.edn` you need to enable the external JS-provider option and setup path where the entry file is written for ESBuild and setup the build hooks to start  the ESBuild processes from within the Shadow CLJS build:
```clj
 :app {:target :browser
       :js-options {:js-provider :external
                    :external-index "target/gen/libs.js"
                    :external-index-format :esm}
       :release {:build-hooks [(dev.build/run-cmd-flush {:cmd ["yarn" "build:libs"]})]}
       :dev {:build-hooks [(dev.build/run-cmd-configure {:cmd ["yarn" "start:libs"]})]}}
```

*NOTE: You could also replace the build hooks with regular Clojure code which starts both the ESBuild process and Shadow process: https://shadow-cljs.github.io/docs/UsersGuide.html#clj-run. This could be simpler as we don't need to start the processes in any specific stage of the shadow-cljs compilation pipeline (which is the intended use of build hooks.)*

In `package.json` you can setup the two commands to run ESBuild relevant configuration:
```js
 "start:libs": "mkdir -p target/gen && touch target/gen/libs.js && node watch.mjs",
 "build:libs": "node build.mjs"
```

And you need the implementation for those two build hooks, which can placed  for example to `src/dev/build.clj`:
```clj
(ns dev.build)

(defn start-process [cmd]
  (let [;; Redirect output to stdout.
        ;; inheritIO also redirects stdin, some tools might not like that.
        builder (-> (ProcessBuilder. ^"[Ljava.lang.String;" (into-array String cmd))
        (.redirectError java.lang.ProcessBuilder$Redirect/INHERIT)
        (.redirectOutput java.lang.ProcessBuilder$Redirect/INHERIT))
        process (.start builder)]
    ;; Also stop processes when shadow-cljs is stopped.
    (.addShutdownHook (Runtime/getRuntime) (Thread. (fn [] (.destroy process))))
    process))

(defn run-cmd-configure
  {:shadow.build/stage :configure}
  [build-state {:keys [cmd]}]
  (let [process (start-process cmd)]
    build-state))

(defn run-cmd-flush
  {:shadow.build/stage :flush}
  [build-state {:keys [cmd]}]
  (let [process (start-process cmd)]
    (.waitFor process)
    build-state))
```

To configure ESBuild it is useful to have one file with common configuration for both dev and release builds, `build_config.mjs`:

```js
export default {
  bundle: true,
  sourcemap: true,
  entryPoints: ["target/gen/libs.js"],
  outdir: "public/js",
  target: ["es2020", "chrome119", "firefox120", "safari17", "edge119"],
  metafile: true,
  logLevel: 'info'
}
```

And then two files to launch ESBuild, `watch.mjs` for dev build:

```js
import * as esbuild from 'esbuild'
import shared from './build_config.mjs'

let ctx = await esbuild.context({...shared})
await ctx.watch()
```

And `build.mjs` for release:

```js
import * as esbuild from 'esbuild'
import shared from './build_config.mjs'

await esbuild.build({
  ...shared,
  minify: true
})
```

You can investigate further and try this out using [example project][example-project]. The example is based on [uix-starter](https://github.com/pitch-io/uix-starter) repository so you can also check the changes compared to that.

## Further improvements and notes

- Can this be used with Shadow-cljs `:target :browser-test`? There are differences in how imports work on Closure and other tools (for example, how CommonJS modules are handled), so it is important that the same tool is used for all builds.
- Shadow-cljs places all JS libs into one file, with no module splitting. It could
be added later. [Issue](https://github.com/thheller/shadow-cljs/issues/1162)
- You can also integrate [SVGR] with ESBuild and other tools to include
SVG images as React components. SVGR can probably also be used with Closure by pre-compiling SVG files into JS files beforehand and just committing those JS files into your repository.

## Conclusion

Hopefully, Closure will be able to keep itself relevant by supporting enough ES features to parse modern JS libs. Meanwhile, you can leverage
other JS bundlers with Shadow-cljs to handle your JS libs. [Cherry] is also
an interesting ClojureScript compiler that drops Closure use altogether. It seems like React is also moving in a direction where some optimizations are done at compilation time ([React Compiler]), and for Cljs apps to benefit from these features directly, it probably means the JS output from the Cljs compiler would have to be processed by JS bundlers (or maybe the Cljs React wrappers can implement the same optimizations themselves.)

[TanStack Query]: https://tanstack.com/query/latest
[UIx]: https://github.com/pitch-io/uix
[Cherry]: https://github.com/squint-cljs/cherry
[SVGR]: https://react-svgr.com/
[React Compiler]: https://react.dev/blog/2024/02/15/react-labs-what-we-have-been-working-on-february-2024
[example-project]: https://github.com/metosin/shadow-cljs-esbuild

Using Shadow-cljs with ESBuild



We are happy to announce the initial release of our JSON encoding and decoding library for Clojure, [jsonista 0.1.0](https://github.com/metosin/jsonista). It’s fast and it has explicit configuration – no global mutable state!

## Yet another JSON library?!

Yes, you read that right: we’ve created yet another JSON library for Clojure. There are a couple of them, but probably the most popular one is [Cheshire](https://github.com/dakrone/cheshire). It’s fast and full of features, so why did we create a new library?

A year ago, Tommi started to focus on the performance of the Clojure web stack and as a result, we got [Muuntaja](https://github.com/metosin/muuntaja), a ground-up rewrite of [ring-middleware-format](https://github.com/ngrunwald/ring-middleware-format). Muuntaja is a Ring middleware and Pedestal interceptor library for parsing and decoding data in HTTP requests and responses. It handles the content type negotiation, but it defers to libraries like Cheshire and [transit-clj](https://github.com/cognitect/transit-clj) for the actual serialization and deserialization.

Muuntaja is much faster than it’s predecessor, but this got us thinking. Could we make the JSON processing itself go faster?

Tommi suggested that we build on [jackson-databind](https://github.com/FasterXML/jackson-databind). It’s a high-level library for mapping Java objects to JSON. This suggestion didn’t make any sense to me: since Cheshire is built on the low-level [jackson-core](https://github.com/FasterXML/jackson-core), there’s no way using databind would be faster than that. To demonstrate Tommi how wrong he was, I whipped up a quick implementation and some benchmarks. Guess what happened?

The databind version was faster. Clearly my intuition was off! Here are the results from [a more recent benchmark](https://github.com/metosin/jsonista/blob/f2afb123b770fd9af1a8766f385b5e77b9f483e7/test/jsonista/json_perf_test.clj):

![](../../img/posts/jsonista-decode.png)
![](../../img/posts/jsonista-encode.png)

Benchmarking is hard and likely we made mistakes. Together with Tommi and Kalle, we decided to put the code in a library anyway. That’s how we got jsonista.

What makes it faster than Cheshire? We believe these are the main reasons:

* Jsonista pushes as much work as possible to Jackson and to Java, whereas Cheshire's performance-critical parts are in Clojure.
* Cheshire's big functions and use of `definline` prevent JVM inlining. Jsonista's methods are small.
* Unlike Jsonista, Cheshire uses dynamic variables. They always come with a performance penalty.

Jackson's authors have taken care to make Jackson fast and Jsonista tries to make full use of that.

## Usage

The coordinates are `[metosin/jsonista "0.1.0"]`. The basic operations are straightforward:

```clojure
(require '[jsonista.core :as j])

(j/write-value-as-string {"hello" 1})
;; => "{\"hello\":1}"

(j/read-value *1)
;; => {"hello" 1}
```

Muuntaja has built-in support for jsonista. You can enable it in your Muuntaja configuration like this:

```clojure
(require '[muuntaja.format.jsonista :as json-format])

(def m
  (muuntaja/create
    (-> muuntaja/default-options
        json-format/with-json-format)))
```

For more information about configuration and custom encoders, see [GitHub](https://github.com/metosin/jsonista) and [the API reference](https://metosin.github.io/jsonista/jsonista.core.html). Once you have tried it out, please let us know how it worked for you!


Faster JSON processing with jsonista

Yet another JSON library?!Link to Yet another JSON library?!

UsageLink to Usage

Related posts

Announcing Open Source Commercial Support

On Seniority and Acceptance

Using Shadow-cljs with ESBuild

Contact