Malli, Data Modelling for Clojure Developers

PrologueLink to Prologue

Malli is a high-performance, data-driven data specification library for Clojure. It is widely used in the Clojure Community having millions of downloads and an active channel on Slack. This is the fifth post on Malli, focusing on new features of the 0.14.0 version. Older posts include:

• Malli, Data-Driven Schemas for Clojure/Script
• Structure and Interpretation of Malli Regex Schemas
• High-Performance Schemas in Clojure/Script with Malli 1/2
• Transforming data with Malli and Meander

Malli SchemasLink to Malli Schemas

If you are new to Malli or to Clojure, here is a sample code to define Schemas and validate values against them:

(require '[malli.core :as m])

(def UserId :string)

(def Address 
  [:map
   [:street :string]
   [:latlon [:tuple :double :double]]])

(def User
  [:map
   [:id UserId]
   [:address Address]])

(m/validate
 User
 {:id 123
  :address {:street "Hämeenkatu 13"
            :latlon [61.4980155, 23.7640067]})
; => true

Malli supports also human and machine-readable error messages, value transformation, value generation, inferring schemas from values, schema serialization, function schemas, static type linting and much more. See README for all features.

New Features in 0.14.0Link to New Features in 0.14.0

The new version is released today, containing small improvements and fixes, but also two important new features:

1. New Development Mode
2. Support for Var Schema References

New Development ModeLink to New Development Mode

This is big. Malli has had pretty errors for a long time, but now pretty errors cover all development time errors - printing descriptive errors and hints on how to fix them. This is the way all libraries should be built.

Coercion in normal mode:

(m/coerce [:enum "S" "M" "L"] "XL")
; Execution error (ExceptionInfo) at malli.core/-exception (core.cljc:136).
; :malli.core/coercion

Same in the new development mode:

;; start the development mode
((requiring-resolve 'malli.dev/start!))
; malli: dev-mode started

(m/coerce [:enum "S" "M" "L"] "XL")

-- Schema Error ----------------------------------------------------- user:16 --

Value:

  "XL"

Errors:

  ["should be either S, M or L"]

Schema:

  [:enum "S" "M" "L"]

More information:

  https://cljdoc.org/d/metosin/malli/CURRENT

--------------------------------------------------------------------------------
; Execution error (ExceptionInfo) at malli.core/-exception (core.cljc:136).
; :malli.core/coercion

Schema creation error:

(m/schema [:map [:name :string?]])

-- Schema Creation Error -------------------------------------------- user:18 --

Invalid Schema

  :string?

Did you mean

  string?
  :string

More information:

  https://cljdoc.org/d/metosin/malli/CURRENT

--------------------------------------------------------------------------------
; Execution error (ExceptionInfo) at malli.core/-exception (core.cljc:136).
; :malli.core/invalid-schema

Invalid reference type:

(m/schema [:ref 'id])

-- Schema Error ----------------------------------------------------- user:20 --

Invalid Reference

  [:ref id]

Reason

  Reference should be one of the following:

  - a qualified keyword, [:ref :user/id]
  - a qualified symbol,  [:ref 'user/id]
  - a string,            [:ref "user/id"]
  - a Var,               [:ref #'user/id]

More information:

  https://cljdoc.org/d/metosin/malli/CURRENT

--------------------------------------------------------------------------------
; Execution error (ExceptionInfo) at malli.core/-exception (core.cljc:136).
; :malli.core/invalid-ref

There is a clean separation between production and development modes:

• Production
  • Exceptions are thrown with spesific :type and context information as ex-data
  • No string formatting, nothing extra, just throw, fail fast and early
  • Helps to keep bundle sizes on ClojureScript smaller (Malli starts from 2kb gzipped)
• Development
  • Each exception :type (or class) can register its own error report using malli.dev.virhe/-format multimethod
  • Uses fipp & virhe under the hood and has a custom EDN printer for Clojure/Script
  • All of malli is available - humanized error messages, generated sample data etc.
  • Pretty error reports are printed before exceptions are thrown so the client application will get exactly the same exception as in production mode

Support for Var Schema ReferencesLink to Support for Var Schema References

Malli supports multiple ways for defining and reusing schemas:

1. Schemas as Vars and Values - the plumatic way
2. Schemas via a Global Registry - the clojure.spec way
3. Schemas via Local Registries - the data way

The User example above uses the first one - Schemas as defined using def and they are embedded as values:

(m/form User)
; [:map
;  [:id :int]
;  [:address [:map
;             [:street :string]
;             [:latlon [:tuple :double :double]]]]]

With 0.14.0, Vars are now first-class schema references. This enables both recursive schemas and allows parent schema to control whether to inline schemas or not.

(def User2
  [:map
   [:id UserId] ;; embeded
   [:address #'Address] ;; reference
   [:friends [:set [:ref #'User2]]]]) ;; recursive reference

(m/form User2)
; [:map
;  [:id :string]
;  [:address #'Address]
;  [:friends [:set [:ref #'user/User]]]]

Testing the recursion:

(require '[malli.generator :as mg])

(mg/generate User2)
; {:id "6cJ7zpi8VX",
;  :address {:street "18Eqw3x8w151K633H73D"
;            :latlon [-0.759521484375 3.693115234375]},
;  :friends #{{:id "1"
;              :address {:street "e"
;                        :latlon [-1.25 0.5]}
;              :friends #{}}}}

There is also a new helper to embed the non-recursive refecences into the parent, like what Prettify TypeScript does for TS.

(m/deref-recursive User2)
; [:map
;  [:id :string]
;  [:address [:map
;             [:street :string]
;             [:latlon [:tuple :double :double]]]]
;  [:friends [:set [:ref #'user/User2]]]] ;; recursive, can't inline!

Var references can be serialized, but the deserializion is disabled by default.

(require '[malli.edn :as edn])

(-> User2 
    (edn/write-string)
    (edn/read-string)) ;; fails!

8-bit ascii colored message seen through Cursive IDE:

Schema RegistriesLink to Schema Registries

If using Vars is not your thing, you can still use global or local schema registry instead. Below is the same example using a local (serializable) schema registry.

(def User3
  [:schema
   {:registry {"id" :string
               "address" [:map
                          [:street :string]
                          [:latlon [:tuple :double :double]]]
               "user" [:map
                       [:id "id"]
                       [:address "address"]
                       [:friends [:set [:ref "user"]]]]}}
   "user"])

(m/deref-recursive User3)
;[:map
; [:id :string]
; [:address [:map 
;            [:street :string] 
;            [:latlon [:tuple :double :double]]]]
; [:friends [:set [:ref "user"]]]]

Round-robin to EDN string and back:

(->> User3 
     (edn/write-string)  ;; serialize 
     (edn/read-string)   ;; deserialize
     (mg/generate)       ;; example value
     (m/validate User2)) ;; validate
; => true

Going ForwardLink to Going Forward

Malli is an inspiring project to develop. We (and many others) use it as the go-to tool with Clojure: it works, it's good, it's under active development, and it's fun to develop and easy to extend. Thanks to Clojurists Together long term funding for 2024, I'm planning to explore how far can we go with a dynamically typed lisp and data-driven schemas. Join the discussion to get involved.

Changelog of the new 0.14.0 version is found here.