Data-Driven Ring with Reitit
The first part introduced the new Reitit routing library. This is the second part and walks through the reitit-ring module and some of the problems it's trying to resolve.
RingLink to Ring
Ring is the de facto abstraction for web apps in Clojure. It defines the following concepts:
requestandresponseas mapshandlersandmiddlewareas functionsadaptersto bridge the concepts to HTTP
A simple Ring application:
(require '[ring.adapter.jetty :as jetty]) ; [ring "1.6.3"]
(defn handler [request]
{:status 200
:body "Hello World"})
(jetty/run-jetty app {:port 8080})
Ring is basically just functions and data. Because of this, pure Ring apps are easy to reason about. There are lot of adapters, routing libraries and middleware available and some great template-projects like Luminus and Duct. The async ring provides common abstractions for both Clojure and ClojureScript (via Macchiato).
In short, Ring is awesome.
MiddlewareLink to Middleware
Middleware is a higher-level function that add additional functionality to handler function. Here's an example (from Ring Wiki):
(defn wrap-content-type [handler content-type]
(fn [request]
(let [response (handler request)]
(assoc-in response [:headers "Content-Type"] content-type))))
The middleware concept is simple, but is it too simple? Composing the middleware chain is not always trivial as the ordering can matter. Only way to make sure that it's composed correctly is to invoke the chain, which is just an opaque function. We should have better tools for managing the middleware chain.
Common pattern with Ring is to mount a tower of middleware before any routing is applied. It's easy to do, but performance-wise this is not good, as not all routes need "everything". We should route first and apply only the middleware that is relevant for the given route.
Welcome Reitit-ringLink to Welcome Reitit-ring
reitit-ring is a standalone routing library for Ring that builds on top of reitit-core. It inherits all the features from the core and adds the following:
:request-methodbased routing- support for ring middleware
- transformation of the middleware chain
- dynamic extensions
- data-driven middleware
- middleware compilation
- partial route specs
- pluggable http-coercion
To get started, let's add a dependency:
[metosin/reitit-ring "0.1.0"]
Here's a simple app, with some middleware, handlers and a default-handler, handling the routing misses (HTTP Statuses 404, 405 and 406).
(require '[reitit.ring :as ring])
(defn handler [_]
{:status 200, :body "ok"})
(defn wrap [handler id]
(fn [request]
(update (handler request) :via (fnil conj '()) id)))
(def app
(ring/ring-handler
(ring/router
["/api" {:middleware [#(wrap % :api)]}
["/ping" handler]
["/admin" {:middleware [[wrap :admin]]}
["/db" {:middleware [[wrap :db]]
:delete {:middleware [[wrap :delete]]
:handler handler}}]]]
{:data {:middleware [[wrap :top]]}}) ;; all routes
(ring/create-default-handler)))
Handlers and middleware can be defined as route data either to top top-level (all methods) or under request-methods.
(app {:request-method :delete
:uri "/api/admin/db"})
; {:status 200
; :body "ok"
; :via (:top :api :admin :db :delete)}
Having a fast data-driven router is cool, but let's not stop there.
Transforming the chainLink to Transforming the chain
Middleware chain is stored as data and can be queried from the router. We can transform the chain via router option :reitit.middleware/transform.
Adding debug-middleware between other middleware:
(require '[reitit.middleware :as middleware])
(def app
(ring/ring-handler
(ring/router
["/api" {:middleware [[wrap :api]]}
["/ping" handler]]
{::middleware/transform #(interleave % (repeat [wrap :debug]))
:data {:middleware [[wrap :top]]}})))
(app {:request-method :get,
:uri "/api/ping"})
; {:status 200
; :body "ok"
; :via (:top :debug :api :debug)}
Dynamic extensionsLink to Dynamic extensions
One of the concerns with ring has been that middleware is too isolated as it doesn't know where the request is heading. We can fix that.
ring-handler matches the route before any middleware is applied. After a successful match, a routing Match is injected into the request. Middleware can read the Match at runtime and access all the route data, including other Middleware on the chain. This enables us to write systems where the (route) data and functions to interpret it (middleware) are cleanly separated.
Here's a middleware that reads :roles from a route data and if set, compares to :roles from a request.
(require '[clojure.set :as set])
(defn wrap-enforce-roles [handler]
(fn [{:keys [roles] :as request}]
(let [required (some-> request (ring/get-match) :data :roles)]
(if (and (seq required) (not (set/subset? required roles)))
{:status 403, :body "forbidden"}
(handler request)))))
(def app
(ring/ring-handler
(ring/router
["/api"
["/ping" handler]
["/admin" {:roles #{:admin}}
["/ping" handler]]]
{:data {:middleware [wrap-enforce-roles]}})))
Invoking the endpoint:
(app {:request-method :get, :uri "/api/ping"})
; {:status 200, :body "ok"}
(app {:request-method :get, :uri "/api/admin/ping"})
; {:status 403, :body "forbidden"}
(app {:request-method :get, :uri "/api/admin/ping", :roles #{:admin}})
; {:status 200, :body "ok"}
Dynamic extensions are an easy way to extend the system, but are ad-hoc in nature.
Data-driven middlewareLink to Data-driven middleware
In Pedestal, Interceptors are data, enabling better docs and advanced chain manipulation. There is no reason why Middleware couldn't be data too. Let's have data-driven middleware.
Instead of just functions, Reitit allows middleware also to be defined maps or instances of Middleware Records. They should have at least :wrap key defined, which is used in the actual request processing - with zero overhead.
(def wrap-middleware
{:name ::wrap
:description "Middleware that conjs the :id into :via in the response"
:wrap wrap})
Internally, all forms of middleware are expanded into Middleware Records via IntoMiddleware protocol.
Extensions can use the Middleware data however they want. Route data validation uses :spec key and we could build an automatic middleware chain dependency resolution with this.
Compiling middlewareLink to Compiling middleware
Each route knows the the exact Middleware chain defined for it. We can use this to optimize the chain for the route at router creation time. For this, there is a :compile key in Middleware. It expects a function of route-data router-opts => ?IntoMiddleware. The compilation is recursive, the results are merged to the parent and by returning nil the Middleware is removed from the chain.
Authorization middleware re-written as Middleware:
(require '[clojure.spec.alpha :as s])
(s/def ::role #{:admin :manager})
(s/def ::roles (s/coll-of ::role :into #{}))
(def enforce-roles-middleware
{:name ::enforce-roles
:spec (s/keys :req-un [::roles])
:compile (fn [{required :roles} _]
(if (seq required)
{:description (str "Requires roles " required)
:wrap (fn [{:keys [roles] :as request}]
(if (not (set/subset? required roles))
{:status 403, :body "forbidden"}
(handler request)))}))})
- mounts only if route has
:rolesdefined - human readable description with required roles
- defines a partial spec for the route data validation
- faster as it does less work at request-processing time
ValidationLink to Validation
Route validation works just like with the core router, but we should use a custom validator reitit.ring.spec/validate-spec! to support both the :request-method endpoints and the :spec from Middleware mounted for that route.
Here's an example using Expound:
(require '[reitit.ring.spec :as rrs])
(require '[reitit.spec :as rs])
(require '[expound.alpha :as e]) ; [expound "0.5.0"]
(def app
(ring/ring-handler
(ring/router
["/api"
;; no middleware -> not validated or enforced
["/ping" {:roles #{:pinger}
:handler handler}]
;; middleware -> enforced and validated
["/roles" {:middleware [enforce-roles-middleware]}
["/admin" {:get {:roles #{:adminz}
:handler handler}}]]]
{:validate rrs/validate-spec!
::rs/explain e/expound-str})))
; CompilerException clojure.lang.ExceptionInfo: Invalid route data:
;
; -- On route -----------------------
;
; "/api/admin" :get
;
; -- Spec failed --------------------
;
; {:middleware ..., :roles #{:adminz}, :handler ...}
; ^^^^^^^
;
; should be one of: :admin, :manager
CoercionLink to Coercion
Coercion is the process of transforming and validating input and output parameters between external formats and clojure data. Reitit-ring ships with full-blown pluggable http-coercion (thanks to work in compojure-api), supporting clojure.spec, data-specs and Plumatic Schema as separate modules.
Unlike with the core router, coercion is applied automatically by coercion Middleware from reitit.ring.coercion. These include:
coerce-request-middleware- request parameter coercioncoerce-response-middleware- response body coercioncoerce-exceptions-middleware- coercion exceptions as http responses
Syntax for :parameters and :responses is adopted from ring-swagger. All have :spec defined, so you can validate the syntax too.
Example application with data-specs coercion:
(require '[reitit.ring.coercion :as rrc])
(require '[reitit.coercion.spec]) ; [metosin/reitit-spec "0.1.0"]
(require '[reitit.ring :as ring])
(def app
(ring/ring-handler
(ring/router
["/api"
["/ping" (fn [_]
{:status 200
:body "pong"})]
["/plus/:z" {:post {:coercion reitit.coercion.spec/coercion
:parameters {:query {:x int?}
:body {:y int?}
:path {:z int?}}
:responses {200 {:body {:total pos-int?}}}
:handler (fn [{:keys [parameters]}]
;; parameters are coerced
(let [x (-> parameters :query :x)
y (-> parameters :body :x)
z (-> parameters :path :z)
total (+ x y z)]
{:status 200
:body {:total total}}))}}]]
{:data {:middleware [rrc/coerce-exceptions-middleware
rrc/coerce-request-middleware
rrc/coerce-response-middleware]}})))
Valid request:
(app {:request-method :post
:uri "/api/plus/3"
:query-params {"x" "1"}
:body-params {:y 2}})
; {:status 200, :body {:total 6}}
Invalid request:
(app {:request-method :post
:uri "/api/plus/3"
:query-params {"x" "abba"}
:body-params {:y 2}})
; {:status 400,
; :body {:spec "(spec-tools.core/spec {:spec (clojure.spec.alpha/keys :req-un [:$spec8596/x]), :type :map, :keys #{:x}, :keys/req #{:x}})",
; :problems [{:path [:x], :pred "clojure.core/int?", :val "abba", :via [:$spec8596/x], :in [:x]}],
; :type :reitit.coercion/request-coercion,
; :coercion :spec,
; :value {:x "abba"},
; :in [:request :query-params]}}
PerformanceLink to Performance
As Reitit-ring provides a rich set of features, so it must be slow right? No, the abstractions actually make it faster and thanks to snappy routing algorithms in reitit-core, it's really, really fast.
Below are the average route resolution times of a mid-size example REST route tree. As all perf tests, the tests may contain errors. Please let us know.
Coercion is the slowest part of the default stack, the this is because the actual coercion is done in 3rd party libraries. Reitit precompiles all the coercers and the overhead is as small as possible.
Road aheadLink to Road ahead
Next steps is to support Swagger and OpenAPI as a separate modules. Requires still some work to get working with ClojureScript.
As everything is data, it would be great to visualize the routing system including Middleware chains. Also, runtime chain visualization and debugging tools would be nice. If you interested in implementing this kind of things, please let us know.
Despite being modular, Reitit-ring is just a routing library, so where can one find more data-driven Middleware like CORS, OAuth, Content Negotiation etc? Should we re-package and host all the common Middleware as optimized data-driven Middleware? Should there be a community repo or an Github organisation for this? Could some parts of Reitit be part of the next Ring Spec? Ideas and comments welcome.
As reitit-ring is fully compatible with Ring, it would also be great to see it as optional routing engine for the established template projects like Luminus and Duct.
Final WordsLink to Final Words
Reitit-ring is an new data-driven web-router for Clojure(Script). It's fast, easy to extend and enables has some new concepts like first-class route data and data-driven middleware. Tools like route data validation and route conflict resolution help keep the routing trees sound.
Big thanks to existing Clojure routing and web libraries for ideas and showing the way. These including Bidi, Compojure, Compojure-api, Kekkonen, Pedestal, Ring-swagger and Yada. Also to HttpRouter for some perf stuff.
For discussions, there is #reitit channel in Clojurians slack.
Pointers to get started:
Happy ring-routing.
