June 21, 2018

TechEmpower framework benchmarks & Duct

Hey people of the internet, in order to continue our serie on Clojure & Duct, I decided to implement the TechEmpower frameworks benchmark this time using the Duct framework, as it is not already listed in Round 16 and I saw no Pull Request about it on their Github, and because this is fun & quick.

TechEmpower frameworks benchmark

This project provides representative performance measures across a wide field of web application frameworks. With much help from the community, coverage is quite broad and we are happy to broaden it further with contributions. The project presently includes frameworks on many languages including Go, Python, Java, Ruby, PHP, C#, Clojure, Groovy, Dart, JavaScript, Erlang, Haskell, Scala, Perl, Lua, C, and others. The current tests exercise plaintext responses, JSON seralization, database reads and writes via the object-relational mapper (ORM), collections, sorting, server-side templates, and XSS counter-measures. Future tests will exercise other components and greater computation.

As of today, the project have been implemented in Clojure with:

  • aleph
  • compojure
  • htt-kit
  • immutant
  • luminus
  • macchiato
  • pedestal
  • reitit

and if it ships with Round 17, duct will come to the party.

This is not my first time, I already added support for the Martini, Goji & go-json-rest frameworks in 2014.

The current implementation for duct is on my GitHub, don’t hesitate to fork & pull request if you see any improvement or bug :)

Implementation

Scaffolding

Type lein new duct hello +ataraxy +postgres and we’re good to go. We’ll directly update the dependencies in our project.clj file so that we’re ready for the various implementations.

project.clj

  :dependencies [[org.clojure/clojure "1.9.0"]
                 [duct/core "0.6.2"]
                 [duct/module.logging "0.3.1"]
                 [duct/module.web "0.6.4"]
                 [duct/module.ataraxy "0.2.0"]
                 [duct/module.sql "0.4.2"]
                 [duct/database.sql.hikaricp "0.3.3"]
                 [org.postgresql/postgresql "42.1.4"]
                 [hiccup "1.0.5"]]

We just added duct/database.sql.hikaricp and hiccup.

In order to run in local since I already have a Postgres installed, I chose that. Edit your dev.edn file so that it looks like this

dev.edn

{:duct.core/environment :development
 :duct.core/include ["hello/config"]

 :duct.module/sql
 {:database-url "jdbc:postgresql://localhost:5432/benchmark"}}

Then connect to your Postgres, create a database named benchmark and create two tables World and Fortune

create database benchmark;
\connect benchmark
create table World(id int primary key, randomNumber int);
create table Fortune(id int primary key, message varchar(255));

Now open your REPL and be ready.

JSON

In order to implement the JSON endpoint whose job is to return {message: "Hello, World!"} we’re going to use ataraxy and the duct.handler.static/ok feature.

Open your config.edn file and add the following:

 :duct.module/ataraxy
 {[:get "/json"] [:json]}

 [:duct.handler.static/ok :hello.handler/json]
 {:body {:message "Hello, World!"}}

That’s it, it is implemented. We declared in ataraxy a route for GET on uri /json that is to be referenced by the name :json then we declared a composite key [:duct.handler.static/ok :hello.handler/json] (here :hello.handler/json is our :json just from above). This composite key returns a body of type map, that duct will serialize to JSON automatically.

Type (reset) in your REPL and in a second terminal type http :3000/json you should be greeted with the correct JSON.

Plain text

The implementation is time is almost the same. Open your config.edn file and edit/add the following.

:duct.module/ataraxy
{[:get "/json"]      [:json]
 [:get "/plaintext"] [:plaintext]}
  
[:duct.handler.static/ok :hello.handler/json]
{:body {:message "Hello, World!"}}

[:duct.handler.static/okplain :hello.handler/plaintext]
{:body "Hello, World!"}

We’re going to create this duct.handler.static/okplain because it is not included in duct. In order to do that create a new file src/hello/handler/plaintext.clj and type te following code.

src/hello/handler/plaintext.clj

(ns hello.handler.plaintext
  (:require [integrant.core :as ig]
            [ataraxy.handler :as handler]
            [ataraxy.response :as ataraxy-resp]
            [ring.util.response :as resp]))

; Create a specific init-key extending ::ok, patching the Content-Type header
; so that it equals "text/plain" instead of "application/octet-stream"
(defmethod ig/init-key :duct.handler.static/okplain [_ response]
  (let [val (ig/init-key :duct.handler.static/ok response)
        patched (update-in (val response) [:headers "Content-Type"] (constantly "text/plain"))]
    (constantly patched)))

In this file we create a new ig/init-key multi-method implementation for the :duct.handler.static/okplain specific keyword.

The implementation calls the ig/init-key implementation for :duct.handler.static/ok which augments the response with a :status 200, and patch the response by adding in :headers and Content-Type which is to be text/plain.

Finally since the result of ig/init-key is a function, we return constantly this patched version of the response.

Go back in your second terminal type http :3000/plaintext you should be greeted with the correct plain text message, take a look at the Content-Type and make sure it’s text/plain.

Without patching this, it would have been application/octet-stream. This is specific to what the TechEmpower frameworks benchmark expect.

Database queries

In the first part we created the table World. The goal of the 2 next tests is to execute a request on the database. The first one will execute a single query by searching with a random id. The second one will get a URL parameter to configure how much times it has to do that.

We’ll start by adding our Boundary in src/hello/boundary/world_db.clj

src/hello/boundary/world_db.clj

(ns hello.boundary.world-db
  (:require [duct.database.sql]
            [clojure.java.jdbc :as jdbc]))

(defn- query [db]
  (first
    (jdbc/query db ["select * from world where id = ?" (inc (rand-int 9999))])))

(defprotocol World
  (make-single-query [db])
  (make-multiple-queries [db num]))

(extend-protocol World
  duct.database.sql.Boundary

  (make-single-query [{:keys [spec]}]
    (query spec))

  (make-multiple-queries [{:keys [spec]} num]
    (repeatedly num #(query spec))))

The implementation defines a function named query which execute a SQL query looking for a specific id which will be a random integer between 1 and 10 000

Our World protocol have two methods make-single-query and make-multiple-queries which will do the same but multiple times.

By extending the World protocol and extending duct.database.sql.Boundary we can retrieve the database spec and the two implementation are just calling the query method, or repeatedly calling it.

Having created the Boundary, we now just need to use it.

Add the needed config to config.edn:

:duct.module/ataraxy
 {[:get "/json"]                 [:json]
  [:get "/plaintext"]            [:plaintext]
  [:get "/db"]                   [:single-query]
  [:get "/queries" #{?queries}]  [:queries]}

; ...

:hello.handler/single-query
{:db #ig/ref :duct.database/sql}

:hello.handler/queries
{:db #ig/ref :duct.database/sql}

Create a new file named src/hello/handler/single_query.clj and implement the functionality.

src/hello/handler/single_query.clj

(ns hello.handler.single-query
  (:require [integrant.core :as ig]
            [ataraxy.response :as response]
            [hello.boundary.world-db :as world-db]))

(defmethod ig/init-key :hello.handler/single-query [_ {:keys [db]}]
  (fn [{[_] :ataraxy/result}]
    [::response/ok (world-db/make-single-query db)]))

We just import our Boundary and use it to respond to the client. That’s all there is to it.

Create a new file named src/hello/handler/queries.clj and implement the functionality.

src/hello/handler/queries.clj

(ns hello.handler.queries
  (:require [integrant.core :as ig]
            [ataraxy.response :as response]
            [hello.boundary.world-db :as world-db]))

; taken from the luminus sample
(defn query-count
  "Parse provided string value of query count, clamping values to between 1 and 500."
  [^String queries]
  (let [n ^long (try (Integer/parseInt queries)
                     (catch Exception _ 1))] ; default to 1 on parse failure
    (cond
      (< ^long n 1) 1
      (> ^long n 500) 500
      :else n)))

(defmethod ig/init-key :hello.handler/queries [_ {:keys [db]}]
  (fn [request]
    [::response/ok
     (let [queries (get-in request [:params :queries] "1")
           num (query-count queries)]
       (world-db/make-multiple-queries db num))]))

As you’ve seen I took the implementation of query-count from the Luminus implementation sample. It just checks that a parameter queries can be parsed as an integer, if so it will try to bound it between 1 and 500, otherwise it will default to 1.

The duct implementation extract the :queries parameter from the request and give it to our Boundary so that it can loop on it that much times.

Go back in your second REPL and hit http :3000/db and http :3000/queries?queries=3 and you should have the desired result.

Fortunes

In the first part we created the table Fortune. The goal of this test is to execute select * on the database, format the messages as HTML and return it to the client.

We’ll start by adding our Boundary in src/hello/boundary/fortune_db.clj

src/hello/boundary/world_db.clj

(ns hello.boundary.fortune-db
  (:require [duct.database.sql]
            [clojure.java.jdbc :as jdbc]))

(defprotocol Fortune
  (get-all [db]))

(extend-protocol Fortune
  duct.database.sql.Boundary

  (get-all [{:keys [spec]}]
    (jdbc/query spec ["select * from fortune"])))

Nothing specific, the get-all function just executes a select * from fortune.

Now that we have our Boundary we need to use it to implement the functionality. Open the config.edn file and modify it.

 :duct.module/ataraxy
 {[:get "/json"]                 [:json]
  [:get "/plaintext"]            [:plaintext]
  [:get "/db"]                   [:single-query]
  [:get "/queries" #{?queries}]  [:queries]
  [:get "/fortunes"]             [:fortunes]}
  
 ; ...
 
 :hello.handler/fortunes
 {:db #ig/ref :duct.database/sql}}

It’s all for the configuration, but we still need to implement the :hello.handler/fortunes.

Create a new file named src/hello/handler/fortunes.clj

src/hello/handler/fortunes.clj

(ns hello.handler.fortunes
  (:require [integrant.core :as ig]
            [ataraxy.response :as response]
            [hello.boundary.fortune-db :as fortune-db]))

; copied from pedestal implementation
(defn prepare-fortunes
  [fortunes]
  (sort-by :message
           (conj fortunes
                 {:id 0 :message "Additional fortune added at request time."})))

; ------- raw string -------

(def ^String base-fortune-pre "<!DOCTYPE html><html><head><title>Fortunes</title></head><body><table><tr><th>id</th><th>message</th></tr>")
(def ^String base-fortune-post "</table></body></html>")
(defn fortunes-str
  "The HTML bit for this is very very small;
  Opt to create the HTML string by hand in a tight loop rather than using Hiccup"
  [fortunes]
  (let [sb (StringBuilder. ^String base-fortune-pre)]
    (doseq [{:keys [id message]} fortunes]
      (.append sb "<tr><td>")
      (.append sb (str id))
      (.append sb "</td><td>")
      (dotimes [c-idx (count message)]
        (let [c (.charAt ^String message c-idx)]
          (case c
            \& (.append sb "&")
            \" (.append sb """)
            \' (.append sb "'")
            \< (.append sb "<")
            \> (.append sb ">")
            (.append sb c))))
      (.append sb "</td></tr>"))
    (.append sb base-fortune-post)
    (.toString sb)))

(defmethod ig/init-key :hello.handler/fortunes [_ {:keys [db]}]
  (fn [{[_] :ataraxy/result}]
    (let [fortunes (prepare-fortunes (fortune-db/get-all db))]
      [::response/ok (fortunes-str fortunes)])))

The important parts are the prepare-fortunes and fortune-db/get-all function which are combined to create our dataset that have to be rendered as HTML. The implementation with StringBuilder have been copied from the pedestal implementation.

Done

On my repository and pull request I also implemented these tests using MongoDB, and running on various server implementations like Jetty, Aleph, HTTP-Kit and Immutant so that we know which combination is faster.

As always you can see the whole code on my Github

Alexandre Grison - //grison.me - @algrison