How to enable clojure.spec'd functions in a test suite

Question

Can clojure.spec'd be included in a generic test suite? I know that we can register specifications and directly specification functions .

(ns foo (:require [clojure.spec :as s] [clojure.spec.test :as stest])) (defn average [list-sum list-count] (/ list-sum list-count)) (s/fdef average :args (s/and (s/cat :list-sum float? :list-count integer?) #(not (zero? (:list-count %)))) :ret number?) 

And later, if I want to run generative tests against this spec function, I can use stest/check .

 => (stest/check `average) ({:spec #object[clojure.spec$fspec_impl$reify__14282 0x68e9f37c " clojure.spec$fspec_impl$reify__14282@68e9f37c "], :clojure.spec.test.check/ret {:result true, :num-tests 1000, :seed 1479587517232}, :sym edgar.core.analysis.lagging/average}) 

But i) to include these test runs in my overall test suite? I am thinking about the integration of clojure.test , which test.check has . The closest I can see ii) is stest/instrument (see here ). But it looks like we just turn on the check for repl. Not quite what I want. In addition, iii) are specification functions registered?

 (defspec foo-test 100 ;; NOT this #_(prop/for-all [v ...] (= v ...)) ;; but THIS (stest/some-unknown-spec-fn foo)) 

Answer 1

Ok, this one decided. It turns out there is no solution out of the box. But some people from the clojure special slack channel put together the defspec-test solution for clojure.spec.test and clojure.test .

So give the code in the question. You can A) define a defspec-test macro that takes your test name and a list of spec'd functions. You can B) use it in your test suite.

Thanks to the Clojure community !! And, hopefully, such a utility function turns it into the main library.

A)

 (ns foo.test (:require [clojure.test :as t] [clojure.string :as str])) (defmacro defspec-test ([name sym-or-syms] `(defspec-test ~name ~sym-or-syms nil)) ([name sym-or-syms opts] (when t/*load-tests* `(def ~(vary-meta name assoc :test `(fn [] (let [check-results# (clojure.spec.test/check ~sym-or-syms ~opts) checks-passed?# (every? nil? (map :failure check-results#))] (if checks-passed?# (t/do-report {:type :pass :message (str "Generative tests pass for " (str/join ", " (map :sym check-results#)))}) (doseq [failed-check# (filter :failure check-results#) :let [r# (clojure.spec.test/abbrev-result failed-check#) failure# (:failure r#)]] (t/do-report {:type :fail :message (with-out-str (clojure.spec/explain-out failure#)) :expected (->> r# :spec rest (apply hash-map) :ret) :actual (if (instance? Throwable failure#) failure# (:clojure.spec.test/val failure#))}))) checks-passed?#))) (fn [] (t/test-var (var ~name))))))) 

IN)

 (ns foo-test (:require [foo.test :refer [defspec-test]] [foo])) (defspec-test test-average [foo/average]) 

Answer 2

The above example may fail if :failure false due to how stest/abbrev-result checks for a failure. See CLJ-2246 for details. You can get around this by specifying your own version of abbrev-result . In addition, the formatting of the fault data has changed.

 (require '[clojure.string :as str] '[clojure.test :as test] '[clojure.spec.alpha :as s] '[clojure.spec.test.alpha :as stest]) ;; extracted from clojure.spec.test.alpha (defn failure-type [x] (::s/failure (ex-data x))) (defn unwrap-failure [x] (if (failure-type x) (ex-data x) x)) (defn failure? [{:keys [:failure]}] (not (or (true? failure) (nil? failure)))) ;; modified from clojure.spec.test.alpha (defn abbrev-result [x] (let [failure (:failure x)] (if (failure? x) (-> (dissoc x ::stc/ret) (update :spec s/describe) (update :failure unwrap-failure)) (dissoc x :spec ::stc/ret)))) (defn throwable? [x] (instance? Throwable x)) (defn failure-report [failure] (let [expected (->> (abbrev-result failure) :spec rest (apply hash-map) :ret)] (if (throwable? failure) {:type :error :message "Exception thrown in check" :expected expected :actual failure} (let [data (ex-data (get-in failure [::stc/ret :result-data :clojure.test.check.properties/error]))] {:type :fail :message (with-out-str (s/explain-out data)) :expected expected :actual (::s/value data)})))) (defn check? [msg [_ body :as form]] `(let [results# ~body failures# (filter failure? results#)] (if (empty? failures#) [{:type :pass :message (str "Generative tests pass for " (str/join ", " (map :sym results#)))}] (map failure-report failures#)))) (defmethod test/assert-expr 'check? [msg form] `(dorun (map test/do-report ~(check? msg form))))