No matter how rapidly the world of software development may change, one constant is the need to ensure the quality, functionality, and reliability of our software applications.
As our demand for more and more complex applications continues to increase, so does the risk, not only that developers might program something incorrectly thereby introducing bugs, but that they might “correctly” build the wrong thing, due to having misunderstood the requirements! Worse than having bugs is having built something that is not fit for purpose and needs to be thrown away and/or rewritten.
This kind of software development waste could have been avoided if a non-technical product expert could read a detailed specification written in what just looks like English, and immediately agree that the specification describes the desired behavior. Developers and product experts would collaborate on iteratively writing as many feature specifications as necessary until each is certain that the other understands what software is going to be built.
To address this and other similar problems in the development of complex and high quality software, came what’s known as “Behavior-Driven Development”, or BDD, where Liz Keogh once defined it as “using examples to talk through how an application behaves… And having conversations about those examples.”
If you’re not already familiar with BDD, imagine if you were able to write something like this to describe the behavior of a simple calculator:
# An example of one of the behaviors our simple calculator needs to implement
Feature: Simple Calculator Addition
More than just two numbers at a time, our simple calculator needs to be able to accept a long list of numbers to add.
Scenario: Adding a list of numbers
Given I have a table of the following numbers:
| 49 |
| 54 |
| 56 |
| 48 |
| 55 |
When I add up all the numbers
Then the result should equal 262
One powerful tool that has gained widespread adoption among developers and testers in the BDD space is Cucumber. a tool written to enable a collaborative approach that bridges the gap between technical and non-technical stakeholders.
By using plain language to define test Scenarios like the one above, Cucumber makes it easier for everyone involved in a project to understand, contribute to, and validate the software’s functionality. This not only improves communication but also enhances the overall quality of the product, making Cucumber an essential component in modern software testing practices.
The power of Cucumber comes from the fact that feature files like the one above can be fed to a testing runtime that interprets the feature file, splitting it into a sequence of steps that can be executed to configure, interact with, and make assertions about software behavior. Each step gets executed in what’s called a step function.
The only trouble is, if you’re writing Clojure and want to do Cucumber testing, you’ll either have to roll your own Gherkin file parser, step function matcher that extracts and transforms data from Gherkin files, and test execution framework that combines the two together; or, you can leverage the existing cucumber-jvm library and try to interop with a library primarily designed for JVM languages with strongly-typed methods and annotations.
Or, if you’d rather just focus on writing functions in Clojure, I’ll be covering the burpless library which I wrote, that handles all the java interop for you, so you can just focus on writing your step functions. If you’ve never used Cucumber before, and would like to start, this blog post is for you!
Getting Ready to Write Cucumber Feature Tests
Let’s build a simple task management system, using Cucumber to drive our implementation. I’m assuming you have the clojure and clj executables installed on your machine; if you don’t, have a look at the guide to installing Clojure.
Create The App
I’ll be using Sean Corfield’s deps-new tool to create a new application called task-managment-system. The command template is:
$ clojure -Tnew app :name myusername/mynewapp
Replacing myusername with my own github username, the command I ran was:
$ clojure -Tnew app :name danielmiladinov/task-management-system $ cd task-management-system $ git init Initialized empty Git repository in /path/to/task-management-system/.git/ $ git add . $ git commit -m "Hello, Burpless!" [main (root-commit) 3892883] Hello, Burpless! 10 files changed, 419 insertions(+) create mode 100644 .gitignore create mode 100644 CHANGELOG.md create mode 100644 LICENSE create mode 100644 README.md create mode 100644 build.clj create mode 100644 deps.edn create mode 100644 doc/intro.md create mode 100644 resources/.keep create mode 100644 src/danielmiladinov/task_management_system.clj create mode 100644 test/danielmiladinov/task_management_system_test.clj
For the rest of this post, if you would like to follow along, freely replace every instance of danielmiladinov in the code examples with your own github username.
Add Burpless as a Test Dependency
The deps-new command we used to create the app skeleton left us with a deps.edn file already filled out with a :test alias which specifies extra paths and deps for testing. Update it in [:aliases :test :extra-deps] to add a new key-value pair for burpless, net.clojars.danielmiladinov/burpless {:mvn/version "0.1.0"}. In addition, update in [:aliases :test] to add two new key-value pairs, :main-opts ["-m" "cognitect.test-runner"] and :exec-fn cognitect.test-runner.api/test. After making my changes, my deps.edn file looks like this:
{:paths ["src" "resources"]
:deps {org.clojure/clojure {:mvn/version "1.11.1"}}
:aliases {:run-m {:main-opts ["-m" "danielmiladinov.task-management-system"]}
:run-x {:ns-default danielmiladinov.task-management-system
:exec-fn greet
:exec-args {:name "Clojure"}}
:build {:deps {io.github.clojure/tools.build
{:mvn/version "0.9.6"}}
:ns-default build}
:test {:extra-paths ["test"]
:extra-deps {org.clojure/test.check
{:mvn/version "1.1.1"}
io.github.cognitect-labs/test-runner
{:git/tag "v0.5.1" :git/sha "dfb30dd"}
net.clojars.danielmiladinov/burpless
{:mvn/version "0.1.0"}}
:main-opts ["-m" "cognitect.test-runner"]
:exec-fn cognitect.test-runner.api/test}}}
Write Your First Feature Test
Let’s write our first feature test, step by step.
Feature File
First, let’s create a new file, test-resources/task-management-system.feature, with the following contents:
Feature: Task Management System
As a user, I want to manage my tasks,
so that I can keep track of my work and priorities.
Background:
Given I have the following tasks:
| title | description | priority | status |
| Finish report | Complete annual report | High | Pending |
| Team meeting | Monthly team sync | Medium | Completed |
| Plan project | Outline new project | Low | In Progress |
| Buy groceries | Buy groceries for the week | Medium | Pending |
Scenario: Listing tasks
When I list my tasks by priority and status
Then I should see the following tasks:
"""edn
[{:title "Finish report"
:description "Complete annual report"
:priority :high
:status :pending}
{:title "Buy groceries"
:description "Buy groceries for the week"
:priority :medium
:status :pending}
{:title "Team meeting"
:description "Monthly team sync"
:priority :medium
:status :completed}
{:title "Plan project"
:description "Outline new project"
:priority :low
:status :in-progress}]
"""
This first Scenario describes a system already populated with some tasks. Each task has a title, description, priority, and status. For simplicity, storage will be in memory only. (Replacing memory storage with some form of durable storage is left as an exercise to the reader.)
Test Runner
Next, let’s edit the test namespace, danielmiladinov.task-management-system-test, that was built out for us by deps-new. It used to look like this:
(ns danielmiladinov.task-management-system-test
(:require [clojure.test :refer :all]
[danielmiladinov.task-management-system :refer :all]))
(deftest a-test
(testing "FIXME, I fail."
(is (= 0 1))))
Replace the entire file’s contents with this instead:
(ns danielmiladinov.task-management-system-test
(:require [burpless :refer [run-cucumber step]]
[clojure.test :refer [deftest is]]
[danielmiladinov.task-management-system :as tms]))
(def steps
[])
(deftest task-management-system-feature
(is (zero? (run-cucumber "test-resources/task-management-system.feature" steps))))
Now we are ready to run our test and see it fail.
Run the Test
Running the test from the commandline is simple:
$ clj -X:test
Here’s the output I got:
$ clj -X:test
Running tests in #{"test"}
Testing danielmiladinov.task-management-system-test
Scenario: Listing tasks # test-resources/task-management-system.feature:13
Given I have the following tasks:
| Title | Description | Priority | Status |
| Finish report | Complete annual report | High | Pending |
| Team meeting | Monthly team sync | Medium | Completed |
| Plan project | Outline new project | Low | In Progress |
| Buy groceries | Buy groceries for the week | Medium | Pending |
When I list my tasks by priority and status
Then I should see the following tasks:
Undefined scenarios:
file:///path/to/task-management-system/test-resources/task-management-system.feature:13 # Listing tasks
1 Scenarios (1 undefined)
3 Steps (2 skipped, 1 undefined)
0m0.046s
You can implement missing steps with the snippets below:
(step :Given "I have the following tasks:"
(fn i_have_the_following_tasks [state ^io.cucumber.datatable.DataTable dataTable]
;; Write code here that turns the phrase above into concrete actions
;; Be sure to also adorn your step function with the ^:datatable metadata
;; in order for the runtime to properly identify it and pass the datatable argument
(throw (io.cucumber.java.PendingException.))))
(step :When "I list my tasks by priority and status"
(fn i_list_my_tasks_by_priority [state ]
;; Write code here that turns the phrase above into concrete actions
(throw (io.cucumber.java.PendingException.))))
(step :Then "I should see the following tasks:"
(fn i_should_see_the_following_tasks [state ^String docString]
;; Write code here that turns the phrase above into concrete actions
(throw (io.cucumber.java.PendingException.))))
FAIL in (task-management-system-feature) (task_management_system_test.clj:10)
expected: (zero? (run-cucumber "test-resources/task-management-system.feature" steps))
actual: (not (zero? 1))
Ran 1 tests containing 1 assertions.
1 failures, 0 errors.
Execution error (ExceptionInfo) at cognitect.test-runner.api/test (api.clj:30).
Test failures or errors occurred.
Full report at:
/var/folders/9q/j3dlhz0n6kd_nwk3_bvg_yc00000gn/T/clojure-14590987010536358961.edn
As expected, the output from the test indicates that we have unmet expectations about our task management system’s behavior. 1 Scenario with 3 steps was found, but since we provided no step functions for them, the first step was marked undefined, and as a result, the rest of the steps in the Scenario were skipped. Correspondingly, the Scenario itself was also marked undefined.
Additionally, the test printed out some snippets for step functions we can copy and paste into our test namespace as a first draft. The tests still won’t pass, but with these step functions provided, the Scenario won’t be undefined the next time we run the test.
Initial Step Functions
Copy the snippets and paste them into your test namespace, adding them to the steps vector. If you prefer, you may adjust the imports in order to remove the fully qualified class names. Similarly, feel free to remove the suggested fn names (there’s no problem in having anonymous step functions with burpless; conversely, it’s there’s no problem in keeping the fn names either – it’s up to you and your personal preferences!).
But we’re still not ready to run the test again yet – steps that are followed by data tables or doc strings need special attention in burpless – functions for datatable steps need to be adorned with the ^:datatable metadata, as do functions for docstring steps need to be adorned with the ^:docstring metadata.
- The
"Given I have the following tasks:"step is followed by a datatable, so its step function needs the^:datatablemetadata. - The
"When I should see the following tasks:"step is followed by a docstring, so its step function needs the^:docstringmetadata.- Also, its
^Stringtype hint on thedocStringargument should be changed toio.cucumber.docstring.DocString.
- Also, its
Due to how DocStrings get autowired as String instances for appropriately-annotated Java method step “functions”, the cucumber-jvm SnippetGenerator defaults to typing docString arguments as String, even though they are passed to our Clojure step functions as instances of io.cucumber.docstring.DocString – calling (.getContent docString) will provide the actual String value. I will address this in a later release of burpless.
After making all of the above changes, my test namespace looks like this:
(ns danielmiladinov.task-management-system-test
(:require [burpless :refer [run-cucumber step]]
[clojure.test :refer [deftest is]]
[danielmiladinov.task-management-system :as tms])
(:import (io.cucumber.datatable DataTable)
(io.cucumber.docstring DocString)
(io.cucumber.java PendingException)))
(def steps
[(step :Given "I have the following tasks:"
^:datatable
(fn [state ^DataTable dataTable]
(throw (PendingException.))))
(step :When "I list my tasks by priority and status"
(fn [state]
(throw (PendingException.))))
(step :Then "I should see the following tasks:"
^:docstring
(fn [state ^DocString docString]
(throw (PendingException.))))])
(deftest task-management-system-feature
(is (zero? (run-cucumber "test-resources/task-management-system.feature" steps))))
Running the test again ($ clj -X:test at the terminal), we now see that the Scenario (and the first test) went from undefined to pending, followed by a long stack trace caused by the thrown PendingException. The next two steps still remain skipped.
You can easily make the tests “pass” by removing the (throw (PendingException.)) forms from each step function. But since we’re not really asserting anything yet, the lack of failure doesn’t constitute evidence of success.
More on Step Functions
Burpless step functions (as well as hook functions) take as their first argument the current value of state, the contents of an atom that the burpless Cucumber runtime maintains on your behalf for the duration of a Scenario.
Additionally, step/hook functions may take arguments from:
- step parameters
- datatables
- docstrings
Based on all the step/hook function’s inputs, the expectation is that the step function should perform whatever calculations / side effects are necessary and proper for the execution of that step, and return the “next value” for the contents of the state atom – you don’t have to, but typically the atom contains a map with one or more keys, pertaining to whatever values are needed to exercise and make assertions about the behavior of the software under test.
Applying this knowledge about step functions to the steps outlined so far in our Scenario above, we can say that:
- The step function for
"Given I have the following tasks:"can ignore itsstateargument and return, say, a map with the the:taskskey populated with the result of extracting thedataTablecontents. Transform it into a sequence of task maps (resembling the data literal from the docstring attached to the"I should see the following tasks:"step) and store them somewhere – potentially in an atom on thedanielmiladinov.task-management-systemnamespace – the choice of just where is up to you, so long as you can again retrieve those tasks later in the"When I list my tasks by priority and status"step. - The step function for
"When I list my tasks by priority and status"should interact with something indanielmiladinov.task-management-system, probably by calling some function, in order to retrieve the list of tasks that were stored by executing the previous step (sorted by priority, of course), and return an updated version of the state containing the sorted task list, so that the following step function can access it. - The step function for
"Then I should see the following tasks"could compare the actual list of sorted tasks that were stored in the state by the previous step against the incoming docstring representing the expected list of sorted tasks.
Implement the Step Functions We Have So Far
Your implementation of course can be different, but here are the changes I made to make my tests pass.
Here’s my danielmiladinov.task-management-system namespace:
(ns danielmiladinov.task-management-system
(:gen-class))
(def tasks (atom []))
(defn by-priority-and-status
"Return a collection of the tasks in the system
in descending priority and status order"
[]
(sort-by (juxt (comp {:high 1 :medium 2 :low 3}
:priority)
(comp {:in-progress 1 :pending 2 :completed 3}
:status))
@tasks))
(defn greet
"Callable entry point to the application."
[data]
(println (str "Hello, " (or (:name data) "World") "!")))
(defn -main
"I don't do a whole lot ... yet."
[& args]
(greet {:name (first args)}))
And here’s my danielmiladinov.task-management-system-test namespace:
(ns danielmiladinov.task-management-system-test
(:require [burpless :refer [run-cucumber step]]
[clojure.string :as str]
[clojure.test :refer [deftest is]]
[danielmiladinov.task-management-system :as tms])
(:import (clojure.lang IObj)
(io.cucumber.datatable DataTable)))
(def to-keyword
"Turns e.g. “In Progress” into :in-progress"
(comp keyword
#(str/replace % #"\s" "-")
str/lower-case))
(def steps
[(step :Given "I have the following tasks:"
^:datatable
(fn [state ^DataTable dataTable]
(reset! tms/tasks (->> (.asMaps dataTable)
(map (fn [m]
(-> (update-keys m to-keyword)
(update :status to-keyword)
(update :priority to-keyword))))))
state))
(step :When "I list my tasks by priority and status"
(fn [state]
(assoc state :actual-tasks (tms/by-priority-and-status))))
(step :Then "I should see the following tasks:"
^{:docstring IObj}
(fn [{:keys [actual-tasks] :as state} ^IObj expected-tasks]
(assert (= expected-tasks actual-tasks))
state))])
(deftest task-management-system-feature
(is (zero? (run-cucumber "test-resources/task-management-system.feature" steps))))
The other thing you should have noticed was the change to the arguments of the "Then I should see the following tasks" step function – it now receives expected-tasks (type-hinted as ^IObj, the parent interface of all Clojure data structures) directly, with no hint anywhere of a docString. This is made possible by burpless‘ built-in support for edn-formatted docstrings.
Burpless Custom DocString Type Conversion For EDN Data Literals
Getting EDN data out of your feature file and into your step functions is pretty simple:
- Use an edn-formatted doc string in your feature file (e.g.
"""edn [{:foo :bar}]""") – making sure to append theednafter the opening triple-quotes of the doc string. - Adorn your step function with the
^{:docstring IObj}metadata. - The last parameter to your step function will be the contents of the doc string, automatically converted into a Clojure data structure for you!
Let’s Add Some More Scenarios
So far we have one Scenario that makes uses of data table and doc string parameters. But most often there are more than one Scenario involved in specifying a feature, and we can also extract multiple input parameters for our step functions directly from the step strings themselves.
Adding more tasks
Let’s add another Scenario about adding more tasks to the system:
Scenario: Adding more tasks When I add a new task with title "Read book", description "Read the new book I bought", with priority :low Then the task "Read book" should appear in my list of tasks, with a status of :pending And the total number of tasks should be 5 When I add a new task with title "Publish blog post", description "Clojure and burpless for great success!", with priority :high" Then the task "Publish blog post" should appear in my list of tasks, with a status of :pending And the total number of tasks should be 6 When I add a new task with title "Pay bills", description "Cell, electric, and internet", with priority :medium Then the task "Pay bills" should appear in my list of tasks, with a status of :pending And the total number of tasks should be 7
Now let’s re-run the test and see what snippets are generated for us:
$ clj -X:test
Running tests in #{"test"}
Testing danielmiladinov.task-management-system-test
Scenario: Listing tasks # test-resources/task-management-system.feature:13
Given I have the following tasks: # danielmiladinov/task_management_system_test.clj:16
| title | description | priority | status |
| Finish report | Complete annual report | High | Pending |
| Team meeting | Monthly team sync | Medium | Completed |
| Plan project | Outline new project | Low | In Progress |
| Buy groceries | Buy groceries for the week | Medium | Pending |
When I list my tasks by priority and status # danielmiladinov/task_management_system_test.clj:26
Then I should see the following tasks: # danielmiladinov/task_management_system_test.clj:30
Scenario: Adding more tasks # test-resources/task-management-system.feature:23
Given I have the following tasks: # danielmiladinov/task_management_system_test.clj:16
| title | description | priority | status |
| Finish report | Complete annual report | High | Pending |
| Team meeting | Monthly team sync | Medium | Completed |
| Plan project | Outline new project | Low | In Progress |
| Buy groceries | Buy groceries for the week | Medium | Pending |
When I add a new task with title "Read book", description "Read the new book I bought", with priority :low
Then the task "Read book" should appear in my list of tasks, with a status of :pending
And the total number of tasks should be 5
When I add a new task with title "Publish blog post", description "Clojure and burpless for great success!", with priority :high"
Then the task "Publish blog post" should appear in my list of tasks, with a status of :pending
And the total number of tasks should be 6
When I add a new task with title "Pay bills", description "Cell, electric, and internet", with priority :medium
Then the task "Pay bills" should appear in my list of tasks, with a status of :pending
And the total number of tasks should be 7
Undefined scenarios:
file:///path/to/task-management-system/test-resources/task-management-system.feature:23 # Adding more tasks
2 Scenarios (1 undefined, 1 passed)
13 Steps (8 skipped, 1 undefined, 4 passed)
0m0.072s
You can implement missing steps with the snippets below:
(step :When "I add a new task with title {string}, description {string}, with priority {keyword}"
(fn i_add_a_new_task_with_title_description_with_priority [state ^String string ^String string2 ^clojure.lang.Keyword keyword]
;; Write code here that turns the phrase above into concrete actions
(throw (io.cucumber.java.PendingException.))))
(step :Then "the task {string} should appear in my list of tasks, with a status of {keyword}"
(fn the_task_should_appear_in_my_list_of_tasks_with_a_status_of [state ^String string ^clojure.lang.Keyword keyword]
;; Write code here that turns the phrase above into concrete actions
(throw (io.cucumber.java.PendingException.))))
(step :Then "the total number of tasks should be {int}"
(fn the_total_number_of_tasks_should_be [state ^Integer int1]
;; Write code here that turns the phrase above into concrete actions
(throw (io.cucumber.java.PendingException.))))
FAIL in (task-management-system-feature) (task_management_system_test.clj:37)
expected: (zero? (run-cucumber "test-resources/task-management-system.feature" steps))
actual: (not (zero? 1))
Ran 1 tests containing 1 assertions.
1 failures, 0 errors.
Execution error (ExceptionInfo) at cognitect.test-runner.api/test (api.clj:30).
Test failures or errors occurred.
Full report at:
/var/folders/9q/j3dlhz0n6kd_nwk3_bvg_yc00000gn/T/clojure-12964336415720618065.edn
As you can see, the step strings for the newly suggested step function implementations do not match the steps from the feature file exactly – they instead contain some substrings of data types wrapped in curly braces – these are called output parameters of Cucumber expressions. This is what lets us reuse the same step function to match multiple actual steps in a scenario, by isolating and parameterizing the parts of the step text that change, and passing them as arguments to your step function.
You can either pause here to add these snippets to your test runner and write the code to make them pass, or you can keep reading to see how I did it below.
My Solution
Well, how did you do? Here’s the changes I made to get the new Scenario to pass:
In my danielmiladinov.task-management-system namespace:
(ns danielmiladinov.task-management-system
(:gen-class))
(def ^:private tasks (atom []))
(defn set-tasks
[new-tasks]
(reset! tasks new-tasks))
(defn add-task
[new-task]
(swap! tasks conj new-task))
(defn by-priority-and-status
"Return a collection of the tasks in the system
in descending priority and status order"
[]
(sort-by (juxt (comp {:high 1 :medium 2 :low 3}
:priority)
(comp {:in-progress 1 :pending 2 :completed 3}
:status))
@tasks))
(defn greet
"Callable entry point to the application."
[data]
(println (str "Hello, " (or (:name data) "World") "!")))
(defn -main
"I don't do a whole lot ... yet."
[& args]
(greet {:name (first args)}))
In my danielmiladinov.task-management-system-test namespace:
(ns danielmiladinov.task-management-system-test
(:require [burpless :refer [run-cucumber step]]
[clojure.string :as str]
[clojure.test :refer [deftest is]]
[danielmiladinov.task-management-system :as tms])
(:import (clojure.lang IObj Keyword)
(io.cucumber.datatable DataTable)))
(def to-keyword
"Turns e.g. “In Progress” into :in-progress"
(comp keyword
#(str/replace % #"\s" "-")
str/lower-case))
(def steps
[(step :Given "I have the following tasks:"
^:datatable
(fn [state ^DataTable dataTable]
(tms/set-tasks (->> (.asMaps dataTable)
(map (fn [m]
(-> (update-keys m to-keyword)
(update :status to-keyword)
(update :priority to-keyword))))))
state))
(step :When "I list my tasks by priority and status"
(fn [state]
(assoc state :actual-tasks (tms/by-priority-and-status))))
(step :Then "I should see the following tasks:"
^{:docstring IObj}
(fn [{:keys [actual-tasks] :as state} ^IObj expected-tasks]
(assert (= expected-tasks actual-tasks))
state))
(step :When "I add a new task with title {string}, description {string}, with priority {keyword}"
(fn [state ^String title ^String description ^Keyword priority]
(tms/add-task {:title title :description description :priority priority :status :pending})
state))
(step :Then "the task {string} should appear in my list of tasks, with a status of {keyword}"
(fn [state ^String title ^Keyword status]
(let [matching-task (->> (tms/by-priority-and-status)
(filter (every-pred (comp (hash-set title) :title)
(comp (hash-set status) :status)))
first)]
(assert (some? matching-task) (format "Did not find task with title %s and status %s" title status))
state)))
(step :Then "the total number of tasks should be {int}"
(fn [state ^Integer expected-number-of-tasks]
(let [actual-number-of-tasks (count (tms/by-priority-and-status))]
(assert (= expected-number-of-tasks actual-number-of-tasks)
(format "Expected to find %d task%s, but actually found %d task%s"
expected-number-of-tasks
(if (= 1 expected-number-of-tasks) "" "s")
actual-number-of-tasks
(if (= 1 actual-number-of-tasks) "" "s"))))
state))])
(deftest task-management-system-feature
(is (zero? (run-cucumber "test-resources/task-management-system.feature" steps))))
What’s Left to Do?
Thank you for reading this far! There’s plenty of functionality left to add to the task management system in order to get it feature complete. Feel free out my GitHub repository that I created for the code in this post, danielmiladinov/task-management-system. Hopefully, you’ve found this post useful in helping you get started with using burpless and will consider adopting it in your (current?) next Clojure project!