No cucumber!
Warning: this blog post is highly opinionated!
Some time ago I’ve heard that cucumbers are considered as “negative-calorie” food. People say, they have lower energy content (16 kilocalories per 100g) than the amount of energy your body needs to digest. Maybe that’s not true and maybe there’s no such thing like “negative-calorie” food.
But what is not true to the vegetable, it applies to the testing tool cucumber.io↗:
It adds more overhead to your software development than you benefit from it.
What is cucumber?
The special thing about cucumber is: You don’t write your tests in your usual programming language but using a special language “gherkin”, that should be almost readable to normal people:
Scenario Outline: A user withdraws money from an ATM
Given <Name> has a valid Credit or Debit card
And their account balance is <OriginalBalance>
When they insert their card
And withdraw <WithdrawalAmount>
Then the ATM should return <WithdrawalAmount>
And their account balance is <NewBalance>
Examples:
| Name | OriginalBalance | WithdrawalAmount | NewBalance |
| Eric | 100 | 45 | 55 |
| Gaurav | 100 | 40 | 60 |
| Ed | 1000 | 200 | 800 |
example from cucumber on wikipedia.org↗
Cucumber test execution will result in a documentation. If you execute it during your continuous integration↗ pipelines, you will always have an up-to-date “living” documentation.
But how does it execute?
Cucumber parses the ghirkin files line by line and processes its keywords “Given”, “When”, “Then” etc. and the variables.
You will need to provide some “glue code”, to process each line. Example using java:
@Then("^the ATM should return (\\d+)$")
public void atm_should_return(int money) {
// here you call your implementation code
}
Then cucumber will find your glue code java method by the provided regular expression and execute it.
Why do people want to use it?
Collaboration with business
Cucumber is said to support collaboration between developers and business people. I even heard that people expected business analysts could write the gherkin specification.
Living documentation
Cucumber will provide a documentation that is always up to date.
Why I don’t like cucumber:
Where is the collaboration with business?
I’ve never experienced some collaboration support from cucumber. In my experience the gherkin files are always written by developers or test engineers.
Who want’s to read the documentation?
Even by using JUnit I can create documentation, just using the @DisplayName annotation or something similar. No need for a complicated framework.
But first we need to answer the questions:
- Who is the target audience for the documentation? Who want’s to read the documentation?
- Is there anyone interested in a gherkin-style documentation?
- Maybe people prefer some tutorials?
First find out the needs of the audience, then provide a proper documentation for them.
Don’t just write documentation, because some manager says: “We need some documentation.”
Don’t just use some tool, because it’s fancy.
Fake documentation
The gherkin file is just decoration. The glue code is where the magic happens. Depending on the glue code you can make any gherkin file documentation executable. You can even fake your documentation and your real software has some completely different behaviour.
Regex
I hate regular expressions.
No support for refactoring
I love using the refactoring capabilities of IntelliJ IDEA↗: renaming, extract method etc.
But I can’t use them for refactoring cucumber annotated methods, because cucumber does not point to a glue code method. It finds the glue code method by a regex in an annotation.
Overhead
When writing unit tests in I often need to switch between test code and implementation code. That’s easy for me because a can split the screen: The test code is on the left side and the implementation code is on the right side.
But with cucumber there are three files I need to read and edit: gherkin file, glue code and implementation code. Not enough space on my screen. Or maybe not enough space in my head?
Shared mutable state
I love java because I do not need to think about memory and freeing variables most of the time. By providing a garbage collector java had eliminiated one of the major sources of programming mistakes decades ago.
Since many years we have Optional which almost eliminates one of the most java programming mistakes: NullPointerExceptions.
The last big problem in java problems is shared mutable state.
You access one variable but someone else has written a different value to it, and now you get something completely different than what you expected.
Therefore I usually strive for a more functional style of code, even when coding java.
It’s just easier to reason about a pure functional method than about code with mutable state and side effects.
I’m not good in functional coding, and often it does not make much sense in java.
If I’m not able to write functional code, then I try to have immutable state.
If it’s not possible to have immutable state, then I try to have only one single place where reading and changing the state is encapsulated.
I try to avoid shared mutable state as much as possible.
Shared mutable state is the root source of all evil bugs.
When implementing glue code I needed to hand over some state from the execution of one line of the gherkin file to the next line, in the glue code: from one method to the other method. In a real program I had added a parameter to the method. But cucumber called the method by regex, so: no additional parameter.
In the end I used the fields of my java class: Some methods save values to those fields, others read values. Well, it works, but that code is not nice. It contains shared mutable state. So cucumber drives me to use evil code style.
Summary
I’ve never experienced the benefits of cucumber.
But I’ve suffered from the overhead of cucumber and so I’m happy that I don’t need to use it.
Any comments or suggestions? Leave an issue or a pull request!