Spock - Testing Exceptions with Data Tables
ExceptionTestingGroovySpockException Problem Overview
How can exceptions be tested in a nice way (e.g. data tables) with Spock?
Example: Having a method validateUser that can throw exceptions with different messages or no exception if the user is valid.
The specification class itself:
class User { String userName }
class SomeSpec extends spock.lang.Specification {
...tests go here...
private validateUser(User user) {
if (!user) throw new Exception ('no user')
if (!user.userName) throw new Exception ('no userName')
}
}
Variant 1
This one is working but the real intention is cluttered by all the when / then labels and the repeated calls of validateUser(user).
def 'validate user - the long way - working but not nice'() {
when:
def user = new User(userName: 'tester')
validateUser(user)
then:
noExceptionThrown()
when:
user = new User(userName: null)
validateUser(user)
then:
def ex = thrown(Exception)
ex.message == 'no userName'
when:
user = null
validateUser(user)
then:
ex = thrown(Exception)
ex.message == 'no user'
}
Variant 2
This one is not working because of this error raised by Spock at compile time:
Exception conditions are only allowed in 'then' blocks
def 'validate user - data table 1 - not working'() {
when:
validateUser(user)
then:
check()
where:
user || check
new User(userName: 'tester') || { noExceptionThrown() }
new User(userName: null) || { Exception ex = thrown(); ex.message == 'no userName' }
null || { Exception ex = thrown(); ex.message == 'no user' }
}
Variant 3
This one is not working because of this error raised by Spock at compile time:
Exception conditions are only allowed as top-level statements
def 'validate user - data table 2 - not working'() {
when:
validateUser(user)
then:
if (expectedException) {
def ex = thrown(expectedException)
ex.message == expectedMessage
} else {
noExceptionThrown()
}
where:
user || expectedException | expectedMessage
new User(userName: 'tester') || null | null
new User(userName: null) || Exception | 'no userName'
null || Exception | 'no user'
}
Exception Solutions
Solution 1 - Exception
The recommended solution is to have two methods: one that tests the good cases, and another that tests the bad cases. Then both methods can make use of data tables.
Example:
class SomeSpec extends Specification {
class User { String userName }
def 'validate valid user'() {
when:
validateUser(user)
then:
noExceptionThrown()
where:
user << [
new User(userName: 'tester'),
new User(userName: 'joe')]
}
def 'validate invalid user'() {
when:
validateUser(user)
then:
def error = thrown(expectedException)
error.message == expectedMessage
where:
user || expectedException | expectedMessage
new User(userName: null) || Exception | 'no userName'
new User(userName: '') || Exception | 'no userName'
null || Exception | 'no user'
}
private validateUser(User user) {
if (!user) throw new Exception('no user')
if (!user.userName) throw new Exception('no userName')
}
}
Solution 2 - Exception
Here is the solution I came up with. It's basically Variant 3, but it uses a try/catch block to avoid using Spock's exception conditions (since those have to be top level).
def "validate user - data table 3 - working"() {
expect:
try {
validateUser(user)
assert !expectException
}
catch (UserException ex)
{
assert expectException
assert ex.message == expectedMessage
}
where:
user || expectException | expectedMessage
new User(userName: 'tester') || false | null
new User(userName: null) || true | 'no userName'
null || true | 'no user'
}
Some caveats:
- You need multiple catch blocks to test different exceptions.
- You have to use explicit conditions (
assertstatements) inside of try/catch blocks. - You can't separate your stimulus and responses into
when-thenblocks.
Solution 3 - Exception
You can wrap your method call with a method that returns the message or the exception class, or a map of both...
def 'validate user - data table 2 - not working'() {
expect:
expectedMessage == getExceptionMessage(&validateUser,user)
where:
user || expectedMessage
new User(userName: 'tester') || null
new User(userName: null) || 'no userName'
null || 'no user'
}
String getExceptionMessage(Closure c, Object... args){
try{
return c.call(args)
//or return null here if you want to check only for exceptions
}catch(Exception e){
return e.message
}
}
Solution 4 - Exception
Here's how I do it, I modify the when: clause to always throw a Success exception, that way you don't need separate tests or logic to tell whether to call thrown or notThrown, just always call thrown with the data table telling whether to expect Success or not.
You could rename Success to be None or NoException or whatever you prefer.
class User { String userName }
class SomeSpec extends spock.lang.Specification {
class Success extends Exception {}
def 'validate user - data table 2 - working'() {
when:
validateUser(user)
throw new Success ()
then:
def ex = thrown(expectedException)
ex.message == expectedMessage
where:
user || expectedException | expectedMessage
new User(userName: 'tester') || Success | null
new User(userName: null) || Exception | 'no userName'
null || Exception | 'no user'
}
private validateUser(User user) {
if (!user) throw new Exception ('no user')
if (!user.userName) throw new Exception ('no userName')
}
}
One extra thing I would change, would be to use a subclass for the failure exceptions too to avoid a Success being accidentally caught when you were really expecting a failure. It doesn't affect your example because you have an extra check for the message, but other tests might just test the exception type.
class Failure extends Exception {}
and use that or some other "real" exception instead of the vanilla Exception
Solution 5 - Exception
Using the example from @AmanuelNega I had a try at this on the spock web console and saved the code at http://meetspock.appspot.com/script/5713144022302720
import spock.lang.Specification
class MathDemo {
static determineAverage(...values)
throws IllegalArgumentException {
for (item in values) {
if (! (item instanceof Number)) {
throw new IllegalArgumentException()
}
}
if (!values) {
return 0
}
return values.sum() / values.size()
}
}
class AvgSpec extends Specification {
@Unroll
def "average of #values gives #result"(values, result){
expect:
MathDemo.determineAverage(*values) == result
where:
values || result
[1,2,3] || 2
[2, 7, 4, 4] || 4.25
[] || 0
}
@Unroll
def "determineAverage called with #values throws #exception"(values, exception){
setup:
def e = getException(MathDemo.&determineAverage, *values)
expect:
exception == e?.class
where:
values || exception
['kitten', 1]|| java.lang.IllegalArgumentException
[99, true] || java.lang.IllegalArgumentException
[1,2,3] || null
}
Exception getException(closure, ...args){
try{
closure.call(args)
return null
} catch(any) {
return any
}
}
}
Solution 6 - Exception
I have solution which not distort your test workflow and you can analyze exception by content of dynamic object placed in where table
@Unroll
def "test example [a=#a, b=#b]"() {
given:
def response
def caughtEx
when:
try {
result = someAmazingFunctionWhichThrowsSometimes(a,b)
} catch (Exception ex) {
caughtEx = ex
}
then:
result == expected
if (exception.expected) {
assert caughtEx != null && exception.type.isInstance(caughtEx)
} else {
assert caughtEx == null
}
where:
a | b || exception | expected
8 | 4 || [expected: false] | 2
6 | 3 || [expected: false] | 3
6 | 2 || [expected: false] | 3
4 | 0 || [expected: true, type: RuntimeException] | null
}
Solution 7 - Exception
Here is an example of how I achieved it using @Unroll and the when:, then:, and where: blocks. It runs using all 3 of the tests with the data from the data table:
import spock.lang.Specification
import spock.lang.Unroll
import java.util.regex.Pattern
class MyVowelString {
private static final Pattern HAS_VOWELS = Pattern.compile('[aeiouAEIOU]')
final String string
MyVowelString(String string) {
assert string != null && HAS_VOWELS.matcher(string).find()
this.string = string
}
}
class PositiveNumberTest extends Specification {
@Unroll
def "invalid constructors with argument #number"() {
when:
new MyVowelString(string)
then:
thrown(AssertionError)
where:
string | _
'' | _
null | _
'pppp' | _
}
}