Means to keep the test duration as short as possible.
#Means to keep the test duration as short as possible.
- new tests should be always Spock tests
- Remove unused code
- Get paths form service instead of from domains. This reduces the domain instantiation time.
- Short-circuit domain reference chains by mocking or null. This reduces the total domain instantiation time.
- Don't write test-objects into the database, try to mock the domain objects first (avoid using the domain factory)
- Speed up filesystem access by using Java in-memory filesystem (requires use of the Path class instead of the File class) or -- at the OS level -- shared-memory filesystem (/dev/shm).
...
...
@@ -17,3 +17,359 @@ Means to keep the test duration as short as possible.
- There is a gradle scan tool to create a report of the test performance.
- Sort tests by test duration and try to reduce it (disadvantage: total test time depends on intrinsic complexity)
- Methods that are used by normal users shouldn't only tested as admin or operator
#Structure for tests:
- properties
- static properties
- services
- ruled properties (TemporaryFolder)
- remaining properties
- methods
- getDomainClassesToMock
- setup
- cleanup
- tests
- abstract method
- private help methods
- override methods
- help classes
Conventions for Tests
---------------------
The default testing framework in Grails was changed to Spock in recent versions. Spock's naming conventions are:
- Unit Test SHOULD end in "Spec" and MUST extend ```Specification```.
- Integration Test should end in "IntegrationSpec" and MUST extend ```IntegrationSpec```.
For "legacy" tests (JUnit), the development team agreed on the following naming conventions for tests:
- Unit tests (classes under ```test/unit```) SHOULD end in "UnitTests".
- Integration tests (classes under ```test/integration```) SHOULD end in "Tests".
Unit Testing (With Fake Objects)
--------------------------------
### The Problem ###
Unit testing can be difficult since the application needs external objects, such as GORM objects (domains), which often require external resources (such as a database).
In order to test the functionality of components (i.e. a ```Service```) the presence of real objects is not strictly needed.
Using fake objects ("mocks" or "stubs") will allow for unit tests instead of integration tests to test functionality.
The basic idea is to replace a part of the business logic with stubs that simply return results instead of doing real work.
The following sections will explain how to do this (in the order of preference of the method).
For a summary on the reasoning for this order, please refer to the subsection "Preferences" in the section "Best practices".
Please also refer to the ["Testing" section](http://docs.grails.org/3.3.9/guide/single.html#testing) of the Grails documentation for further details.
### Mocking vs. Stubbing vs. Dummies vs. … ###
Mocks and Stubs are quite similar and often the term "mock" is used when "stub" is meant.
This document uses the term "fake" if the difference does not matter.
Simply put, mocks allow for behavior testing by providing mechanisms to inspect if methods where called (and in which order).
For a playful and thorough discussion, see the articles "[The little Mocker](http://blog.8thlight.com/uncle-bob/2014/05/14/TheLittleMocker.html)" by Robert Martin (Uncle Bob) and "[Mocks Aren't Stubs](http://martinfowler.com/articles/mocksArentStubs.html)" by Martin Fowler.
### Faking with Spock ###
Spock should be used for all newly created tests and is also the default in Grails for some time.
It comes with an [extensive documentation](https://spockframework.github.io/spock/docs/1.0/index.html) that you should consult, as it contains a lot of examples.
It is recommended to at least read the [Spock Primer](https://spockframework.github.io/spock/docs/1.0/spock_primer.html) chapter, as the following sections are only a brief introduction to get you started.
It also provides its own mocking framework that is very easy to use. It should be preferred over all other methods mentioned below.
The mechanisms below are described since a lot of tests are not migrated to Spock and use at least one of those listed.
They should not be used in Spock tests, although they may be handy for complicated cases.
> **Note:** It is not possible in Spock to mock or stub (parts of) the class under test.
> The whole point of mocking is to get defined behavior of the collaborators.
> Requiring to mock parts of the class is a strong indicator of a design smell, usually a violation of the [Single Responsibility Principle](https://en.wikipedia.org/wiki/Single_responsibility_principle).
> It should be fixed by improving the design.
#### Mocking ####
Mocking is the way Spock does [Interaction Based Testing](https://spockframework.github.io/spock/docs/1.0/interaction_based_testing.html).
It allows you to check if some method you expect to be called in a collaborator is actually called (or not called at all).
Besides the number of invocations it allows to check for specific arguments and the order of method calls.
Spock is a lenient mocking framework, meaning that unexpected method calls are allowed and answered with a default response in order to prevent over-specification of a test.
(Which makes test brittle.) If you explicitly want to check that no other methods were called, the documentation lists a way to do that.
> **Note:** Beware that a mock is constructed in the original sense of the word, so the implementations of the methods of the collaborator are replaced with default implementations, usually returning null.
> If this is not what you want, you need stubbing, maybe in combination with mocking.
To create a mock, you can use one of the following syntax:
```groovy
defsubscriber=Mock(Subscriber)
```
or
```groovy
Subscribersubscriber=Mock()
```
Spock's term for an assertion on a mock is interaction.
An interaction is composed of a cardinality (how often a method is called) and the target and method constraint (which method on which object is called.
It can optionally have an argument constraint to check if the method was called with the correct value.
```
1 * subscriber.receive("hello")
| | | |
| | | argument constraint
| | method constraint
| target constraint
cardinality
```
A full example may look like the following: Two ```Subscribers``` are registred with a ```Publisher``` in order to be informed.
If the ```Publisher``` sends a message, it is supposed to call the receive method on all of its Subscribers once.
```groovy
classPublisherSpecextendsSpecification{
Publisherpublisher=newPublisher()
Subscribersubscriber=Mock()
Subscribersubscriber2=Mock()
defsetup(){
publisher.subscribers<<subscriber
publisher.subscribers<<subscriber2
}
def"should send messages to all subscribers"(){
when:
publisher.send("hello")
then:
1*subscriber.receive("hello")
1*subscriber2.receive("hello")
}
}
```
Here, an argument constrained is used to check that the received message is equal to the input message.
If the argument is no concern, an underscore (_) can used to ignore the passed argument(s).
It is also possible to use the underscore on all other constraints.
So if you do not care on which object the ```receive()``` method was called, you could write the above test like this:
```groovy
then:
2*_.receive("hello")
```
Keep in mind that these are not equivalent, though. The later version would pass the test if the ```reveive()``` method was called twice on ```subscriber``` and never on ```subscriber2```, or vice versa.
If you want to test if getters or setters where called, you can state them in Groovy's field syntax or explicitly in the interaction:
```groovy
1*subscriber.status// same as: 1 * subscriber.getStatus()
```
You can mix interactions with conditions. If you do, the convention is to put interactions first, as this:
```groovy
when:
publisher.send("hello")
then:
1*subscriber.receive("hello")
publisher.messageCount==1
```
Besides those example, a lot of other things are possible, for example declaring the interaction at mock creation time.
There are also a lot of extensions to the constraint syntax that make tests more readable.
You can check all these features out in the documentation.
#### Stubbing ####
"Stubbing is the act of making collaborators respond to method calls in a certain way."
In difference to mocking, the cardinality is not needed, since the response should be sent whenever the method is called.
Otherwise, the syntax is almost the same, except for an added _response generator_ on the right side.
Assuming the ```receive()``` method from above returns a status code when called, a stub might look like this:
```groovy
subscriber.receive(_)>>"ok"
```
This will return the constant value "ok" every time the method receive() on the subscriber object is called. As with mocking, the terminology for a stubbed interaction is almost identical:
```
subscriber.receive(_) >> "ok"
| | | |
| | | response generator
| | argument constraint
| method constraint
target constraint
```
Declaring a stub is also mostly identical:
```groovy
defsubscriber=Stub(Subscriber)
```
or
```groovy
Subscribersubscriber=Stub()
```
> **Note:** In Spock, a stub can only be used for stubbing whereas a mock can be used for mocking and stubbing. Using a stub in the declaration shows the role of that object in the test setup and should be preferred if only stubbing is needed.
A stub usually has a very limited number of interactions, so they are often declared at construction time:
```groovy
defsubscriber=Stub(Subscriber){
receive("message1")>>"ok"
receive("message2")>>"fail"
}
```
The above example shows that it is possible to return different values for different invocations using an argument constraint to differentiate between the two. Calling receive() with a message "message2" will always provide the "fail" status code.
It is also possible to provide sequences of return values that will return the next value in the list on each following invocation. You can use the triple-arrow operator for this:
This will return "ok" for the first invocation, "error" for the second and third, and "ok" for all subsequent invocations. For more advanced uses, such as computing return values based on the arguments, see the documentation.
To perform side effects such as throwing an exception, use a closure:
This will return "ok" for the first, third, fifth and all subsequent invocation, "fail" for the second, and throw an exception for the fourth.
#### Mocking and Stubbing ####
Sometimes, you need to combine both in order to provide a response and check for the correct invocation. As stated above, this is only possible with a mock. The interactions can be combined:
```groovy
1*subscriber.receive("message1")>>"ok"
1*subscriber.receive("message2")>>"fail"
```
It also means that they have to happen in the same method call.
Other mocking frameworks allow for splitting the declaration of mocking and stubbing to two different places, such as a "setup" and "assert" blocks.
This will __not__ work with Spock, as it is a lenient mocking framework. Consult the documentation for a more detailed explanation.
### Faking via Map Coercion ###
> **Note:** This section is kept for legacy tests. It does not apply to Spock tests, as better mechanisms exist. (It can be used, though.)
If all you want to do is stubbing Groovy's map coercion is usually sufficient.
It works by providing a map with properties and converting it to a proxy via the "as" keyword.
Methods can be stubbed by using the method name as key and providing a closure with the same signature as the real method as a value.
Below you will find an example that was adapted to match an example from the Grails documentation.
It stubs out the ```searchWeb()``` method in the SearchService so that the controller (the class under test) does not have to rely on a real web search in order to be tested:
```groovy
defsearchMock=[
searchWeb:{Stringq->},
]asSearchService
controller.searchService=searchMock
controller.search()
```
The drawback of this method is that it cannot be used when static methods are involved.
Also, it only stubs, so it's not possible to do behavior testing.
But for a lot of times stubbing is all that is needed.
### Faking via MetaClass ###
Another method is using fakes by modifying the MetaClass of an object.
When used with singletons such as services, this has side effects that could break unrelated tests.
It violates the principles of isolation and is discouraged. Usually one of the other methods above are sufficient.
If it is used the test has to ensure to revert the changes in the tear-down.
This method has the drawback that it only works for public methods.
To fake implementation details it's usually needed to provide a seam by making internal methods protected.
Because of this using a mocking framework is usually better and more robust as it already takes care of the nasty reflection work.
### Best Practices ###
#### Prefer not to Spy ####
Most of the times, the need to spy is pointing towards a design smell.
There might exist a better solution that does not require spy and that's the solution that should be preferred.
#### Preferences ####
The preferred way is to use the Spock framework for mocking and stubbing.
You should never mix it with the Grails Mocking Framework because this may lead to unintended side effects.
The same goes for ```MetaClass```. If you really have to use it, also use the ```@ConfineMetaClassChanges``` annotation.
For the JUnit tests, Map coercion is currently the most effective approach to create fake objects for testing OTP components in unit tests. If they are too limited you should chose the most appropriate approach. These would be (in order of preference):
1. Map Coercion. It's usually sufficient and the simplest method.
1. MetaClass (with ```@DirtiesRuntime``` annotation). Because of the side effects, it should be avoided.
1. Inheritance.
#### Avoid GroovyTestCase ####
Unit tests that run in the Grails context must not extend GroovyTestCase anymore.
This will cause problems.
Test that do not run in the Grails context (that means tests for classes that live in ```src/```) should not extend it either as it will generate a JUnit 3 test case.
This does not allow the use of ```TemporaryFolder```, for example. It also has other drawbacks. If you find such a test, consider re-writing it in Spock.
#### Using thrown() or shouldFail ####
In Spock, you should use ```thrown()``` if you expect an exception to be thrown.
The method also returns the exception for further inspection.
The ```shouldFail()``` method is injected in tests for Grails components automatically.
Code that lives outside of the Grails framework can extend ```GroovyTestCase```, as this is only a problem in Grails tests.
In Spock tests, ```thrown()``` should be used.
#### Testing custom domain constraints ####
Testing custom constraints can be done without any mocks since Grails provides a fake GORM layer in unit tests.
The tests should not call ```save()```.
They should call ```validate()``` instead and check if the errors property is set and contains the expected values.
The procedure is described with examples in the [Grails manual](https://grails.github.io/grails-doc/latest/guide/testing.html#unitTestingDomains)(subsection"testing constraints").
#### Temporary test files ####
Test files should be created using ```TemporaryFolder```.
It provides a unique base directory and takes care of clean-up. See the [JUnit documentation](https://github.com/junit-team/junit/wiki/Rules#temporaryfolder-rule) for more information.
> **Note:** This does not work with JUnit 3 tests, that is tests extending GroovyTestCase or using a mixin. Consider re-writing those.
### Recommended Reading/Watching ###
#### Books ####
- Michael Feathers: Working Effectively with Legacy Code
