Testing

The student will learn:

At the conclusion of this lecture and related exercises, the student will be able to:

Unfortunately, you will see the terms "unit" and "integration" used differently as we go through the testing topics and span tooling. There is a conceptual way of thinking of testing and a technical way of how to manage testing to be concerned with when seeing these terms used:

Conceptual - At a conceptual level, we simply think of unit tests dealing with one subject at a time and involve varying levels of simulation around them in order to test that subject. We conceptually think of integration tests at the point where multiple real components are brought together to form the overall set of subjects — whether that be vertical (e.g., to the database and back) or horizontal (e.g., peer interactions) in nature.

Test Management - At a test management level, we have to worry about what it takes to spin up and shutdown resources to conduct our testing. Build systems like Maven refer to unit tests as anything that can be performed within a single JVM and integration tests as tests that require managing external resources (e.g., start/stop web server). Maven runs these tests in different phases — executing unit tests first with the Surefire plugin and integration tests last with the Failsafe plugin. By default, Surefire will locate unit tests starting with "Test" or ending with "Test", "Tests", or "TestCase". Failsafe will locate integration tests starting with "IT" or ending with "IT" or "ITCase".

Maven runs these tests in different phases — executing unit tests first with the Surefire plugin and integration tests last with the Failsafe plugin. By default, Surefire will locate unit tests starting with "Test" or ending with "Test", "Tests", or "TestCase". Failsafe will locate integration tests starting with "IT" or ending with "IT" or "ITCase".

Neither tools like JUnit or the IDEs care how are classes are named. However, since our goal is to eventually check these tests in with our source code and run them in an automated manner — we will have to pay early attention to Maven Surefire and Failsafe naming rules while we also address the conceptual aspects of testing.

I will try to use the following terms to mean the following:

That means to not be surprised to see a conceptual integration test bringing multiple real components together to be executed during the Maven Surefire test phase if we can perform this testing without the resource management of external processes.

If we take a look at the transitive dependencies brought in by spring-boot-test-starter, we see a wide array of choices pre-integrated.

At a high level:

The success and simplicity of JUnit 4 made it hard to incorporate new features. JUnit 4 was a single module/JAR and everything that used JUnit leveraged that single jar.

The next iteration of JUnit involved a total rewrite — that separated the overall project into three (3) modules.

The JUnit 5 modules have several JARs within them that separate interface from implementation — ultimately decoupling the test code from the core engine.

The following example output shows the lifecycle of the setup and teardown methods combined with two test methods. Note that:

The following example output shows the lifecycle of the setup/teardown methods combined with two test methods. The default logger formatting added the new lines in between tests.

State used by tests can be expensive to create or outside the scope of individual tests. JUnit allows this state to be initialized and shared between test methods using one of two test instance techniques using the @TestInstance annotation.

There are often times during an integration test where shared state (e.g., injected components) is only available once the test case is instantiated. We can make instance state sharable by using the PER_CLASS option. This makes the test case injectable by the container.

There are three to four primary general purpose assertion libraries available for us to use within the spring-boot-starter-test suite before we start considering data format assertions for XML and JSON or add custom libraries of our own:

The built-in JUnit assertions are functional enough to get any job done. The value in using the other libraries is their ability to express the assertion using natural-language terms without using a lot of extra, generic Java code.

The following example shows a small peek at the syntax for each of the four assertion libraries used within a JUnit Jupiter test case. They are shown without an import static declaration to better see where each comes from.

Assertions will report a generic message when they fail. If we change the expected result of the example from 2 to 3, the following error message will be reported. It contains a generic message of the assertion failure (location not shown) without context other than the test case and test method it was generated from (not shown).

The above examples showed several ways to assert the same thing with different libraries. However, evaluation would have stopped at the first failure in each test method. There are many times when we want to know the results of several assertions. For example, take the case where we are testing different fields in a returned object (e.g., person.getFirstName(), person.getLastName()). We may want to see all the results to give us better insight for the entire problem.

JUnit Jupiter and AssertJ support testing multiple assertions prior to failing a specific test and then go on to report the results of each failed assertion.

JUnit Jupiter and AssertJ provide direct support for inspecting Exceptions within the body of the test method. Surprisingly, Hamcrest offers no built-in matchers to directly inspect Exceptions.

AssertJ has built-in support for date assertions. We have to add a separate library to gain date matchers for Hamcrest.

The second approach ("test double") has a few options:

spring-boot-starter-test brings in a pre-integrated, mature open source mocking framework called called Mockito. See the example below for an example unit test augmented with mocks using Mockito. It uses a simple Java Map<String, String> to demonstrate some simulation and inspection concepts. In a real unit test, the Java Map interface would stand for:

There is also a strong push to express acceptance criteria in code that can be executed versus a document. Although far from a perfect solution, JUnit, AssertJ, and Mockito do provide some syntax support for BDD-based testing:

When we run our test — the following natural-language text is displayed.

We can define a global setting for the display name generator using junit-platform.properties

This can also be used to express:

The unit test is being expressed in terms of BDD conventions. It is broken up into "given", "when", and "then" blocks and highlighted with use of BDD syntax where provided (JUnit and AssertJ in this case).

The following example moves up a level in the hierarchy and forces us to test a class that had a dependency. A pure unit test would mock out all dependencies — which we are doing for TipCalculator.

The final unit test example shows how we can leverage Mockito to instantiate our subject(s) under test and inject them with mocks. That takes over at least one job the @BeforeEach was performing.

There are two primary things that will change with our Spring Boot integration test:

To obtain a Spring context and leverage the auto-configuration capabilities of Spring Boot, we can take the easy way out and annotate our test with @SpringBootTest. This will instantiate a default Spring context based on the configuration defined or can be found.

By default, Spring Boot will look for a class annotated with @SpringBootConfiguration that is present at or above the Java package containing the test. Since we have a class in a parent directory that represents our @SpringBootApplication and that annotation wraps @SpringBootConfiguration, that class will be used to define the Spring context for our test.

Alternatively, we could have made an explicit reference as to which class to use if it was not in a standard relative directory or we wanted to use a custom version of the application for testing.

Assuming the components required for test is known and a manageable number…​

We can explicitly reference component classes needed to be in the Spring context.

Prior to adding the Spring context, Spring Boot configuration and logging conventions were not being enacted. However, now that we are bringing in a Spring context — we can designate special profiles to be activated for our context. This can allow us to define properties that are more relevant to our tests (e.g., expressive log context, increased log verbosity).

When we run our test we get the following console information printed. Note that

The @SpringBootTest annotation is a general purpose test annotation that likely will work in many generic cases. However, there are other cases where we may need a specific database or other technologies available. Spring Boot pre-defines a set of Test Slices that can establish more specialized test environments. The following are a few examples:

The Maven Surefire plugin looks for classes that have been compiled from the src/test/java source tree that have a prefix of "Test" or suffix of "Test", "Tests", or "TestCase" by default. Surefire starts up the JUnit context(s) and provides test results to the console and target/surefire-reports directory.

Surefire is part of the standard "jar" profile we use for normal Java projects and will run automatically. The following shows the final output after running all the unit tests for the module.

Consult online documentation on how Maven Surefire can be configured. However, I will demonstrate at least one feature that allows us to filter tests executed.

One new JUnit Jupiter feature is the ability to categorize tests using @Tag annotations. The following example shows a unit integration test annotated with two tags: "springboot" and "tips". The "springboot" tag was added to all tests that launch the Spring context. The "tips" tag was added to all tests that are part of the tips example set of components.

We can use the tag names as a "groups" property specification to Maven Surefire to only run matching tests. The following example requests all tests tagged with "tips" but not tagged with "springboot" are to be run. Notice we have fewer tests executed and a much faster completion time.

The Maven Failsafe plugin looks for classes compiled from the src/test/java tree that have a prefix of "IT" or suffix of "IT", or "ITCase" by default. Like Surefire, Failsafe is part of the standard Maven "jar" profile and runs later in the build process. However, unlike Surefire that runs within one Maven phase (test), Failsafe runs within the scope of four Maven phases: pre-integration-test, integration-test, post-integration-test, and verify

Aside from the integration tests, all other processes are normally started and stopped through the use of Maven plugins. Multiple phases are required for IT tests so that:

With the robust capability to stand up a Spring context within a single JVM, we really have limited use for Failsafe for testing Spring Boot applications. The exception for that is when we truly need to interface with something external — like stand up a real database or host endpoints in Docker images. I will wait until we get to topics like that before showing examples. Just know that when Maven references "integration tests", they come with extra hooks and overhead that may not be technically needed for integration tests — like the ones we have demonstrated in this lesson — that can be executed within a single JVM.

In our example Spring context, we will have a TipCalculator component located using a component scan. It will have the name "standardTippingImpl" if we do not supply an override in the @Component annotation.

That bean gets injected into BillCalculatorImpl.tipCalculator because it implements the required type.

Our intent here is to manually write a stub and have it replace the TipCalculator from the application’s Spring context.

We can have the @TestConfiguration class automatically found using an embedded static class.

Alternatively, we can place the configuration in a separate/stand-alone class.