This API is meant to provide a series of convenience tools for developers wishing to test Maven plugins. The primary focus of the library is on integration testing, since it is not well-supported with current tools and approaches.
Maven plugins represent a somewhat special need in terms of integration testing. Like many component-oriented systems, Maven provides much of the runtime context essential for testing a plugin in its "natural environment". This means that, in order to properly test the integration of any particular plugin into the larger system, the developer should really execute Maven builds that test particular features of the plugin in question.
However, running test builds from within the plugin's own build presents some special problems. Since we're using Maven to build the Maven plugin, we have to deal with the fact that a previous version of the plugin could be active within the current build (not a great practice, but sometimes unavoidable). This means that Maven must either be smart enough to unload the existing plugin-version from the runtime context and load the newer version - and correspondingly, unload the test version and reload the older version for other, later builds in the same reactor. As an alternative, the integration-testing of the plugin could fork separate Maven builds, and track the output in order to verify results and determine whether the build succeeds. The latter approach, while much slower and somewhat dependent on local-system configuration, is currently the most robust way to avoid existing versions of the plugin under test.
In addition to plugin-version inconsistencies, integration-testing of Maven plugins must cope with the need to cleanup after themselves. Remember, Maven only loads plugins from its own local repository (possibly after resolving them from a remote repository). Because of this, the only practical way to use the current plugin within an integration-test build is to install it into some sort of local repository, then direct Maven to use that local repository for the build. To avoid side effects in other builds (remember, we're building Maven plugins here...and they're used to build other projects), it's critical to avoid using the main local repository used for normal Maven builds. Therefore, plugin integration tests require a custom local repository, to which the integration-testing builds can be directed. Further complicating this testing scenario is the fact that the plugin's parent POM or one of its ancestor POMs could be a snapshot which has been installed in the main local repository, but which is not reachable using the plugin-POM's <relativePath/> parent-element.+
+ All of this seems to indicate that the best approach for ensuring the presence of all required POMs and plugins is to use the normal local repository to construct a new local repository for testing purposes. Work on this is ongoing.
Finally, in order to support integration-testing of plugins, it's important to give the developer an API by which to bootstrap the testing environment (and ideally, detect when it's already been bootstrapped), and to also provide a robust API for executing builds and verifying results.
Since failure to grasp the use cases of any system is a virtual guarantee that it will not meet the needs of the user, it's important to understand how integration tests are really used.
First and foremost, integration tests are used as a one-way valve, to ensure that no feature of the application - in our case, a Maven plugin - regresses as the result of unrelated development. A good set of tests - of which the integration tests are, well, integral - can provide the developer with freedom to implement new features with the confidence that he's able to effectively build on the foundations of previous versions.
However, it's important to understand that regression-prevention is not the sole purpose of an integration test. A secondary but critical use case deals with how integration tests are added to the plugin. One has only to consider how bugs are reported on a flawed system to understand. Since users usually report bugs, bug reports originate outside of the system. In the case of a Maven plugin, this means users are most likely able to supply examples of the failing behavior in the form of failing builds. In order to make immediate and direct use of what the user gives us, it's critical that integration tests are designed to be augmented using build examples from the user, with as little rewriting as possible.
To that end, the approach for integration testing in this library will be the execution of sample builds that use the plugin in question, then verify the results, either in the logs, the resulting exit code, or the project files it produces.
To summarize, this library strives to satisfy the following requirements for integration testing of Maven plugins:
For more information on the APIs described below, see the JavaDocs .
In order to make test projects more resilient to changes in the plugin's version, this tool allows the developer to specify a testing version for the plugin, so any reference to the plugin can be unambiguous and avoid accidentally referring to a plugin version resolved from outside.
