This page discusses class loading and forking under Maven Surefire which is a shared component used by both the Surefire and Failsafe Maven plugins, with an eye towards troubleshooting problems.
If you're having problems, you'll probably want to tinker with these three settings: forkCount, useSystemClassLoader, and useManifestOnlyJar.
Initially, the problem seems simple enough. Just launch Java with a classpath, like this:
java -classpath foo.jar:bar.jar MyApp
But there's a problem here: on some operating systems (Windows), there's a limit on how long you can make your command line, and therefore a limit on how long you can make your classpath. The limit is different on different versions of Windows; in some versions only a few hundred characters are allowed, in others a few thousand, but the limit can be pretty severe in either case.
It turns out setting the CLASSPATH as an environment variable may remove most of the practical length limitations, as documented in SUREFIRE-727. This means most of the length-related problems in this article may be outdated.
There are two "tricks" you can use to workaround this problem; both of them can cause other problems in some cases.
1. Isolated Class Loader: One workaround is to use an isolated class loader. Instead of launching MyApp directly, we can launch some other application (a "booter") with a much shorter classpath. We can then create a new java.lang.ClassLoader (usually a java.net.URLClassLoader) with the desired classpath configured. The booter can then load up MyApp from the class loader; when MyApp refers to other classes, they will be automatically loaded from our isolated class loader.
The problem with using an isolated class loader is that your classpath isn't really correct, and some applications can detect this and object. For example, the system property java.class.path won't include your jars; if your application notices this, it could cause a problem.
There's another similar problem with using an isolated class loader: any class may call the static method ClassLoader.getSystemClassLoader() and attempt to load classes out of that class loader, instead of using the default class loader. Classes often do this if they need to create class loaders of their own. Unfortunately, Java-based web application servers like Jetty, Tomcat, BEA WebLogic and IBM WebSphere are very likely to try to escape the confines of an isolated class loader.
2. Manifest-Only JAR: Another workaround is to use a "manifest-only JAR." In this case, you create a temporary JAR that's almost completely empty, except for a META-INF/MANIFEST.MF file. Java manifests can contain attributes that the Java virtual machine will honor as directives. For example, you can have a Class-Path attribute, which specifies a list of other JARs to add to the classpath. So then you can run your code like this:
java -classpath booter.jar MyApp
This is a bit more realistic, because in this case the system class loader, the thread context class loader and the default class loader are all the same; there's no possibility of "escaping" the class loader. But this is still a weird simulation of a "normal" classpath, and it's still possible for apps to notice this. Again, java.class.path may not be what you'd expect ("why does it contain only one jar?"). Additionally, it's possible to query the system class loader to get the list of jars back out of it; your application may be confused if it finds only our booter.jar there!
If your application tries to interrogate its own class loader for a list of JARs, it may work better under an isolated class loader than it would with a manifest-only JAR. However, if your application tries to escape its default class loader, it may not work under an isolated class loader at all.
One advantage of using an isolated class loader is that it's the only way to use an isolated class loader without forking a separate process, running all of the tests in the same process as Maven itself. But that itself can be pretty risky, especially if Maven is running embedded in your IDE!
Finally, of course, you could just try to wire up a plain old Java classpath and hope it's short enough. In the worst case your classpath might work on some machines and not others. Windows boxes would behave differently from Linux boxes; users with short user names might have more success than users with long user names, etc. For this reason, we chose not to make the basic classpath the default, though we do provide it as an option (mostly as a last resort).
Surefire provides a mechanism for using multiple strategies. The main parameter that determines this is called useSystemClassLoader. If useSystemClassLoader is true, then we use a manifest-only JAR; otherwise, we use an isolated class loader. If you want to use a basic plain old Java classpath, you can set useManifestOnlyJar=false which only has an effect when useSystemClassLoader=true.
The default value for useSystemClassLoader changed between Surefire 2.3 and Surefire 2.4, which was a pretty significant change. In Surefire 2.3, useSystemClassLoader was false by default, and we used an isolated class loader. In Surefire 2.4, useSystemClassLoader is true by default. No value works for everyone, but we think this default is an improvement; a bunch of hard-to-diagnose bugs get better when we useSystemClassLoader=true.
Unfortunately, if useSystemClassLoader is set incorrectly for your application, you're going to have a problem on your hands that can be quite difficult to diagnose. You might even be forced to read a long documentation page like this one. ;-)
If you're having problems loading classes, try setting useSystemClassLoader=false to see if that helps. You can do that with the POM snippet below, or by setting -Dsurefire.useSystemClassLoader=false. If that doesn't work, try setting useSystemClassLoader back to true and setting useManifestOnlyJar to false.
<project>
[...]
<build>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-failsafe-plugin</artifactId>
<version>2.21.0</version>
<configuration>
<useSystemClassLoader>false</useSystemClassLoader>
</configuration>
</plugin>
</plugins>
</build>
[...]
</project>
If you've read this far, you're probably fully equipped to diagnose problems that may occur during class loading. Here's some general tips to try: