Run "Cucumber-JVM Starter: Java with Maven" from src/main and via maven-failsafe

[hennzen]

I am tidying up an existing Java-JUnit-Cucumber-Playwright project from its convoluted (but working) setup using "Cucumber-JVM Starter: Java with Maven" as a starting point with latest versions.

From this working example I dig my way forward to get the following configuration right:

• run it from within maven-failsafe-plugin
  (with goals being integration-test and verify)
• run RunCucumberIT.java instead of RunCucumberTest.java
  (to stick with failsafe's conventions)
• run from within src/main instead of src/test
  (reason being: our test app is an app itself with Unit tests of its own living under src/test)
• use @SelectPackages("io.cucumber.skeleton") as suggested in skeleton app
  (had been using @SelectClasspathResource before but with latest Cucumber 7.34.2 I get a non critical warning saying WARNING: TestEngine with ID 'junit-platform-suite' encountered a non-critical issue during test discovery: (1) [WARNING] [cucumber] The classpath resource selector 'features' should not be used to select features in a package. Use the package selector with 'features' instead (via @Suite com.mycompany.RunCucumberIT).)
  For a fresh setup I desperately try to NOT have any warnings!)

Running mvn verify won't disover any tests.

[INFO] -------------------------------------------------------
[INFO]  T E S T S
[INFO] -------------------------------------------------------
[INFO] Running io.cucumber.skeleton.RunCucumberIT
[ERROR] Tests run: 1, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 0.005 s <<< FAILURE! -- in io.cucumber.skeleton.RunCucumberIT
[ERROR] io.cucumber.skeleton.RunCucumberIT -- Time elapsed: 0.005 s <<< ERROR!
org.junit.platform.suite.engine.NoTestsDiscoveredException: Suite [io.cucumber.skeleton.RunCucumberIT] did not discover any tests
        at java.base/java.util.stream.ForEachOps$ForEachOp$OfRef.accept(ForEachOps.java:184)
        at java.base/java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:197)
        at java.base/java.util.Iterator.forEachRemaining(Iterator.java:133)
        at java.base/java.util.Spliterators$IteratorSpliterator.forEachRemaining(Spliterators.java:1939)
        at java.base/java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:509)
        at java.base/java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:499)
        at java.base/java.util.stream.ForEachOps$ForEachOp.evaluateSequential(ForEachOps.java:151)
        at java.base/java.util.stream.ForEachOps$ForEachOp$OfRef.evaluateSequential(ForEachOps.java:174)
        at java.base/java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
        at java.base/java.util.stream.ReferencePipeline.forEach(ReferencePipeline.java:596)

[INFO] 
[INFO] Results:
[INFO] 
[ERROR] Errors: 
[ERROR]   RunCucumberIT » NoTestsDiscovered Suite [io.cucumber.skeleton.RunCucumberIT] did not discover any tests
[INFO] 
[ERROR] Tests run: 1, Failures: 0, Errors: 1, Skipped: 0
[INFO] 
[INFO] 
[INFO] --- failsafe:3.5.4:verify (default) @ cucumber-java-skeleton ---
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------

You can find my setup here.

What I changed: cucumber/cucumber-jvm-starter-maven-java@3e4d952

Is this expected behaviour for my current setting?

[mpkorstanje]

You have configured your classpath such that feature files are present twice. Cucumber complains about this, but that complaint is swallowed due to junit-team/junit-framework#5327.

Specifically the complaint is:

Found 2 resources named io/cucumber/skeleton/belly.feature on the classpath [
  file:///home/mpkorstanje/Projects/github/hennzen/cucumber-jvm-starter-maven-java-main-failsafe/target/classes/io/cucumber/skeleton/belly.feature, 
  jar:file:///home/mpkorstanje/Projects/github/hennzen/cucumber-jvm-starter-maven-java-main-failsafe/target/cucumber-java-skeleton-0.0.1.jar!/io/cucumber/skeleton/belly.feature
].

As you can see: the features are available once by setting testClassesDirectory and once by the jar file that the integration tests run against. You can verify this for yourself by putting a breakpoint in the catch block of org.junit.platform.engine.support.discovery.EngineDiscoveryRequestResolution.resolveCompletely.

(reason being: our test app is an app itself with Unit tests of its own living under src/test

It might be worth considering using JUnit Launcher API instead of the Suite Engine. It is more suited to programmatic usecases.

[hennzen]

First of all, thanks for the elaborate reply!

Features being available twice I reckon was because of src/main. So in my own app I moved RunCucumberIT.java and all of the glue code to src/test, used @SelectPackages("com.mycompany.features"), removed the <testClassesDirectory> from maven-failsafe's configuration and snap everything's working.

Now I have to check more seriously with my coworker why src/main is so essential, because having Unit tests (via maven-surefire) AND E2E-tests (via maven-failsafe) both living in src/main is absolutely no problem at all.

I skimmed through the JUnit Launcher API docs and couldn't distill a minimal working example right away (have to admit that, as a frontend/JS dev, my experience with the JUnit universe is very thin yet).

If I cannot solve that src/main thing organisationally, my options are

• dive into JUnit Launcher API
• continue using @SelectClasspathResource("features") and live with that Cucumber warning about using package selector instead (s.o.)

[mpkorstanje]

The suggestion to use the JUnit Launcher API is as a replacement for whatever system is currently used to start the tests. But that was assuming you deployed the tests as an app. Now at this point I'm a bit confused about what your actual setup might be.

I wouldn't continue to use @SelectClasspathResource. The warning is effectively a deprecation warning.