Configure parallelism on features and scenarios level in the same time.

[DJ-Glock]

I use cucumber with junit.

I have several feature files. Each feature file is designed to test one application instance: A.feature, B.feature, C.feature. Each feature file contains one scenario template(outline) with N examples (usually more than 10).
Application instances are independet from each other. And each application instance can process input data in M threads, usually 2. So it does not make sense to run scenarios for particular application instance in more than M (2) threads, because it will lead to flaky tests and unstable execution time.
In the same time running scenarios for one instance (with execution-mode.feature=same_thread) in one thread can take much time.

What do I want to achieve:
Run feature files in parallel.
Run scenarios from particular feature file in parallel, but limited to 2 simultaneously running scenarios.
Overall threads limit should be equal to CPUs count or fixed pool size, let's say 6 (parallel.config.fixed.max-pool-size=6)

Example:

Being executed simultaneusly:

A.feature
- scenario 1
- scenario 2

B.feature
- scenario 1
- scenario 2

C.feature
- scenario 1
- scenario 2

Waiting in the queue:

A.feature
- scenario 3
- scenario 4
...
- scenario 10

B.feature
- scenario 3
- scenario 4
...
- scenario 10

C.feature
- scenario 3
- scenario 4
...
- scenario 10

Graphical visualisation for better understanding:

I suppose that it may be achieved using CustomStrategy:
https://github.com/cucumber/cucumber-jvm/tree/main/cucumber-junit-platform-engine#parallel-execution

To control properties such as the desired parallelism and maximum parallelism, Cucumber supports JUnit 5s ParallelExecutionConfigurationStrategy. Cucumber provides two implementations: dynamic and fixed that can be set through cucumber.execution.parallel.config.strategy. You may also implement a custom strategy.

But it could be tricky especially with the fact that I do not understand for now, at what point of time these properties are being set. And could they be changed dynamically (another words based on feature files configuration, not only pom.xml).
My idea was to try using tags on scenario level, something like @feature=A, @feature=B etc to limit parallelism for pool of scenarios. But I'm on the beginning of my journey, so need an advice if it's possible or not.

Question on SO: https://stackoverflow.com/questions/77388756/configure-parallelism-on-features-and-scenarios-level-in-the-same-time-cucumbe

[DJ-Glock]

As discussed on SO with @mpkorstanje, it's not possible at the moment and requires additional development on JUnit5 side.

JUnit 5 handles parallel execution for Cucumber. What you are looking to do is currently not possible. While JUnit 5 does support exclusive resources, it does not support resource pools. You would have to ask the JUnit 5 team if they would support this and possibly provide a pull request.

But you could implement this in your own code with something like https://commons.apache.org/proper/commons-pool/

Then for each system you're testing against, the pool should contain 2 tokens.

@Resource1
Feature: Example 1

Scenario Outline: Examples

@resource2
Feature: Example 2

Scenario Outline: Examples
Then before starting the scenario you block and acquire the resources from the pool.

static ObjectPool<?> pool = ...

@Before
public void before(Scenario scenario) {
    var tags = scenario.getTags(); // Will contain either @Resource1 or @Resource2.

    // block and acquire a resource from the pool
}

@After
public void after(Scenario scenario) {
    var tags = scenario.getTags();

    // return a resource to the the pool
}

You would have quite a few threads waiting to acquire a resource from the pool, and definitely have more threads then CPUs, which is not pretty, but I do reckon it is manageable.

Note: Haven't used this myself.

[mpkorstanje]

To express this in JUnit Jupiter terms, this would be equivalent to running at most 2 test methods per class in parallel while still running all classes in parallel.

[DJ-Glock]

This discussion was trasfered into the issue #3524