0 votes

I am trying to generate test runs from multiple test cases containing different variants. I have tried a combination of include and exclude specific variants and variations of only include or exclude. I only want to generate test runs for the variant field code used in the test case so I'm expecting to generate 20 tests from these 2 test cases but instead I get the following:

The top 2 test cases are including Login Process and Roles variants for the first and only Platform for the second. All the other variants are excluded, counting the included variants for each test. When I select all the included variants I get 0 test runs will be generated.

When I only use the included variants and use the same selection above it generates 128 test runs. Then if I only use exclude only it will I get the same result as above. If I choose one or the other I get the expected number of tests to generate...

 

Is what I am trying to do even possible or am I doing something wrong in the variant setup either in the test case or test variant?

asked May 10, 2018 in TestTrack by twstahl (190 points)

1 Answer

+1 vote
Best answer

The typical use case for including/excluding test variants is to control which test variants apply to the test case, but it assumes that at least one test variant value will apply to every test case. For example, you might have an OS variant with values of Windows, Linux, & Mac. If a test case only ever applies to Mac, then you would “include only Mac variant” because new OS values would not apply. If a test case applies to all OS values except one, you can use the exclude option.

You have an interesting use case. It sounds like you want to exclude a test variant field entirely from a test case, saying that this test variant does not make sense for this test case. There is no option to do that in Helix ALM, so you are trying to use the exclude option to exclude all test variant values. The following example shows why this is not working.

Setup test variant TV1 with values of TV1a and TV1b. Setup a second test variant TV2 with values of TV2a and TV2b. Create a test case TC1 that excludes variants TV1a and TV1b. Create a test case TC2 that excludes variants TV2a and TV2b. Generate test runs for TC1 and TC2 using test variant options of <All> for TV1 and TV2. Helix ALM will attempt to generate the following test runs.

  • Test run for TC1 using TV1a & TV2a (invalid because TC1 excludes TV1a)
  • Test run for TC1 using TV1a & TV2b (invalid because TC1 excludes TV1a)
  • Test run for TC1 using TV1b & TV2a (invalid because TC1 excludes TV1b)
  • Test run for TC1 using TV1b & TV2b (invalid because TC1 excludes TV1b)
  • Test run for TC2 using TV1a & TV2a (invalid because TC2 excludes TV2a)
  • Test run for TC2 using TV1a & TV2b (invalid because TC2 excludes TV2b)
  • Test run for TC2 using TV1b & TV2a (invalid because TC2 excludes TV2a)
  • Test run for TC2 using TV1b & TV2b (invalid because TC2 excludes TV2b)

This shows that there are no valid test variant values for the selected options, which is why Helix ALM reports that zero test run records will be generated.

One possible solution is to add a <Does not apply> value to your test variant lists. So in the example above, the test variant TV1 would have values of TV1a, TV1b, and <Does not apply>. Test variant TV2 would have values of TV2a, TV2b, and <Does not apply>. Test case TC1 would include TV1 value of <Does not apply> and exclude TV2 value of <Does not apply>. Test case TC2 would include TV2 value of <Does not apply> and exclude TV1 value of <Does not apply>. Now when we generate test runs for TC1 and TC2 using test variant options of <All> for TV1 and TV2, Helix ALM will generate the following four test runs, which are all valid.

  • Test run for TC1 using <Does not apply> & TV2a
  • Test run for TC1 using <Does not apply> & TV2b
  • Test run for TC2 using TV1a & <Does not apply>
  • Test run for TC2 using TV1b & <Does not apply>
answered May 10, 2018 by RichC (5,950 points)
selected May 10, 2018 by twstahl

I will give this a try and let you know.

 

This worked perfectly I was able to generate 20 test runs!

248 questions
262 answers
106 comments
2,047 users