Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

JUnit is a test framework for software code that uses annotations to identify methods that specify a test. JUnit is an open-source project hosted at Github. A JUnit test is a method, contained in a class that is only used for testing (a.k.a Test class). This method executes the code under testing and can assert to check for the expected result versus the actual result.

Executing JUnit test cases in ZebraTester

Executing JUnit test cases with ZebraTester has been supported since ZebraTester version 5.1. A ZebraTester wizard automatically generates load test JUnit plug-ins (no manual programming needed). This allows users to run their own JUnit load tests on any load generator, providing helpful support for load generator clusters. Many thousands of JUnit test cases can run in parallel at the same time, and that all of the results are combined into one test report.

Setting up a JUnit Test

Step 1: Set up the Plugin from Within ZebraTester

Step

Illustration

1. Prepare a Session with one "Non Executable" URL Call

First, you need a recorded session that contains only one URL call (that never will be executed).

If you haven’t created any session, you can use the PageScanner tool or the Add URL to session. Or, you can load any existing session and remove all URL calls except the first one from that session and set the User's Think Time to a value of zero.

Image Modified

2. Configure the URL call as "non-executable"

For this example, we are going to take the URL call to the Apica Website and step you through the ZebraTester UI as to how to mark the call as non-executable. This is a special case as you would not normally do this except for a JUnit test case.

  1. Click on URL

  2. In the URL Details/Var Handler Edit the HTTP Request

  3. Click either of the Special Options buttons/Icons to set the flag to ON or OFF

  4. Place a check in the “Disable the execution of the URL during Load Test”

  5. Apply the Special Option

Image Removed

Image RemovedImage RemovedImage Removed

Image AddedImage AddedImage AddedImage Added

3. Return to the Main Menu

Click refresh to reload the current page.

The URL call should now be shown with a yellow exclamation mark. ⚠

Image Modified

4. Add the JUnit Packages and the Test Package to the Session

By clicking from the (1) ZebraTester Main Menu on the URL that we set as non-executable, we open the Var Handler page, and from that, we (2) look for the black folder icon that opens the Declare External Resources (for Self-Developed Plug-ins) menu.

At the Declare External Resources menu and (3) add all Java *.jar files that are needed to execute the JUnit test.

This means that you have to add the jar files of JUnit itself (for example, junit-4.11.jar and hamcrest-core-1.3.jar) and the jar file that contains your JUnit test (for example CollectionTest.jar), so you end up with a table/list of all declared external resources at the bottom of the dialog. (4)

Info

Note: We are continuing with the Apicasystems.com URL from above.

Image ModifiedImage RemovedImage RemovedImage AddedImage Added

5. Add a System Properties File (Optional)

You can also declare a System Properties File as an external resource. Such a file will be parsed on the load generators just before the JUnit tests are executed. The key and values of such a file are stored in the Java virtual machine as system properties.

(lightbulb) Hint: To debug the parsing of such a file you can enable the option "debug loops" when starting a load test. The debug output is written to the *.out file of the load test job.

Image Modified

The lines of the file can contain the following commands:

  1. key = value

  2. .includeProps = "<include-file-name>"

  3. .includeProps = ["<include-file-name-1>", "<include-file-name-2>", ... , "<include-file-name-N>"]

  • If a value ends with a backslash (\) the next line is appended to the value.

  • Lines that are staring with a hash sign (#) are ignored.

Example saved and declared as an External Resource called “SystemProperties.txt”:

# General settings

loggingBufferSize = 1000

memoryStatsEnabled = true

# JSON data

jsonRequest = {\

     "defaultID": 1, \

     "keys": [\

          {"id":1, "type":"A25", "value":"57FCCA5C9F39BC2E73B5"}, \

     ],\

}

Step 2: Generate a JUnit Load Test Plug-In

Step

Illustration

Image RemovedImage Added

Once you have loaded all External Resources described in the JUnit testing How to Create a JUnit Load Test within ALT page, navigate to the Project Navigator to the Plugins directory and call the JUnit wizard with the Icon

Image RemovedImage Added

In the wizard, enter the full class name of the JUnit test or Search for it in the “Find Class in External Resources.”

Image RemovedImage Added

Select which JUnit test methods should be executed during the load test

Image RemovedImage Added
  • Methods Execution Order
    You can select "Java/Reflection", "Alphabetical", "Randomly per Loop" or "In Parallel Threads".

  • Note: When selecting "In Parallel Threads" this means always "per simulated user". So, if you start a load test with 1,000 users containing a JUnit class with 3 test methods then 3,000 JUnit threads will be run in parallel at the same time.

  • System Properties File
    If you have declared a System Properties File as External Resource you should select it here.

  • Combine Error Types
    Similar measured errors that occur during the load test execution are grouped by ZebraTester to "Error Types" in order to get a better overview. In “Combine Error Types” you can choose how to group the Errors.
    You can select "Class + Method Name", "JUnit Failure Text", "Exception Text" or "Stack Trace".
    For example, if "JUnit Failure Text" is selected then all errors that contain exactly the same "JUnit Failure Text" will be in the same group.

  • Plug-in Class Name
    The Java class name of the (new) generated JUnit plug-in.

  • Plug-in GUI Label
    The label (text) of the plug-in that is displayed in the GUI.

  • Plug-in Input Parameter
    🚫 Do not add or remove the plug-in input parameter. Leave them "as is".

Image RemovedImage Added

Generate the JUnit Plug-In Code

Image RemovedImage Added

Compile the Load Test Plug-in

Image RemovedImage Added

Step 3: Add a JUnit Load Test Plug-In to Session

Once the Plug-In is generated, you can add it to a Load Testing Session.

Step

Illustration

Iin the Project Navigator, navigate to the directory where the load test program should be generated. In this screenshot this is called ‘MyTests’

Image RemovedImage Added

Return to the Main Menu and add the JUnit Plug-In to the desired URL Call.

Image RemovedImage Added

In the URL Details/Var Handler, locate the Load Test Plug-ins and click Add Plug-in

Image Modified

From the Var Handler, under, Add Load Test Plug-in, select the Defined Plug-ins from the dropdown box and Continue.

Image RemovedImage Added

Add the Plug-in along with any additional parameters.

Image RemovedImage Added

Note the Plug-in has been added, with any additional parameters.

Save the session using

Image Removed

Image Removed

by clicking the “Save Session” icon.

Image Added

Step 4: Generate the Load Test Program with a JUnit Load Test Plug-In

Step

Illustration

From the Main Menu, Generate Load Test

Image Modified

On the Generate Load Test Program dialog, enter your Jave Java Classname and click Continue

Image RemovedImage Added

In the Project Navigator, you’ll find your class file created in your selected directory. Click the Execute Load Test Icon.

Image RemovedImage Added

Before Executing a Load Test with a Plug-in, ZIP the Plug-ins and all external Local Resources before continuing to Executing. >>ZIP and execute

Image RemovedImage Added

All JUnit plug-ins support to configure a Pacing Time (0 = no Pacing Time).

The Pacing Time defines how much minimum elapsed-time a simulated user should spent in the plug-in.

Image RemovedImage Added

Example 1: If you configure a pacing time of 10 seconds and a simulated user executes the plug-in in 3 seconds the user will be delayed for 7 seconds in that executed loop (3 + 7 seconds = 10 seconds).

Example 2: If you configure a pacing time of 10 seconds and a simulated user executes the plug-in in 2 seconds the user will be delayed for 8 seconds in that executed loop (2 + 8 seconds = 10 seconds).

Example 3: If you configure a pacing time of 10 seconds and a simulated user executes the plug-in in 13 seconds the user will not be delayed in that loop (13 > 10 seconds)

During the execution of the load test you can examine the measured errors at real-time

Image RemovedImage Added

Under the Error Overview (Real-Time) Latest Error under the Latest Error Snapshots you can drill into any row with the magnifier icon.

Image RemovedImage Added

Drilling down into the Error log for details

Image RemovedImage Added

After the load test has been completed, and you have acquired the test result, you can display the execution time for each JUnit method

Image Removed

Generating the Load Test Program with a JUnit Load Test Plug-In (WIP)

Generating the Load Test Program and Executing the Load Test can be done as normal, however, there will be additional steps to gather/zip the external resource files.

JUnit Best Practices (WIP)

JUnit Test Automation

Filter by label (Content by label)
showLabelsfalse
max5
spacescom.atlassian.confluence.content.render.xhtml.model.resource.identifiers.SpaceResourceIdentifier@1fe877
sortmodified
showSpacefalse
reversetrue
typepage
cqllabel in ( "zebratester" , "testing" , "junit" ) and type = "page" and space = "DAZT"
labelszebratester junit testing

...

hiddentrue

...

Image Added

JUnit Best Practices

Configuring Timeouts for all JUnit Test Methods

In order to prevent a blocking/freezing of a JUnit load test, Apica strongly recommends that you configure a timeout for each test method for all your JUnit test cases. You can use large values for such timeouts as 15,000 milliseconds.

Example: 

Code Block
@Test(timeout=15000)
public void sortedSet() {
…

Passing User-Specific Variables to JUnit Tests

JUnit by itself does not support passing any (local) per test-run specific variables to a test method.

However, as an example, when testing a Web service, it's often required that for each simulated user or each JUnit test run, we need to use a different username and password for authentication. So how do we do this?

The easiest way is to use a Text-File containing the values of such per test-run specific variables. Example:

Code Block
Miller; abcd
Meier; topsecret
Sutter; password

 

To support this example, you should modify the program code of the JUnit test case as follows:

Code Block
import java.io.*;
import dfischer.utils.Base64Decoder;
import dfischer.utils.StaticInputFile;  // defined in prxsniff.jar – see also ZebraTester Java API Doc
import dfischer.utils.StaticInputFileLineContent;
import dfischer.utils.AbstractInputFileReader;
import dfischer.utils.SymmetricEncryptContext;
...

@Test(timeout=15000)

public void sortedSet() {
  String username = "Miller"; // default value, used for non load test execution
  String password = "abcd"; // default value, used for non load test execution
  if ((System.getProperty("PrxJUnitLoadTest") != null) && System.getProperty("PrxJUnitLoadTest").equalsIgnoreCase("true")) {
try {
        SymmetricEncryptContext decryptContext = null;
       String decryptContextProperty = System.getProperty("PrxSymmetricEncryptContext");
       if (decryptContextProperty != null) {
          decryptContext = SymmetricEncryptContext.fromSerializedByteArray(Base64Decoder.decodeToBytes(decryptContextProperty));
       }
      int fileID = StaticInputFile.concurrentOpen(decryptContext, "Accounts.txt", "#", ";", true, AbstractInputFileReader. EOF_REOPEN_FILE);
      StaticInputFileLineContent line = StaticInputFile.getNextLine(fileID);
      username = line. getValue(0);
      password = line. getValue(1);
      }
      catch (IOException ie) {
  throw new RuntimeException("failed to read Accounts.txt", ie);
      }
    }
}

...

Note that the System property "PrxJUnitLoadTest" =  "true" is automatically set by ZebraTester when executing a JUnit load test.
The file containing the user accounts must also be declared as an external resource:

...

When using a cluster of load generators you can also split the file across the cluster members:

...

Modifying a JUnit Plug-In

To modify an already generated JUnit Plug-In navigate to the Plugins directory and click on the corresponding Template File:

...

After you have modified the plug-in, generate and compile the plug-in once again.

Note

Please note that you have to confirm in Project Navigator that the old plug-in code should be overwritten and compiled (in two steps).

...

After these steps, navigate to the directory where the load test program is located and update the plug-in in this directory:

...

Once you Update the plug-in, the load test can be directly started.

Info

There is no need to generate and compile the load test program once again in such a case. This is only needed if you declare additional external resources in the load test program.

Integration with Performance Monitoring Agent

The JUnit load test measured data can also be combined with the measured Operating System performance data on any machine where the Apica Performance Agent is installed (e.g. on Application Servers and on Database Servers). In order to accomplish this, you must select the corresponding "Monitoring Controller Template" when starting the JUnit load test:

...

The captured system performance data will also be added to the "External Measured Data":

...

JUnit Test Automation

There are four tools available that allow you to automatically generate ready-to-run load tests by using only a “JUnit Load Test Plug-In Template” as required input. The “JUnit Load Test Plug-In Template” contains information about which JUnit test cases should be executed during the load test.

Info

The JUnit test automation is fully documented in the “ZebraTester Application Reference Manual”, chapter 13 “JUnit Load Test Automation Tools”.