1. Assertions - HTTP Response Verification & Actions on Failure

Introduction to Response Verification

Avoid executing tests without controlling the received content of URL calls by comparing them to the originally recorded data.

Many errors from Web server applications are embedded inside valid HTTP 200 responses. Therefore, the content of the responses must be also be checked to detect content errors under load.

For this reason, ZebraTester examines the content of all recorded URL calls and automatically applies a content check per each URL call using a heuristic algorithm. This algorithm performs content checks by searching for an ASCII-text string inside the received content; however, if this seems impossible, or if this doesn't seem to make sense, the received content is checked by its size (content length) instead of by searching an ASCII-text string.

So you can use one (or more) variables from the Var Handler as part of a test-string on the URL using the Response Verification view.

Icon/View

Screenshot

Icon/View

Screenshot

Default View: Waterfall

There will be two alternate views of the recorded data; the default is a waterfall view of the URL Calls, with the specific recorded information for each row.

 

The other is the Alternate Response Verification View that will test the responses against the parameters you set.

An eye icon-toggle, found in the upper right, indicates if you are in the waterfall view (grey eye) or the response Verification View (green eye). If you are in the waterfall view (Above), click the grey eye icon to access the “Alter View,” which is short for the Alternate Response Verification View.

The Response Verification View

This will produce a view with a rightmost column titled: “Response Verification.”

In this view, look for the magnifier icon on each line that will open the HTTP Response Verification & Failure Action Dialog.

Close-up view of hovering over the magnifier icon. Notice each line has its own icon:

HTTP Response Verification & failure Action

After clicking the magnifier icon, this dialog will set what parameters and thresholds to verify/fail a returned response.

Note that you can set this for one URL or all URLs that match a specific content type.

The Response Verification Options

You select certain responses and then use them as verifications (with the above options) that will be recalled (and applied) when generating the load test program execution.

All response verification modifications are stored together with the recorded web surfing session and reapplied if the web surfing session is reloaded.

During the execution of a load test program, the HTTP response code and the received MIME type of each URL call are always compared with the originally-recorded response from the web surfing session (if not disabled manually). The response verification menu allows the specification of how received content is to be tested:

Verify HTTP Status Code

If enabled, the HTTP status code of the URL call is verified. Additionally, multiple alternative HTTP status codes can be selected, affecting these status codes as valid/successful URL status codes. Apica recommends keeping the default settings.

  • Disabling this option - or configuring multiple alternative HTTP status codes - may result in "false positive" measurements.

Verify Content-Type

If enabled, verify the received content type (MIME type) of the URL call. Instead of configuring a simple search text fragment, a text operation or a variable can also be configured. Apica recommends keeping the default settings.

  • Disabling this option may result in "false positive" measurements.

Verify Header Text

If enabled, the configured text fragment is searched inside the received HTTP response header; an error is returned if this text fragment is not found.

  • Instead of configuring a simple search text fragment, a text operation or a variable can also be configured. Please see the Text Operations section below for strings and examples.

Verify Content

If enabled, verify the received content of the URL call.

Verify By Size

This means that only the size of the content is verified. This is fast and a good approach for completely static content, like images. You may also set an acceptable deviation of 0% if the content never changes.

  • Instead of configuring a fixed size, a variable can also be used to verify the size.

Verify Content

Verify By Text

Search for a text fragment inside the received content.

  • This is the best method for verifying dynamically-generated content.

  • If the recorded content contains ASCII text, then the recoded content is analyzed, and you see rated suggestions (0..1) for suggested text fragments.

  • Alternatively, the content can also be verified by a text operation or a variable.

Max. Stored Content Size

By configuring a value other than unlimited, all response data are read as usual during the load test execution, but only a part is stored internally.

  • For example, this means that error snapshots, which are made in case of failures, may not contain all received response data.

  • On the other hand, configuring the maximum stored content size with a value of less or equal than 5 megabytes can save a lot of Java memory during the load test execution and allows you to receive response content data of a large size (multiple gigabytes), even when many hundreds of Web users are simulated by only one load generator.

  • If a value other than unlimited is configured, the following restrictions apply during the execution of the load test:

    • If the content data are received in a compressed format, they are not automatically decompressed if the response content is larger than the configured value.

    • The stored content data's real size may be a little bit smaller/larger than the configured maximum value (+/- 32 kilobytes).

Failure Action

The Failure Action determines what happens in case the URL call fails.

  • None - Continue Loop means that the simulated user continues to execute the current loop (repetition of web surfing session). Such failures are also named non-fatal errors. This option should only be used if no variables have to be extracted from the URL call's response - or in other words - only if succeeding URL calls do not depend on the response of this URL.

  • Abort Loop >> Next Loop means that the current loop (repetition of web surfing session) of the simulated user is aborted and that the simulated user executes after that the next loop (if more loops per user are planned, or if the duration of the load test is not exceeded). Such failures are also named fatal errors. Optionally, you can activate the checkbox Terminate User, which affects how all simulated users for which this failure occurs are removed from further load test execution.

Reset Settings

Reset the URL setting only or All URLs with a specific HTTP Status Code AND Content-Type.

By clicking on the Apply Default Settings button at the bottom of the window, you can undo your changes, and the default settings are reapplied.


Text Operations

Instead of searching the occurrence of a simple text fragment, the following Text Operations are also supported:

String

Description

String

Description

 ![<search text>]

 The search text must not occur inside the received content.

 #<int>[<search text>]

 The search text must occur exactly <int> times inside the received content.

 #<int>-[<search text>]

 The search text must occur a minimum of <int> times inside the received content.

 #<int1>-<int2>[<search text>]

 The search text must occur a minimum of <int1> times, but not more than <int2> times, inside the received content.

Search Text Examples

Text

Description

Text

Description

hello

The search text "hello" must occur at least once inside the received content.

![ORA-01652]

The search text "ORA-01652" must not occur inside the received content.

#1[Dear Mr.]

The search text "Dear Mr." must occur exactly one time inside the received content.

#1-2[Order Number]

The search text "Order Number" must occur a minimum of one time and a maximum of two times inside the received content.

#3-[new order]

The search text "new order" must occur a minimum of three times inside the received content.

Note: One or more variable text patterns in the form of ${<variable name>} are supported as a part of the search text; for example, Welcome ${gender} {$name}.

Can't find what you're looking for? Send an E-mail to support@apica.io