Understanding Loadtest Results

The Results view is accessible for completed jobs and provides access to all information collected during a test run. It also allows you to compare different instances with each other:

Result Details

The Result Details tab contains a number of charts which provide metrics related to the performance of the Load Test. It displays the metrics for a specific Test Result within the Test Instance you are viewing. It contains a header which displays the Concurrent Users which were specified when the test was configured. It also displays the start and end times of the Test. Select the Test Result to view by selecting a number of users from the “Users” dropdown; the metrics displayed on the charts are specific to the test run which was selected. The charts can be manipulated to show or hide additional data, printed, or expanded to provide a better view of the chart; see for more details.

 

The following screenshot shows metrics for a Test Result which ran a .class file with 2 users (the .class file name has been obfuscated). Response times can be compared to the number of Active Users - virtual users who have started but not yet finished the Test.

Pages & URLs

The Pages & URLs tab shows detailed data about URL and page response times, and provides access to response time breakdowns.

This page contains detailed results for each individual URL call in the test. 

Instance

The Instance section shows basic information about the test.

Item

Description

Item

Description

 Concurrent Users

Number of Users

 Test Sequence Times

Start and end for the test sequence run. 

Create Quick PDF Report

Button to Quick Report form the results.

Results

In the Results section you can select which run to display in the charts and tables. Each run in the instance is indicated by a tab showing the number of users.

By default, information about all runs is displayed:

To select a specific run, click the tab with the desired number of users:

The table corresponding to the run is shown in the Pages & Urls section.

Errors

Errors show detailed information about any possible errors if any have occurred.

Instance

The Instance section shows basic information about the test.

Item

Description

Item

Description

 Concurrent Users

Number of users

 Test Sequence Times

Start and end for the test sequence run. 

Create Quick PDF Report

Button to create a PDF report form the results.

Error Overview

The Error Overview table shows the number of errors per page and test.

Each page is a row in the table, with a column for each test.

This view allows you to see which tests contain errors and select them for display in the details chart and table.

Chart

The Errors Over Time chart shows error occurrences during the test for the selected location.

The chart shows the number of errors during the test for the selected test.

Select Test

To display the chart, you need to select the test in the dropdown menu.

Live Error Feed

The Live Errors Feed table shows information about the last recorded errors chart.

Column

Description

Column

Description

Page

Page order number and name in scenario.

URL

HTTP Methods and URL associated with the error.

URL Index

Error number relative to the URL.

Time Offset

Time when the error occured, relative to the start of test.

Error Type

HTTP Status Codes and message for the error.

Actions

Buttons for possible actions to perform on the error.

Actions

Column

Action

Description

Column

Action

Description

 

Details

Open the Error Details dialog.

Error Details

The Error Details dialog provides access to a number of tabs containing detailed information relating to the error.

Summary

The Summary section shows the received content of the response.

 

Row

Description

Row

Description

Page

Page order number and name in scenario.

Error Data

Time when error occured: Timestamp, time after start (number of seconds, and converted time).

URL

HTTP Method and URL associated with the error.

URL Exec Step

Execution step when error occured.

Exec Agent Socket Create Time

Creation time for the execution agent connection.

TCP Connection Reuse Count

Number of reuses of the TCP connection.

TCP Local Address

TCP local IP address.

TCP Local Port

Local port for TCP connection.

Remote TCP Address

TCP remote address.

TCP Remote Port

Remote port for TCP connection.

SSL Session Id

Session ID for SSL connection.

Error Log

Scrollable log view. Clicking the header opens the full log in a separate browser window.

Log

The log in the Summary section can be viewed in a separate browser tab.

Request

In the Request Headers tab, information about the request preceding the error is displayed.

The table displays the HTTP Methods and URL associated with the error at the top. Following that is a table containing all headers sent with the request.

Request Content

The Request Content tab shows the sent content of the request if any exists.

Response Headers

The Response Headers tab shows the received response headers.

The table displays the HTTP Status Codes and text error at the top.

Following that is a table containing all headers provided in the response.

Response Content

The Response Content tab, shows the received content of the response, if any.

To view the content in a browser, click “Preview Response Content”. Your default browser opens to display the content:

Integrations

Depending on which integrations you have enabled, various integration tabs may be available as part of the results view.

Transactions

The Transactions tab presents information about User Defined Transactions.

Summary

The Transactions Summary table shows an overview of the transactions for a selected result.

With the All Transaction Times Percentile chart, you get a statistical view of how the transaction times are distributed.

The All Transaction Times chart shows the transaction times for the selected transaction.

In the Full Passed Transaction Times Percentile chart, you get a statistical view of how the transaction times are distributed for the passed transactions.

Settings Used

The Settings Used tab shows a summary of all settings used for a load test.

Instance

The Instance section shows basic information about the test.

Concurrent Users

Number of Users.

Test Sequence Times

Start and end for the test sequence run.

Create Quick PDF Report

Button to the Quick Report form results.

Results

In the Results section, you can select which run to display.

To select a specific result set:

  • Click the tab with the desired number of users

The Settings Used tab shows a summary of all settings used for a load test.

Item

Description

Item

Description

Scenario

Scenario

Scenario used in the job.

Loadtest Options

Number of Concurrent Users

Number of virtual users in the test.

Loadtest Duration

Total duration of the test.

Rampup Time

Time to ramp-up.

Run Multiple Sequential Tests

Used execution mode.

Location

Cluster

Location cluster(s) used to generate load.

Scenario Options

User input Text Files

Connected input file(s).

User Defined Vars

Variables defined in the scenario.

Advanced Options

Max Loops Per User

Maximum number of loops per users.

Request Timeout (sec)

Time to wait for responses.

Additional Options

Additional command line options provided as part of the job.

Distribute Load on All Available Datacenters

Enabled load distribution

Client Options

User Agent

Client User-Agent.

Network Bandwidth Downlink (Mbit/s):

Desktop

Limit on downloaded desktop traffic.

Mobile

Limit on downloaded mobile traffic.

Network Bandwidth Uplink (Mbit/s):

Desktop

Limit on uploaded desktop traffic.

Mobile

Limit on uploaded mobile traffic.

Client Side Monitoring

Enabled Client Side Monitoring.

DNS

DNS Hosts file

Custom hosts file.

DNS Server

Custom DNS server used for lookups.

Resolve DNS for Each Executed Loop

Whether DNS lookup is performed for each loop.

DNS Translation file

Custom DNS server translation file used.

Reporting

Email Preliminary Report To

Address for test reports.

Reporting History

Limit on number of tests to include in test history.

Execution

Schedule

Type of scheduling for the job.

Schedule Date

Date for schedule.

Test Information

Attached to Project

Project the test belongs to.

Job Comment

Comments added to the job settings.

Tags

Scenario Tags applied to the scenario.

 

Client Side Monitoring

If the job uses Client Side Monitoring, this tab shows real browser rendering times corresponding to the behavior of a real user accessing the application during the load test.

You will be able to see the total Response Time measured from a real browser for each separate run on this page.

Instance Results

The Instance section shows basic information about the test:

 

Item

Description

Item

Description

 Concurrent Users

Number of Users

 Test Sequence Times

Start and end for the test sequence run. 

Create Quick PDF Report

Button to the Quick Report form results.

Results

In the Results section, you can select which run to display in the charts and tables.

To select a specific result set:

  • Click the tab with the desired number of users

Location Overview Chart

For each location, a Location Overview chart displays a graph containing the page response times and the number of URL errors for all runs.

Select the desired location, e.g. Los Angeles, from the dropdown menu to pick which location to show in the chart.

The chart shows the page response times and the number of URL errors overall runs for the selected location.

Run Details

The Run Details shows information about the run and provides access to related screenshots.

The run details contain a timestamp for when the test started (with offset showing time elapsed from the start of test in parentheses).

If screenshots are available, a link to open them for viewing is also shown.

Snapshot

To see the screenshots:

  • Click the View Screenshot link.

The screenshots are displayed as an overlay on your browser:

The snapshots provide a browser view of the page at the time the URL occurred.

Navigate

When you hover the mouse over the left or right side of the snapshot, navigation buttons are shown, allowing you to browse through multiple snapshots, if available.

Select Run

The Select Run dropdown menu allows you to pick which test run to show in the run Run Details and Pages sections

URL Errors

The URL Errors table shows any errors that have occurred for the page during the test.

Each URL is displayed as a row in the table.

Column

Description

Column

Description

#

ID number.

Code

Error code, if any.

URL

URL associated with the error.

Time

Time elapsed from the start of the test.

Error

Error message.

Pages

The Pages table(s) shows information about the page(s) in the test and the URLs called for each page during the test run.

Each page is displayed in a table, with URLs displayed in rows:

Column

Description

Comment

Column

Description

Comment

URL

Called URL.

 

Time

Time until received response.

 

Size

Response size.

 

More Information

Show more information about the URL.

Click to see more information about the response.

 0s, 1s, 2s….

Graphical representation of the response times.

 Click to see timing details.

The “More Information” Column

More information about the response is available for each URL. To see more information about a particular URL, click the “See More” link. A popup containing details about the URL is shown:

Item

Description

Item

Description

Mime Type

MIME Type for the response message.

HTTP Response

Response HTTP Status Codes.

Error Message

Returned error message (if the URL is associated with an error).

View Details

To see the timing details about a particular URL:

  • Click the representation for the desired URL

A popup containing details about the URL is shown:

Information about the HTTP call is shown at the top.

Item

Description

Item

Description

HTTP Response

Response HTTP Status Codes.

Error Message

Returned error message (if the URL is associated with an error).

The Response Time Details section shows a graphical representation of the flow and a legend with times.

Item

Description

Item

Description

 Blocked

 Connect Duration

 Send Duration

 Wait Duration

 Receive Duration

The Total Response Time is displayed as a summary of the detailed view.

 

 

Cannot find what you're looking for? Send an E-mail to support@apica.io.