System Tests

The fourth of eight (8) Test Types is the first to use uncontrolled data.

System Testing

The System Test validates that the components of the Solution which were previously tested using controlled data, continue to work together when data values are replaced with uncontrolled, actual Production-like data. This is accomplished by merging:

  1. The RICE objects that have gone through Functional and Integration Testing; with
  2. Any migrated / converted data from legacy systems; along with
  3. Any Production integrations (Transactional data); in
  4. A Production-ready configuration (Operational data, including security).

Overview

For key Actions (i.e., Business Processes, Activities, Tasks & Steps), not necessarly all, or those which may not be covered by Integration Tests, Functional Test Cases are modified to create System Test Cases which use representative Production data values. Similarly, for each process or sub-process (i.e., integration of functions), Integration Test Scenarios are updated to create System Test Scenarios, again replacing the use of pre-defined, controlled data for Test Conditions now using uncontrolled data  representative of Production values. These scripts and scenarios are run to validate that the process is operating as expected when Production data is introduced into the equation. As with the Integration Test Scenarios, expected output values are defined for various stages of the test.

The System Test is typically done in a full copy of the Production environment. This ensures that the appropriate configuration values are being tested, along with legacy data that is converted into the QA environment (if data is to be converted for Production).

On This Page
    Add a header to begin generating the table of contents

    System Testing offers an excellent opportunity to begin introducing selected end-users to the new Solution by having them participate in this test cycle. System Test provides a structured approach for key users to test Solution functionality and to compare that functionality to their business requirements. Users follow Test Scripts, which should include future-state policies and procedures to process transactions in the new Solution.

    The following assumptions apply for System Test:

    • Testers have the proper exposure to the Solution QA environment and have the understanding required to perform the test. Sign-Off by key team members (SMEs and/or Business / Process Owners) is required.
    • System Test occurs in a "Production like" environment. From a technical support perspective, this environment should be the same as what will be in the production environment. All User IDs and Security Roles should be established and used by the Testers. No one should be testing using a SysAdmin type ID.
    • System Test will necessarily repeat many of the same tests executed in Functional and Integration Testing. However, here the emphasis is on using Production-specific data and on the end-user's ability, by following Operating Procedures, test scripts and/or other documentation, to use the new Solution and obtain the expected results.

    Objective

    The objective of System Test is to validate that each of the three (3) primary system components -

    all operate together to support converted, integrated and/or generated transactional data.  In other words, that everything works correctly to produce the desired outcomes.

    The specification against which the System Test is run includes all Acceptance Criteria, design criteria (Features & Stories) and Business Process documentation where applicable.

    Data is processed (as far as possible) in an uninterrupted flow beginning at each logical Solution input point and continuing up to each logical Solution endpoint.  All functional capabilities (on-line and batch) should be thoroughly validated against requirements and Acceptance Criteria.

    Scope

    All functions with a Solution are potentially within the scope of System Testing, as are some functions up- or down-stream in related Solutions.

    Following each Release the scope of System Testing will expand to encompass new Features or functionality delivered.  While the scope may expand, the time allowed to accomplish the testing does not.  So these tests must focus on the Solution advances affected by change.

    Timing & Responsibility

    System Testing occurs following completion of the Integration Test cycle for each Release of the Solution.

    System Testing is the responsibility of a QA group, if one is associated with the Solution, or potentially the DevSecOps group if a QA Group is not associated.

    A.K.A.

    System Testing is also known as:

    • Process Testing; or
    • End-to-End Testing

    Dependencies with Other Test Types

    • Prior:  System Tests should draw upon existing Functional and Integration Test components for the majority of their components.

    Automation

    The initial execution of System Tests is typically not automated.  Some tests developed for System Test may be automated for subsequent Release cycles, and for other types of testing.

    Solution Types

    to migrate

    Test Conditions

    to migrate

    Test Data

    to migrate

    Test Scripts

    System Test Scripts should be either a clone, or a simplified version, of existing Functional or Integration Test Scripts.  It would be unusual to extend an earlier script beyond that which was done for any earlier test cycle.  If a script must be extended, understand why and verify that the new steps were covered by some earlier test cycle.

    Test Cases

    System Test Cases should be either a clone, or a simplified version, of existing Integration Test Case.

    It would be unusual to extend a script beyond that which was done for Functional or Integration Testing.  Again, If a case must be extended, understand why and verify that the new script or conditions were covered by some earlier test cycle.

    Test Scenarios

    System Test Scenarios should be based upon Integration Test Scenarios.  It is common to group or extend multiple Integration scenarios into larger, more complex series of functions for System Test scenarios.

    The next type of testing to review is NFR Testing.

    Scroll to Top