Slide 1

Flexible Testing Techniques

One tool for all your embedded unit
and integration testing needs

Cantata is a dynamic testing tool for C/C++ unit and integration testing. Its flexible set of test automation capabilities enable developers to comply with the testing requirements of software safety standards and to implement industry best practices.

Cantata test automation empowers you to select the most effective and efficient combination of dynamic testing techniques to suit your C and C++ code implementation styles.

Cantata tests can be extended directly in any code editor, extending the flexibility to anything possible in the C/C++ programming language.

Focused Flexibility

No single tool suite can support every style of testing, and Cantata is no exception. Cantata is focused on the unit and integration testing requirements of standard compliant dynamic testing C and C++ code on embedded systems.

With the Cantata flexible toolset you can choose which combination of automated testing capabilities are right to drive your unit tests, integration tests or to enhance your higher level tests.

"Module testing early during development is becoming more important due to the increasing complexity of software. In practice Cantata has proved successful and increased unit testing efficiency. Cantata from QA Systems offers an ideal solution for the creation and execution of unit and integration tests, including coverage analysis."

Matthias Schmidt, Test management, Verification and Validation,
Bosch Engineering GmbH

"Through years of experience in unit testing using Cantata for various platforms and languages… we are confident in suggesting Cantata to our clients."

Padmakumar TV, Senior Engineering Specialist,
TATA Elxsi

Cantata capabilities by test technique

REQUIREMENTS DRIVEN TESTING

Cantata supports requirements driven testing to verify that the software does what it should do (derived from low level requirements, specifications or models).

The Cantata test harness drives the code under test via a test script calling a function with selected inputs (parameters and data) and automating the checking of actual against expected behaviour.

Requirements imported into the Cantata IDE make it easier to define test cases and trace requirements with test scripts or test cases.

(See also Cantata Requirements Traceability).

ROBUSTNESS TESTING

Cantata supports dynamic robustness testing to verify that code copes with abnormal conditions using techniques recommended by all software safety standards such as:

  • abnormal system initialization
  • invalid inputs & data
  • failure mode handling
  • boundary values
  • arithmetic edge conditions
  • timing constraints
  • memory allocation
  • error seeding

Robustness testing is made easy with Cantata Robustness Rule Sets of pre-defined values for basic data types, in looping test cases. All accessible global data is also automatically checked for inadvertent changes. Cantata test scripts and wrapping make it easy to create abnormal test pre-conditions and inject errors during test execution. (See also Cantata Call Control)

STRUCTURAL TESTING

Structural testing identifies source code untested by requirements driven and robustness tests, and addresses those gaps.

Cantata code coverage (used within or outside Cantata tests) pin-points any such unexecuted code elements in specific or all contexts. Gaps can be resolved by adding new requirements based test cases, documenting reasons why the code should not be executed in a particular context, or removing redundant code.

Cantata AutoTest automatically generates whole test scripts or additional test cases to achieve 100% structural coverage for C and C++ code. (See also Cantata Code Coverage & Cantata AutoTest).

Test Driven Development (TDD)

Cantata allows tests to be written as soon as function prototypes are created within header files. This allows tests to be constructed before the body of the software under test is fully implemented. Cantata for TDD enhances traditional black-box TDD techniques, by giving access to full featured white-box testing on encapsulated code internals such as private/static data and functions.

BLACK BOX TESTING

Black-box testing verifies code only via its public interfaces, and does not require knowledge of internal code implementation for checking correct behavior.

Cantata brings power to this technique with user-selected or pre-defined parameterised looping tests, so the same functions can be called with large data sets of input parameter or data values. The sequence of expected calls and expected return values can vary according to the input data sets, alongside global data outputs checked throughout the test run.

Cantata table driven test data sets can also be imported/exported via CSV. A GUI combinatorial effect calculator aids test vector selection. Cantata automatic coverage optimisation identifies the precise set of test case vectors needed to reach coverage targets.

WHITE BOX TESTING

White-box testing verifies code by granting access to the encapsulated code internals (private / static data and functions).

Cantata grants testers direct access to these without polluting production code with conditional compilation, by using fully automatic accessibility instrumentation only during testing. Test scripts can directly call private or static functions and set / check parameters and data declared private or static to a file or local function.

Cantata white-box tests can set up trigger conditions, and directly check values or internal behaviour more efficiently than with black-box tests.

Cantata call control can also be used on calls inside the compilation boundary for white-box testing.

PROCEDURAL AND STATE MACHINE TESTING

Cantata procedural tests verify the processing logic or algorithms in functions using appropriate sets of parameter / data inputs, and checking that actual call order, data read/writes and return values are as expected. Cantata parses the source code to generate test scripts with the necessary test hooks on function parameters and accessible data for users to set and check.

State machines are tested by setting trigger conditions and creating events, to verify correct state transitions. Cantata white-box testing makes this particularly efficient through direct access to local function static data storing the machine state.

Cantata user defined context code coverage can also be used to verify code has been tested in different states. (See also Cantata Code Coverage).

OBJECT ORIENTED TESTING

Cantata OO style tests are implemented as classes for testing methods, templates, or clusters of classes. They feature automated:

  • test case re-use via a parallel inheritance hierarchy
  • test class inheritance structure for inherited classes
  • concrete implementation of abstract base classes (ABCs) or pure virtual methods (PVMs)

To break the class dependency chain and make C++ isolation testing easy, Cantata automatically finds and resolves dependencies on undefined references that are not directly called by the software under test. (See also Cantata Call Control).

ISOLATION UNIT TESTING

Cantata does not dictate what code item is tested as a unit, or how much each unit is isolated (decoupled) from the rest of the system. Complete flexibility is given by automatic configurable generation of stubs and isolates to simulate, and wrappers to intercept calls made both externally and internally by the unit under test. The correct internal processing and correct use of interfaces (parameters and data), can be verified with Cantata as an isolated executable as soon as a unit compiles, without the need for the rest of the system.

Cantata generates an initial executable test case for each function / method in the selected source file(s) under test. Cantata unit tests call a function, initializing all input / output parameters, accessible data and expected call order. Test cases are independent of each other and the platform on which they run, avoiding a daisy-chain of dependencies.

INTEGRATION TESTING

Cantata driven integration tests are just large unit tests, but with a cluster of multiple units built as a single executable. Such a tests can verify the same internal behavior of unit, but more importantly the interactions between units such as:
inter-unit memory allocation
order of data read / writes
order of calls between units
Cantata integration tests provide the power to choose test script driver entry-points, with both internal and external simulations (stubs and isolates) or interceptions (wrappers) for flexible use of top-down, bottom-up or all-as-one integration testing styles. Cantata wrappers can also be used where Cantata is not the test diver, adding further control over integration tests

The techniques set out above may be applied in various combinations to comply with the requirements of software safety standards and industry best practice. For more detailed information on designing test cases and the requirements of a specific standard, please see our extensive set of resources.