ITF
INTEGRATED TEST FRAMEWORK

Test generation, building & running on target with detailed diagnostics for unit and integration testing.

ITF
INTEGRATED TEST FRAMEWORK

Test generation, building & running on target with detailed diagnostics for unit and integration testing.

Cantata is a dynamic testing tool for unit and integration testing C and C++. The test framework provides a high degree of automation for the generation, execution and analysis of tests that fits into your development workflow. Its integrated test framework automation enable developers to comply with the testing requirements of software safety standards and to implement industry best practices.

The Cantata test framework empowers you to select the most effective and efficient ways to develop and manage tests to suit your workflow

The Cantata test framework can also be easily extended or integrated directly with continuous integration and static analysis tools for further synchronization and automation of testing activities.

Cantata provides an integrated test framework (test scripts and a supporting library) which enables any combination of testing styles for both unit and scalable integration testing. Tests can be edited in a GUI or directly as C/C++ code, and run as executables on both host and multiple target platforms.

User code is driven by portable test scripts with target compatible libraries, and built as a single test executable to run on multiple host or target platforms. Instrumentation is used for white-box access and code coverage, so production code is never modified for testing.

"Cantata can be used for different targets without problem. We have done this for both our A380 EMU and EIPM systems."

Philippe Lomazzi, Head of Software Development, Meggitt Avionics

"Soon, all developers were testing to exactly the same standard. Maintaining each other's test scripts became much easier as a result of this standardization."

Philippe Lomazzi, Head of Software Development, Meggitt Avionics

"We found errors using Cantata that we know we would otherwise have missed until later."

Dave Parker, Software Engineer, Marconi Communications

Setting up testing projects to fit your workflow

Cantata testing Eclipse projects can be set up in 5 flexible ways to best suit your normal workflow:

  • Use an existing Eclipse project
  • Create a workspace above the source code
  • Create a project in or above the source code
  • Link the source directory into a project
  • Copy the source code into a project

Testing projects can be set up for bottom-up, top down or isolation unit testing approaches, and for either Cantata or externally driven integration testing.

Generating tests

Cantata test scripts can be generated from one or more selected source code files. The source code is parsed to obtain a comprehensive model (.csi file) of the code, so structured tests can be generated from it. C/C++ test scripts are generated within test projects with corresponding Cantata Makefiles for automated regression testing.

Cantata test scripts can be alternatively be generated from one or more selected header files as soon as function prototypes are created within them. This allows the test framework to be constructed before the body of the software under test is fully implemented.

Test scripts can contain several styles of test cases:

  • Single test case per function in the code
  • Multiple table driven test cases, using multiple user defined input values
  • Robustness tests, using multiple pre-defined input values
  • AutoTest (see AutoTest capability for details)

Editing tests

Test scripts and Makefiles can be edited using an intuitive graphical Test Case Editor or any C/C++ code editor. Changes are bidirectionally synchronised between the two methods.

Initial default values are set for inputs, expected outputs, and expected call order for fast iterative test case editing. Function calls are controlled (simulated or intercepted) in the test script.

See the Unique call control capability for details.

Cantata provides full edit ability on both automatically generated and GUI or code editor created test cases. The ability to duplicate and modify existing test cases, using the interface, drastically speeds up test script generation.

Controlling build and run on target

All Cantata related files are stored in the “Cantata” directory at the top-level of a project. This allows the test files to co-exist with your source files in a self-contained hierarchy and to be easily managed using version control tools.

Tests are built with the code under test, using your standard build mechanism on host as binary compatible executables, and downloaded to run on a target platform via the IDE or command line.

Cantata can be deployed to an extensive range of targets (see Cantata Works in your environment). The flexible deployment and use of different build and I/O options, means Cantata works seamlessly with your target development environment, and allows you to test the code with software, hardware or processor in the loop.

Platform independent tests can be re-used over multiple targets, through controlling multiple Cantata target deployment configurations.

Automating Continuous Integration and regression testing

Once tests are created and pass, continuing to re-run them so that regression errors are not introduced to previously working code is why people do regression testing. Automated regression testing with Cantata is supported through use of continuous integration tools such as Jenkins®.

As Cantata tests are developed in the GUI, a suite of Makefiles are generated to compile, link, deploy, execute and retrieve results from the target platform. Cantata Makefiles can be used with your existing code tree and your own makefiles (avoiding separate copies of code in a sandbox just for testing).

Cantata Makefiles can also perform any combination or pre and post build or test actions and giving you complete control over environment initialisation and reporting. They enable integration with other tools such as static analysers (to keep coding standards and testing synchronised) and Jenkins® (for continuous integration) throughout development and regression testing.

Diagnosing and reporting

Cantata provides powerful drill-down Eclipse views for pinpoint diagnostics of functional test and integrated code coverage results. Where even finer grained diagnostics are required, test execution can be stepped through under the control of a de-bugger.

Configurable reports are generated in RTF, XML and HTML along with certification ready ASCII and HTML text.

Diagnostics and reports can also be enhanced and aggregated over multiple user through the optional Cantata Team Reporting add-on.

Producing certification evidence

Cantata comes with comprehensive tool safety standard specific guidance for the building, execution and management of tests for testing to software safety standards.

Cantata certification ready functional test and code coverage results are automatically produced in asci text format, directly from the target platform.

Together with the appropriate tool certification / qualification kit, and target deployment evidence, this provides all the necessary evidence of software testing required for device software certification submission.

The Cantata integrated test framework set out above may be used to comply with the requirements of software safety standards and industry best practice. For more detailed information on building and running tests and the requirements of a specific standard, please see our extensive set of resources.