IVCT (Integration Verification and Certification Tool)
Establishing a HLA certification process in NATO
The integration of distributed simulations and tools into interoperable federations is a complex and time consuming task requiring extensive testing of individual components, interfaces and of the integrated system.
To support this task, the “North Atlantic Treaty Organization (NATO)” relies on standards and agreements and their abidance. The Allied Modelling and Simulation Publication “NATO M&S Standards Profile” (AMSP-01) provides a list of recommended M&S related standards and best practices.
Efficient interoperability using the HLA (High Level Architecture) standards requires compliance with a shared reference data exchange model (Federation Object Model), a federation agreement and the ability to certify conformance. The modular RPR FOM (Real-time Platform Reference FOM) and the more recent NETN FOM (NATO Education Training Network FOM) provide building blocks for creating federation agreements and FOMs.
Prior to the arrival of the HLA IEEE 1516-2010 standard, the United States provided to NATO and nations a tool for federate HLA certification at the API (Application Programming Interface) level and SOM (Simulation Object Model) conformance checking.
The NMSG-134 (NATO Modelling Simulation Group) is responsible for defining the new NATO distributed simulation certification process and related tools with the intent to significantly reduce cost, risk, and integration time of simulation components in education, training and on the battlefield, reliably supporting the warfighter, where there is no time for tinkering.
This includes the CONOPS (Concept of Operations), the design and development of the IVCT (Integration Verification & Certification Tool) and the test cases for distributed simulation interoperability.
A concept of capability badges was introduced to capture the interoperability requirements, test cases, sequencing and conformance statements.
Main goals of the NMSG- 134 are:
- Reduce Cost of Distributed Simulation Integration
- Reduce Risk in Distributed Simulation Integration
- Reduce Integration Time
- Increase Degree of Interoperability in Distributed Simulation.
Derived from all the above, the principal objectives of the Group are:
- Maintenance of the NATO Education and Training Network Federation Architecture and FOM (Federation Object Model) Design Document (FAFD) including update and release of new versions of the technical specification;
- Dissemination of NETN FAFD through presentations, papers and standardization activities within NMSG, SISO and at other venues;
- Development of test, verification and certification procedures
- Design and implementation of IVCT to support certification, integration and verification of federates in specific federation.
- Development of IVCT tests to support NETN FAFD test, verification and certification
The Integration Verification and Certification Tool (IVCT) principal capabilities are:
- HLA Federate service compliance testing
- NETN FAFD (AMSP-04) testing
- Framework – Core Test Engine as Open Source
- Maintained by a NATO-Certification Entity supported by NATO-Accredited Test Laboratories
- Provided for use in test, integration and certification
- Initial version developed in MSG-134 with contribution from several NATO and PfP nations
- Certificates provided with “Interoperability Capability Badges”
Key Roles and Responsibilities
- The Accreditation Authority (AA) is a NATO appointed organization responsible for maintaining the business model and procedures used by Accredited Test Laboratories (ATL) and Certification Entities (CE).
- The Certification Entity (CE) is an organization with the authority to issue certificates of compliance to systems that have successfully passed testing of interoperability requirements (IR). The CE is responsible for the management aspects of certification, and is the initial point of contact for customers that want to certify their system with the right to refuse the certification. The CE also maintains the official version of the Integration, Verification and Certification Tool (IVCT) and delivers it, along with the executable test cases, to accredited test laboratories.
- An Accredited Test Laboratory (ATL) is a test laboratory given the official authority by the CE to perform certification tests of interoperability requirements and whose test results are recognized by the CE as valid for issuing certificates of compliance.
Key Concepts and Components:
– A simulation Interoperability Requirement (IR) is related to how distributed systems interact and exchange information in order to collectively meet overall simulation objectives. IRs are specified to ensure that a system component can be easily combined- and interoperate with other system components. The ability of a system to interoperate can be described as the set of fulfilled IR requirements.
– An Interoperability Capability Badge (CB) is defined as a token of achievement in terms of passing testing related to interoperability requirements (IR) associated with the CB. Successful compliance testing, verification, and certification of a system’s compliance with sets of IRs can be expressed as a set of CBs representing this achievement.
– An Abstract Test Case (ATC) is a complete, implementation independent, specification of the actions required to verify a specific test purpose expressed as a set of interoperability requirements (IR) associated with the ATC.
– An Executable Test Case (ETC) is a script or compiled program that can execute as part of IVCT to implement the ATC procedure for verifying the associated IRs.
– The Integration, Verification and Certification Tool (IVCT) is a core technical framework provided by the Certification Entity (CE) and used to support test and verification of simulation interoperability requirements.
– A Conformance Statement (CS) is a written statement declaring a system’s compliance with identified interoperability requirements. A CS is provided by the owner of the System-under-Test (SuT) to identify which standard sets of IRs the SuT should be certified against. In the CS the sets of IRs are referenced as Capability Badges (CB).- A System under Test (SuT) is the simulation system which is the target of compliance testing.
The main deliverable is a software package to support integration, verification, and testing of simulation interoperability. A supporting database for maintaining information about interoperability requirements and capability badges used in certification is also delivered.
|NATO Simulation Interoperability Test and Certification Service – Concept of Operations (CONOPS)||NATO MSG-134 – CONOPS – v1D6|
|IVCT Software and Documentation||Open Source – GitHub|
|NETN FAFD published as AMSP-04 covered by STANREC 4800||Document|
|Papers, presentations, publications and marketing material||Papers and Presentations|