In this article we will mention about text based formatted messages and their usage program in Turkish Armed Forces, which is called as MEFORS.
Information Exchange and Formatted Messages
Effective interoperability of units and subunits can be achieved if they share and able to use some vital information such as common operating picture in the battlefield. Within this context interoperability can be defined as the ability of systems to exchange and use information.
Information systems can be designed by using several common structures. For example all units accessing the information system may use same database, or distributed systems make use of same common database and employ a replication mechanism between databases. These types of solutions are based on homogeneous databases. But because of changes in requirements, maintaining exactly the same database or data structure is not always possible. The solution to this problem is using heterogeneous databases in information systems and employing Extract, Transforms and Load method (ETL) for exchanging data. This way of achieving interoperability has an important drawback; when more than two databases exist in the architecture, the number of interfaces and number of ETL procedures increases geometrically [O(n2)]. This problem can be solved by using an architecture that employs a common exchange language (Figure 1).
This architecture has several advantages. First of all, ETL procedures become simple mappings that convert one structure to the other. Secondly, the number of mappings to be implemented is in direct proportion to the number of parties. Finally and most importantly, a standardized language is used to define the information to be exchanged. Consequently, this standardization results in effective and easy implementation of mappings.
The exchange language must provide sound and complete semantic and syntactic interoperability. Semantic interoperability means that the common language must contain all of the concepts, meanings and views in such a way that do not cause any ambiguity. Syntactic interoperability means structure level interoperability that makes applications to understand the structure of the language. Another factor to be considered in design of a common language is the dynamics of the systems. This expresses the change rate of the data in the databases.
Text Based Formatted Messages as the Common Language
Formatted messages can be used as the common language satisfying the requirements explained in the previous section. Figure 2 shows the architecture where formatted messages are used as the underlying common language.
The text based formatted messages have been used for exchanging data for over twenty years in military community. Its origin is based on exchanging data between different organizations such as NATO countries or units and subunits which are placed in an order of battle. The usage of text based formatted messages is a very simple, yet powerful solution to the interoperability problem in that, a text-based structure provides platform independence and ease of access, while a well-defined formalism enables straightforward processing and information exchange. There are several formats that define the syntax of the messages. These syntaxes have emerged from different communities. For example US have defined the US Message Text Format (USMTF) for their national systems. NATO has defined the ADatP-3 (Allied Data Publication No 3) to be used by NATO countries. Although they have different syntaxes, the approaches are virtually the same. The information in the message is grouped into semantic packages. A message is composed of groups and/or sets. Groups are composed of sets, sets are composed of fields, and fields are atomic entities that do not posses any further structure. We can explain formatted messages by using the analogy with “mails” exchanged between humans. If we setup an analogy between message and mail, then groups in a message correspond to paragraphs in a mail. Similarly sets in groups correspond to sentences and fields in sets correspond to words in sentences. Figure 3 depicts a sample ADatP-3 message that illustrates message, group, set and field concepts.
It solves the syntactic interoperability problem, because it is based on a common agreed syntax. Furthermore the semantics is constructed by analyzing the information exchange requirements of the systems that will be joining the interoperability architecture. This solves the semantic interoperability problem.
Formatted message structures are defined by the standardization organizations. For example, NATO has defined over 300 messages using the information exchange requirements. Besides the message construction rules, ADatP-3 catalog also contains the definition of messages, sets, and fields.
Advantages and Applications of Formatted Messages
Using messages for interoperability has several advantages over replication and direct access to an information system. When messages are used for exchanging information, the overall system becomes a loosely coupled system. This means that asynchronous processing is possible. This is ideal for data integration or data collection when continuous online connection is not possible.
Extending the system is much easier as compared to tightly coupled systems. A newly joined part extends the system by just taking into consideration the structures adopted from exchanged messages.
The systems shown in Figure 2 can be information systems or subunits reporting some information. This means that formatted messages can be used to exchange data automatically between systems or between humans and systems. System-to-human or human-to-human exchange corresponds to data collection or reporting whereas system-to-system exchange corresponds to replication-like or ETL-like data exchange. There are data model based alternatives to message based system-to-system data exchange. When message based and data model based exchange systems are compared, they have both advantages and disadvantages on each other. But the details of this comparison are beyond the scope of this article.
One of the most important properties of text based formatted messages is that it is both human-readable and machine-readable. Therefore it can be processed both automatically and manually. Manual processing opportunity is really a big advantage in situations where automatic processing means are unavailable, which is quite common in military usage environments.
Components of a Formatted Message Based Exchange System
Let’s assume that there are two systems that interoperate by exchanging messages, and in these systems, the main components are “Standard Management”, “Message Sending System” and “Message Receiving System”
In this architecture, “Message Standard Management” is responsible for the development of the definitions of the messages (message meta-models) that are exchanged between interoperating bodies. This process involves analysis and meta- modeling of the systems that exchange data.
Message send-receive lifecycle starts with preparing message manually via available human interfaces or automatically mapping data structures to messages. Then message is sent via communication system. At the receiving site, after message is received via communication link, it is either converted to an available human interface or automatically mapped to data structures.
MEFORS, Message Formatting System
STM has participated in the development of all components of the formatted message based interoperability solution. We have been transforming information exchange requirements into national messages for ten years. We have defined over three hundred national messages by using NATO syntax rules and procedures. These messages have been used by several information systems in Turkish Armed Forces.
In these national projects we have developed message-to-information system and information system-to-message mappings. We developed very innovative solution architectures for information system interfacing. Briefly, these architectures bring easy maintenance opportunities which are very important for information systems that are subject to frequent changes.
We have implemented interfaces to SMTP, X400 and ACP 127 communication protocols. These interfaces are used to make information systems’ user interfaces independent from communication networks.
Systems, during their life cycle, become distributed and heterogeneous as they evolved. In this manner, exchanging information in an effective way becomes vital importance especially for subunits operating in a common battlefield. Formatted messages are very powerful in solving interoperability problems which may arise in heterogeneous systems. This is because of the fact that, in an architecture implemented with formatted messages, the number of system interface implementations becomes lesser, mappings are implemented very easily, it is possible to separate common language development from integration effort and finally human readability of formatted messages make it possible to reach information in an alternative way.