Introduction
Proposing a new standard is a tough and uncertain task. A standard has to be complex enough to please everyone and simple enough to frighten no one. IDMEFv2 mixes multiple concepts cyber and physical. But IDMEFv2 can potentially be used by a lot of different type of users, security tools developers , integrators, operators, โฆ so it’s ease of adoption is very important.
To facilitate people contribution and consensus we have defined some principles we are trying to follow to define IDMEFv2.
PS: Those principles can still be discussed as we are working on the very first version of the format.
Perimeter : Cyber and Physical incidents.
IDMEFv1 was covering cyber intrusion, IDMEFv2 covers cyber and physical incidents.
This is probably the most disruptive principle of IDMEFv2. IDMEFv2 first drafts have been designed in a “real life” architecture for protection of critical infrastructures. During those pilot it appeared as obvious that there was a need for standardisation of message exchange between cyber detection analyser and managers, there was also a very similar need between physical detection analyser and managers and most important there was a need of message exchange between cyber and physical detection architecture for detection of complex and combined scenarios of attacks or simply of chained incidents. Therefore we had two choices , first choice was to define two dedicated standards, a cyber incident detection format and a physical incident detection format, able to interoperate, the second choice was to define one single standard for the two domains. Concepts being so close and the frontier between cyber and physical world being inexorably fading away we choose the second option, one standard for all.
Purpose : Incident detection.
The purpose of IDMEFv2 is incident detection. To keep it simple IDMEFv2 should be composed of classes and attributes for incident detection only.
Thus it’s important to keep in mind what IDMEFv2 is not :
- IDMEFv2 is not a universal log or event format. It describes only event of interest in regards of incident detection. It doesn’t and can’t replace all existing log format.
- IDMEFv2 is not an incident analysis format. It doesn’t replace IODEFv2 (Incident Object Definition Exchange Format). IDMEFv2 can help create a IODEFv2 message. It can also be joint as an attachment to a IODEFv2 message to offer technical details.
- IDMEFv2 is not a “trouble ticketing” or “case” format. When an incident is detected, the operator might need to “open a case” so the incident can be resolved. This needs to be done in a ticketing tool. An IDMEFv2 message can be attached to a ticket but should not contains information about case resolution (e.g. owner, status, priority, history, etc.) . It may contain a “Ticket ID” attribute to point toward an external ticket system.
- IDMEFv2 is not a Threat Intelligence format. IDMEFv2 might contains information for threat intelligence expert , it can transport Threat Intelligence personalised information but it doesn’t replace dedicated threat intelligence format like STIX.
IDMEFv2 should store information for incident detection. After an incident has been detected it might be necessary for the operator/analyst to fetch information that are available in raw logs for example but not stored in IDMEFv2. Although the limit is not always clear, IDMEFv2 attributes should not store information that are only useful for analysis and not for detection. First we don’t want to replace existing standards, secondly a format which would try to solve all problems (logging, detection, analysis, etc) is destined to fail.
Adoption : Definition of IDMEFv2 focuses on “adoption”.
It seems obvious but can be sometime forgotten in the middle of heavy technical discussion.
We ARE NOT aiming everyone 100 % satisfaction, we are aiming consensus, compromise and simplicity for a wide adoption.
Adoption is not necessary synonym of complexity or exhaustivity.
Minimize the learning curve
Incident (and intrusion) detection is a complex field. The recent rise of IoT and IIoT makes it even more complex adding new potential attack surfaces. One has to learn a lot of concepts to deploy efficient incident detection systems. IDMEFv2 needs to be easy to learn and to implement. It should manipulate essentially widespread concepts and at the same time leave optional ways of dealing with very complex needs.
Usability
The same way as “learning curve principle”, IDMEFv2 should manipulate widespread and easy to deploy technologies.
This principle partly led us to choose JSON representation as well as HTTPs for transport implementation. Nowadays JSON is widely adopted and specially in incident detection tools, log management console and NOSQL database. HTTP is obviously one of the most used and popular Internet protocol. Other format and transport are possible but we will focus on those two choices first.
Simplicity vs exhaustivity
Following the “Less is more” quote as well a the KISS method (Keep It Simple, Stupid) IDMEFv2 should be as simple as possible, while still answering all the needs.
IDMEFv1 was very exhaustive : 33 classes, 108 attributes, 260 possible combinations, potential infinite recursivity, etc. In theory this could be an advantage but in practice it had many disadvantage : long learning curve, complex code implementation, more misinterpretation, โฆ This REX led to a much smaller number of classes and attributes in IDMEFv2.
Agnostic format, classes and attributes
To respect most of the principles we have already presented IDMEFv2 classes and attributes should be as agnostic as possible. We are not creating a format composed of cyber classes/attributes and physical classes/attributes. We need to use concept and define classes and attributes that can be used indifferently in the two domains as much as possible.
Attribute Enumeration
As often as possible, IDMEFv2 must propose enumeration for attributes value.
The purpose of standardizing incident detection message exchange goes beyond the simple exchange of information between analyser and managers. Alerts must be compared, analyzed, aggregated, correlated, etc. Thus it is important that some attributes propose “closed list” values other wise comparison/filtering/ordering is impossible. This is a difficult choice as “closed enumeration lists” are always limited and frustrating but this is essential. Means will be proposed to extend the close list if essential.
Theory AND practice
Many theoretically very good standards have failed when confronted to reality. IDMEFv2 must avoid this pitfall. IDMEFv2 has been tested in real situation during the first stage of it’s definition. Today, libraries and tools are available to continue testing as the definition is being tuned.
Avoid the NIH syndrome
Fight the NIH (Not Invented Here) syndrome
Incident detection is not a brand new field. There is a lot of existing formats which cover part of the problem. It is important when possible to reuse existing concepts which have already proven to be efficient. During the analysis phase, many different log/event/incident/alert formats have been studied to extract interesting concepts. Among others, IDMEFv2 reuses concepts, classes or attributes from :
- IDMEFv1 : Intrusion Detection Message Exchange Format v1
- IODEF v2 : Incident Object Definition Exchange Format
- IDEA : Intrusion Detection Extensible Alert
- Log Management and SIEM tools IBM (Leef), HP (CEF) , Splunk, Elastic (ECS), Logpoint
- RIST – ENISA : Reference Security Incident Taxonomy
Ecosystem & interoperability
Incident detection is a part of a more global puzzle using other exchange formats. IDMEFv2 has to feet correctly in this larger architecture avoiding to overlap other format perimeter when it’s possible. IDMEFv2 should also interoperate with existing formats.
Legacy
IDMEFv2 should be usable into existing architecture using proprietary formats. The ultimate goal would be that all detection tools, may they be analyser or managers, cyber or physical natively use the IDMEFv2 format, but obviously this can’t happen suddenly so it has to be “easily” possible to start using IDMEFv2 in an existing system without having to change all it’s component at once.
End to end format : From the probe to the operator.
IDMEFv2 alerts are created by analyzers and sent to managers. But the life of an alert doesn’t end when it’s received by a manager. It can be correlated, aggregated, stored, indexed, etc. and then it will be displayed to operators in a GUI (Graphical User Interface). It might even be used to created other messages in other format. This end to end usage has to be taken in consideration while designing the format: from the analyzer to the operator.
Modularity
One should be able to use only some parts of IDMEFv2
IDMEFv2 aim to cover all type of incident detection : Cyber , Physical, Availability and also Cyber Threats, Natural Hazard Threats, etc. It is very important that IDMEFv2 can be used on smaller perimeters.
As examples :
- IDMEFv2 can be used in a full cyber intrusion detection architecture with no physical detectors nor sensors.
- IDMEFv2 can be used in a full physical incident detection architecture with no cyber detectors nor censors
Cope with the past and prepare the future
IDMEFv2 must be able to deal with all the existing concept in incident detection but also leave room for incident detection evolution in the future.
As already mentioned bellow, the field of incident detection, at least for the cyber part, isn’t new and concepts are already used. IDMEFv2 must be able to interoperate with all this past. On the other end, standardising a format is a long task that can’t be renewed frequently. The field of incident detection is evolving very fast : rise of IoT and IIoT, growth of cyber-criminality, new types of attacks, smart-“everything”, etc. Organisations must also be prepared to face natural hazard threats and incidents rise due to climate change. IDMEFv2 must anticipate all those needs.
