The relevance of training as a soft mitigation measure: why the existing Capability Maturity Models are not adequately addressing the issue

2018-12-13T11:26:01+00:0013/12/2018|Tags: , , |


Training, or better cyber training, of people in the organisations is a soft mitigation countermeasure useful to reduce the cyber risk. Cyber training has some unique characteristics versus training that need a broader discussion. It is one of the most effective solutions today because it focuses on the root cause of most of the security-related incidents: the humans. Almost 95% of cyberattacks start with the involvement of humans in the early stages (the DOGANA project, among others, demonstrated the impact of the human-related attacks within the current cybercrime strategies). At the same time, it is also a well-known problem that training people with an efficient process and with lasting results, which means have a long retention time and long-lasting behavioural changes, is a challenge. How the Hermeneut cyber risk assessment methodology integrates with the desired maturity level of the cyber training strategies and the expected mitigation performances? To answer this question, we need to analyse the current state of the art in the existing Capability Maturity Models (CMMs) and especially those that include training and especially cyber training.

Within Hermeneut, the importance of linking the cyber risk estimations to a proper CMMs methodology emerged clearly. However, existing CMMs have some issues when it comes to training, as a few of them adequately address the issue; and unfortunately even less (actually almost zero) address cybersecurity training.

What are CMMs and their relation to the Hermeneut results

As cybercrime creates continuously and rapidly evolving risks, the tools and techniques to defend and recover must evolve and become mature just as rapidly. Today’s cybersecurity training market still misses a scalable, measurable, but most of all, open and accepted maturity model — a methodology that helps organisations and their supply chains to reach the correct maturity level, which addresses the specific risk maps of organisations at a reasonable cost. Existing CMMs propose a “one-size-fits-all” training approach that does not fit all the possible specificities of the organisations.

A good, yet brief explanation of Maturity Models is:

“Maturity models establish a systematic basis of measurement for describing the “as is” state of a process. A process’s maturity can then be compared to management’s expectations or contrasted with the maturity of other similar processes for benchmarking purposes. Insights also can be derived from the model for determining improvement options that help a process to satisfy its intended objectives over time”.

CMMs and maturity are a hot topic in today’s industry and, recently, also in cybersecurity processes (see e.g. here and here) and eLearning (see e.g. here and here). Even if no unified, formalised, widely-accepted standard exists for training on cybersecurity, there are several separate propositions. However, there are still no fully defined CMM propositions to cover the training in cybersecurity. CMMs are composed of different elements: (a) levels, (b) components, (c) expectations and (d) supporting tools. A CMM describes process components that lead to better outputs and better outcomes when applied throughout an organisation. A low level of maturity implies a lower probability of success in consistently meeting a specified objective while a higher level of maturity implies a higher probability of success. The correct maturity level is that level that addresses the specific risk map of the organisation at a cost aligned to the organisation’s expectations and investments priorities.

The Hermeneut output directly feeds the CMM model for training because it defines the exposure to the cyber risk of the company. As a project and cyber risk evaluation mechanisms, Hermeneut gives to the CMM the right metrics and the correct indication of costs and KPI to accurately define the desired maturity level.

Limitations of the existing CMMs

Several CMMs exist that include training, but their focus is not on cybersecurity training. For example:

  • Too general approaches to training: Bloom’s Taxonomy (which, while not a normative system, is highly regarded by many trainers as a mature way to think about educational programs), Phillips ROI Process Model, Six Sigma plus DMAIC (DMAIC stands for Define, Measure, Analyse, Improve and Control, and is the core tool used to drive Six Sigma projects);
  • Addressing training in the Industry context, but too generic or for other types of skills (e.g. management): Carnegie Mellon CMM, CMMI, Organisational Project Management Maturity Model (OPM3);
  • Vendor-related and not specific for cybersecurity training: Gartner’s eLearning Maturity Model, Zeroed-In Technologies, Human Capital Contribution Model (HCCM);
  • Cybersecurity-related CMMs not covering training or covering it as a too generic process: SANS CMM for Endpoint Security and Cyber Security CMM.

Generically speaking, therefore, some of the common problems of the existing CMMs are:

  • “One size (mis)fits all”: current approaches concentrate on generic training and/or eLearning to drive down costs (e.g. OPM3 has sections dedicated to training, but not specific for cyber security: section 5200 provides Project Management Training, and section 5210 for continuous training in the use of tools, methodology and deployment of knowledge);
  • The frameworks concentrate on the training processes (often seen as an industrial process that must produce constant quality) rather than readiness outputs and agile processes;
  • No assessment methods other than the traditional Q&A (often multiple choice) and attendance assessment rather than readiness level;
  • Do not consider the use of the immersive business environment (e.g. nomadic working style) that can overcome the traditional metrics;
  • Applicable to single organisations and not to supply chains.

One of the most evolved CMMs in cybersecurity training is openSAMM (including the supporting community, the tools and the stakeholders of the Open Web Application Security Project). It avoids the common problems discussed above, being agile, flexible and supported by a complete set of support tools. Unfortunately, it does not deliver a full solution for cybersecurity as it focuses on issues related to secure development, and when dealing with cybersecurity training, it exhibits similar issues (for example, the OpenSAMM Education & Guidance section defines training processes as “increasing security knowledge amongst personnel in software development through training and guidance on security topics relevant to individual job functions”).

Do we need to create a CMM specific for cybersecurity training?

Education and Training across an organisation are an essential aspect of all organisational initiatives for change; the curriculum is usually driven by carrying out an analysis of the opportunities and risks associated with any change plan. Cyber risk is currently generally addressed outside the broader risk register, but consensus within the risk professional community and the more forward-thinking cyber community is that, in a relatively short period, cyber risks will fold into the general risk processes. Moreover, Cybersecurity training has some unique characteristics, compared to training in general:

  • Cyber Security Training is not only a method to increase employees’ skills or sustain the business goals, but also a method of defence of the organisation: on the longer term, an improved cybersecurity training improves the overall security of the organisation;
  • Metrics to effectively measure cybersecurity training success are different from other training CMMs. In particular, it is a form of training that must be applied to every individual in an organisation or across a supply chain; the form of training may be different for different groups but the objectives are common: defence of the organisation/the supply chain. Metrics must be applied to the individual and the organisation (and ideally the supply chain) as a whole and must link to a demonstrable measure of risk reduction.

A CMM for training in cybersecurity is essential because it integrates and extends existing approaches: some of the questions of cybersecurity training are identical to those of general training (e.g. reuse of the Bloom’s taxonomy, Legacy of Kirkpatrick or Philips ROI methodology, etc.). Cybersecurity training has specific, and to a degree organisationally relevant, metrics as part of a Capability Maturity Framework that can be organisational specific when driven from the risk assessment. A way to push the existing solutions to another level is hence a CMM framework for creating, managing and measuring an organisational cybersecurity training program. An ideal CMM should, therefore, use business and cyber risk-assessment processes, such as that done with Hermeneut, to specify what a given organisation needs as the desired maturity level, and then determine the journey, through education and exercises to reach that level.

The CMM and the supply-chain role, a still not integrated combo

The supply-chain has a role in this scenario too, and its actors should have a maturity level adequate to the cyber risk of the leading organisation. Therefore, the CMM must be applied to supply-chain actors also. However, what is missing from the currently available CMMs are the risk-driven controls, methods and metrics to measure the supply chain too. The intention is to create consistent expectations across the supply chain, for example, specifying a security baseline or standard in contracts or service level agreements. A source of inspiration on this matter is the Information Assurance Maturity Model (IAMM) developed by CESG and now included in the National Cyber Skills Centre in UK policies.

The EU context demands for more research

The EU is standardising its education frameworks (e.g. e-CF is now a formal standard, EN 16234), with the final aim to adequately and efficiently integrate also the ICT Security workers and employers in the European e-Skills market. Unfortunately, the offering of certified tracks in ICT Security remains under-developed. The European Qualification Framework (eQF) is following a similar strategy [1], fostering social dialogue to find common sectorial agreements and job matching approaches between workers and enterprises and, establishing governance mechanisms based on continuous improvement and quality labels. Security is still not wholly included in these European frameworks because of the profound and highly dynamic changes in society and cybercrime that impact the qualification profiles of ICT security professionals (as cybercrime is highly multi-disciplinary and responding to cyber issues implies cyber tracks that spans both technical and human sciences).

The open challenge is hence a more profound rethinking of how to integrate cyber risk estimation solutions (such as Hermeneut), dedicated maturity level and CMMs, and soft mitigation strategies.


[1] As the first sector-specific implementation of the European Qualifications Framework (EQF), the e-CF fits for application by ICT service, user and supply organisations, multinationals and SME’s, for ICT managers, HR departments and individuals, educational institutions including higher education and private certification providers, social partners, market analysts, policy makers and other organisations in public and private sectors.


This article has been written by project member Cefriel. Cefriel is a digital innovation centre creating and rethinking products, services and processes to enhance and develop digital technologies.