What We Do
How we do it
Resources
SECURITY ADVISORIES
May 11, 2022
CVE-2022-26923 - Active Directory Domain Services Elevation of Privilege Vulnerability
THE THREAT Microsoft has disclosed a new vulnerability impacting Active Directory Certificate Services (ADCS) tracked as CVE-2022-26923 (Active Directory Domain Services Elevation of Privilege Vulnerability). If exploited successfully, an authenticated attacker can escalate privileges in environments where ADCS is running on the domain. eSentire is aware of technical details and tooling [2] for…
Read More
View all Advisories →
Company
ABOUT ESENTIRE
About Us
eSentire is The Authority in Managed Detection and Response Services, protecting the critical data and applications of 1200+ organizations in 75+ countries from known and unknown cyber threats. Founded in 2001, the company’s mission is to hunt, investigate and stop cyber threats before they become business disrupting events.
Read about how we got here
Leadership Work at eSentire
LATEST PRESS RELEASE
May 17, 2022
Cybersecurity Leader eSentire Continues Its Commitment to Rigorous Security Standards Earning PCI DSS Certification
Waterloo, ON, May 17, 2022 — eSentire, the Authority in Managed Detection and Response (MDR), maintains one of the most secure and robust IT environments of any MDR provider in the industry. To that end, eSentire today announced that it has received the Payment Card Industry Data Security Standard (PCI DSS) certification, considered one of the most stringent and comprehensive payment card…
Read More
Partners
PARTNER PROGRAM
e3 Ecosystem
We provide sophisticated cybersecurity solutions for Managed Security Service Providers (MSSPs), Managed Service Providers (MSPs), and Value-Added Resellers (VARs). Find out why you should partner with eSentire, the Authority in Managed Detection and Response, today.
Learn more
ECOSYSTEM PARTNER RESOURCES
Apply to become an e3 ecosystem partner with eSentire, the Authority in Managed Detection and Response.
Login to the Partner Portal for resources and content for current partners.
Search
Resources
Blog — Feb 25, 2022

How to Measure the Success of Your Phishing and Security Awareness Training (PSAT) Program

Speak With A Security Expert Now

Ransomware and crippling cyberattacks are so pervasive they have permeated our social fabric down to the late night comedy. HBO’s Last Week Tonight with John Oliver covered the subject of ransomware and even how cybercriminals race their Lamborghinis through the streets of Moscow, lavishing their protection within the borders of their native country, Russia in this case.

The majority of these crippling cyberattacks begin with a simple phishing email or business email compromise (BEC) scam. While most companies provide some form of annual cybersecurity awareness training to reduce the likelihood of employees being hooked, they focus on overly simplistic lures taken from public events that fail to represent the real danger of targeted cyberattack campaigns.

What’s worse is they rely on disengaged metrics like employee attendance and simple test results as a proxy to measure business risk reduction. However, there is little to no connection between the two. In fact, this shortsighted approach creates a false sense of security which only exposes the business further to cyber risk.

Stepping back for a moment, the purpose of phishing and security awareness training (PSAT) is to drive behavioral changes amongst your employees and the corporate ecosystem to improve business resiliency through reduced exposure to related cyber risks, alleviated resource constraints, and increased ability to meet regulatory compliance requirements.

So, why would you look at attendance lists and the so-called “smiley sheets” to assess business resiliency? This is why most programs fail–they don’t represent the cyber risks, they don’t provide the tools and skills necessary for employees to identify cyberattacks, and they focus on the wrong measures.

Plus, think about the wasted effort and costs. Whether you see it or not, there is a cost associated with your employees sitting in the PSAT classroom sessions. That salary easily eclipses the direct expenses associated with PSAT training sessions, such as the cost of lunch or even the speaker.

So how do you know your PSAT program is working?

Successful PSAT programs move beyond the overly represented focus on compliance (or as I like to call it, the Ten Commandments of IT). They provide contextual phishing examples, empower employees to protect themselves and the business, and provide clear threat reporting mechanisms.

When it comes to measuring and reporting PSAT effectiveness there are two models you can explore: the SANS Security Awareness Roadmap and the Kirkpatrick model.

The SANS Security Awareness Roadmap

The SANS Security Awareness Roadmap provides a tiered approach to maturing your program similar to other maturity models, in that it helps your organization move from a state of disregard for cyber risk and towards a more integrated, team approach:

Level

Focus

Leadership Approach

Employee Behavior

1

No program

Leadership does not recognize the cyber risks nor value of PSAT

Employees are oblivious and do not discuss cybersecurity

2

Compliance

Leadership understands cybersecurity obligations and risks

Employees do not understand cybersecurity, and attend PSAT as unproductive obligation

3

Behavior

Leadership understands cybersecurity obligations and risks and funds the program

Employees understand the risk and can identify threats and proactively report potential cyber threats

4

Culture

Long-term commitment to PSAT with metrics-driven decision making

Employees are empowered to make recommendations and proactively seek training and report cyber threats.

5

Outcomes

Leadership aligns business objectives, cyber risk prioritization, and PSAT resources

Employees are empowered to make recommendations and proactively seek training and report cyber threats.

This is a good framework that can help you determine where your company is in terms of maturity, and identify the gaps you need to fill to move up the safety stack.

The Kirkpatrick Model

The Kirkpatrick model is a long-standing standard for validating the efficacy of investments in training and learning. Like the SANS Security Awareness Roadmap, the Kirkpatrick model consists of four levels of maturity and indicators, focusing on Reaction, Learning, Behavior, and Results. Below, I’ve provided the levels and their focus, what they evaluate, and some specific examples of how to measure PSAT success.

Level

Focus

Evaluation

Examples to Measure PSAT Success

1

Reaction

The degree to which participants find the training favorable, engaging, and relevant to their jobs.

  • Subjective feedback forms to assess learner engagement, instructor performance, and content usability or format.

  • Passive metrics collected from online learning systems like % or # of employees trained and pass/fail metrics.

2

Learning

The degree to which participants acquire the intended knowledge, skills, attitude, confidence, and commitment based on their participation in the training.

  • Open rates measure detection based on Inbox preview data.

  • Click rates measure detection once opened and read.

  • Surrender rates measure user credentials given over.

3

Behavior

The degree to which participants apply what they learned during training when they are back on the job.

  • Report rates measure the number of suspicious lures reported using this mechanism.

  • Engagement rates measure subsequent communications with the cybersecurity team.

  • Policy rate measures the number of policy violations.

  • Regulatory rates measure the number of regulatory or legal violations.

4

Results

The degree to which targeted outcomes occur as a result of the training and the support and accountability package.

  • Losses to fraudulent financial transfers.

  • Losses based on cyberattacks.

  • Costs of data breaches.

  • Operation savings based on optimizations or reduced workload.

All too often, program coordinators proudly display their “smiley sheet” results and tout the number of employees trained as a measure of their value. Yet these results only measure subjective response, which in no way indicates the trainees’ ability to identify a cyber threat and avoid falling for the cybercriminal’s intent.

When asked which metric I depend on most, I always say, “Engagement”. It’s the most qualitative measure and it tells you that you’ve got through to your employees.

Ask them why they reported the suspicious email to see what indicators triggered for them since this will tell you that your program works when it comes to empowering employees to protect the business:

The second metric I look at is the time employees proactively (unrequested) spend with the IT/Security team. This action shows passion, which in turn shows their devotion to cybersecurity and their desire to protect the business.

Make it personal and make it worth it

When it comes to engaging employees, metrics can measure your success but they normally don’t motivate attendance and participation. Nor do compliance requirements. We’ve all taken cybersecurity awareness courses with the class window in the background as we do our “real work”.

One of the best examples I’ve seen of a successful phishing and security awareness training program is a company that created tailored campaigns, including whimsical posters, swag, and a competition ladder of escalating phishing testing to crown a cyber champion. It’s an innovative approach that led to record attendance, motivated employees, and reduced operational costs. It also meant their company reduced the risk of phishing.

Remember, just because cybersecurity is a serious topic, doesn’t mean your company can’t have fun!

How our Phishing and Security Awareness Training program can help

Designing, developing, and executing a fun–yet, educational–PSAT program is hard. Checking a box for compliance is easy, but moving the needle takes more leverage and more force. The better the PSAT program, the more effort is required since the complexity increases.

eSentire’s Managed Phishing and Security Awareness Training puts your organization on the forefront of user protection against the latest social engineering cyberattacks.

Our end-to-end service alleviates the resources required to operationalize an effective user resiliency program. By leveraging software paired with dedicated social engineering experts, we ensure that your users are continuously tested and hardened against even the most sophisticated phishing attempts. Our robust phishing libraries consist of hundreds of templates, ensuring users are tested against real-world scenarios, not commoditized and easy to spot templates.

More importantly, our PSAT training modules are automatically assigned to users that fall victim to simulated testing, ensuring education is integrated at the moment of failure and drives sustained behavioral change. Our PSAT program generates measurable results, extending well beyond checking a compliance box. We not only help you meet regulatory requirements, but also ensure that your organization is resilient against the latest social engineering tactics.

To learn how eSentire’s Managed Phishing and Security Awareness Training can help drive behavioral change with your employees across your organization, book a meeting with a cybersecurity specialist today.

Skip To:

  • So how do you know your PSAT program is working?
  • Make it personal and make it worth it
  • How our Phishing and Security Awareness Training program can help
View Most Recent Blogs
eSentire
eSentire

eSentire is the Authority in Managed Detection and Response, protecting the critical data and applications of 1200+ organizations in 75+ countries from known and unknown cyber threats. Founded in 2001, the company’s mission is to hunt, investigate and stop cyber threats before they become business disrupting events. Combining cutting-edge machine learning XDR technology, 24/7 Threat Hunting, and proven security operations leadership, eSentire mitigates business risk, and enables security at scale. The Team eSentire difference means enterprises are protected by the best in the business with a named Cyber Risk Advisor, 24/7 access to SOC Cyber Analysts & Elite Threat Hunters, and industry-leading threat intelligence research from eSentire’s Threat Response Unit (TRU). eSentire provides Managed Risk, Managed Detection and Response and Incident Response services. For more information, visit www.esentire.com and follow @eSentire.