Measure phishing simulations with a small set of connected signals: who received the exercise, how difficult it was for that audience, who clicked, who reported it, how quickly reports reached the security team, and what changed afterward. A click rate is useful evidence, but it cannot establish whether training is working without that context.
For IT directors and MSPs reviewing a managed awareness program, the practical goal is a report that leads to a decision: improve the lesson, make reporting easier, repair the response process, or investigate a weakness in the surrounding controls.
NIST's August 17, 2026 discussion of human-centered cybersecurity provides a timely reason to revisit this approach. It introduces a concept paper on future guidance and emphasizes people's needs, abilities, and working conditions. It is not a new training mandate or a finalized measurement standard.
Define the campaign before comparing its results
Start every review with a short campaign record. These are recommended operating practices for making results interpretable:
- Audience: record the participating roles, number of people, and relevant exclusions. Keep each MSP customer's results identifiable within its own authorized reporting scope.
- Exposure: record how many recipients were sent the simulation and how many messages were delivered, blocked, or returned. A person who never received the message had no opportunity to recognize it.
- Scenario: retain the exact approved message, requested action, and assessment of its difficulty for the audience.
- Observation window: document when measurement starts and ends, including whether elapsed-time measures include nonworking hours.
- Event definitions: state what counts as a click, report, or other simulated unsafe action. Ask how the platform distinguishes a person's action from automated activity.
- Counting method: show unique people and the denominator behind each percentage. Repeated clicks by one recipient should not silently become several people.
If the denominator changes from recipients sent a message to recipients with confirmed delivery, label that change and avoid presenting the two rates as a continuous trend. Retain the counts so reviewers can reconstruct the calculation. For small teams, show counts alongside percentages because one person's action can move the rate substantially.
Use the NIST Phish Scale to interpret difficulty
The NIST Phish Scale User Guide, published November 15, 2023, describes a method for rating how difficult a phishing email is for people to detect. It considers both the message's observable cues and how its premise fits the recipient's context.
That distinction matters when comparing departments or successive campaigns. A message closely aligned with someone's work can present a different recognition challenge from a poorly matched message. A higher click rate therefore needs investigation before it is described as deteriorating employee performance.
Have the campaign owner apply the guide's worksheets to the actual message and intended audience. Keep that assessment with the results. Compare exercises with similar audiences, difficulty, delivery conditions, and observation windows wherever possible. If those conditions differ, explain the difference in the review.
The Phish Scale adds context; it does not turn a campaign result into a probability of a future breach. Neither a lower click rate nor a higher report rate proves that training caused the change.
Build a scorecard that connects behavior to response
The following scorecard is a practical recommendation, not a NIST-prescribed benchmark. Use measures your tools can substantiate and identify unavailable data explicitly. The basic rates below use recipients sent the simulation as their denominator; show delivery information beside them.
| Measure | What to record | What to examine |
|---|---|---|
| Click rate | Unique recipients with a recorded click divided by recipients sent the simulation. | Interpret alongside difficulty, delivery, and event definitions. |
| Report rate | Unique recipients who report the simulation through an approved channel divided by recipients sent it. | Check whether people can find and use the reporting route. |
| Reporting before an unsafe action | Count recipients whose report precedes any recorded click or other defined unsafe action, where event ordering is available. | Distinguish early recognition from reporting after a mistake; preserve both as useful signals. |
| Time to report | Elapsed time from delivery to report for people who reported; show the median and number of reporters. | Display reporting coverage beside speed. People who never report must not disappear from the interpretation. |
| Report handling | Time from report receipt to acknowledgement and triage, where that workflow is included in the exercise. | Identify response delays separately from employee recognition delays. |
| Follow-up completion | Agreed improvements, their owners, due dates, and evidence of completion. | Check whether the review produced an operational change that can be reassessed. |
Keep simulated-message reporting separate from real suspicious-message reporting. For real reports, record the team's disposition, such as confirmed malicious, benign, or unresolved. A simulation score and the handling of actual threats answer different questions.
Turn each finding into a specific next action
Use the scorecard to choose a response proportionate to the evidence:
- Clicks increased: inspect difficulty, audience changes, delivery, and event quality before assigning additional training. Review the requested action and the decision that needs practice.
- Few people reported: test the approved reporting route on the devices employees use. Confirm that instructions are easy to find and that people receive acknowledgement.
- Reports arrived quickly but waited for review: assign an owner for the queue and clarify coverage and escalation. Another employee lesson will not repair an unowned response process.
- Results improved: document what else changed and reassess under comparable conditions. Avoid claiming causation from a single before-and-after comparison.
- A lesson was completed but the same decision remains difficult: review the lesson's relevance and the surrounding workflow with the affected team.
Encourage employees to report suspected mistakes promptly, including after clicking. Use private, constructive follow-up and ask what made the message plausible or the reporting process difficult. NIST's SP 800-50 Revision 1, finalized September 12, 2024, treats cybersecurity learning as an ongoing program with behavior change, evaluation, and improvement. Completion records serve a purpose, but the review should also examine what people and the organization can do.
Ask a managed training provider for the evidence
Before agreeing to a program or renewing one, request an anonymized sample report and use it to answer these questions:
- Can we see the people counts, denominators, campaign conditions, and event definitions behind each result?
- How will scenario difficulty and relevance be assessed for our employees?
- Which reporting and timing measures are available, and which would require additional workflow or tooling?
- Who reviews findings, arranges follow-up, and verifies that agreed improvements happened?
- What campaign frequency, customization, administration, and reporting are included in the proposal?
Network Box USA's current Security Awareness Training service includes simulated phishing campaigns, click-rate and behavior reporting, targeted e-learning, employee progress reports, and managed campaign operations. Confirm the specific measures, reporting channels, and responsibilities in the agreed scope; the scorecard above is a set of evaluation criteria, not a claim that every metric is included automatically.
Training should also fit the technical protections around email. Network Box USA's Managed Cloud Email Security filters email threats, while awareness training addresses how people recognize and report social engineering that reaches them. Review the responsibilities for both when deciding what an exercise should measure.
For your next review, bring the last campaign report, its message and audience details, and one improvement you want to verify. Discuss a managed awareness program with Network Box USA to define practical training, reporting, and follow-up for your team.