The typical social engineering test is a phishing simulation. A few thousand employees receive a fake email. The report shows that 14 percent clicked the link and 6 percent entered their password. Management is concerned. The awareness team schedules extra training for the people who clicked. The next simulation shows 11 percent. Progress is reported.
The report says nothing about the question that matters most: if a real attacker tried, would they succeed?
A familiar situation
A real attack on an organisation does not look like a single email. It starts with an attacker studying the organisation's website and public profiles. They call the service desk pretending to be a new employee from a regional office, explaining that their laptop was not delivered and that they need access to email urgently. The service desk employee wants to help and follows the procedure for password resets, which requires the employee number and date of birth. Both can be found or guessed. The attacker then sends a message in the collaboration platform to a colleague in finance, referring to an urgent payment requested by a director who is travelling.
No email was clicked. Every individual step looked reasonable. The weakness was in the process: identity verification at the service desk relied on information that is not secret, and there was no check when a newly reset account immediately started requesting payments.
A phishing simulation would not have found any of this.
Why click rates are a weak measure
- They measure one channel. Attackers use phone, chat, SMS, physical access and suppliers as well. Many effective attacks avoid email altogether.
- They depend on the email. A difficult simulation produces a high click rate, an easy one a low rate. Comparing results over time says more about the test design than about the organisation.
- They focus on the first mistake. In a real attack, one click is the beginning. What matters is what happens next: is it reported, detected and contained?
- They blame individuals. When results are reported per person or per department, the message is that the problem lies with the people who clicked. Often it lies with the process that made the request seem normal.
What should be tested instead
Verification
The most important control against social engineering is consistent verification of unusual requests through a channel the employee chooses, not the one offered by the requester. Testing should check whether this happens in practice: at the service desk, in finance, in HR, at reception. Does the employee call back on a known number, or on the number in the email signature? Does the procedure rely on information an attacker can find?
Escalation
When an employee has doubts, do they know whom to contact? Is that person available? Is the employee supported when they delay a request from someone who appears to be senior? A good test includes moments where the employee needs to escalate, and records whether that route works.
Incident reporting
Reports are often the most valuable outcome of a test. How quickly does the first report arrive? Does it reach someone who can act? Is the attack contained, for example by blocking the sender or warning other employees, before more people are affected? An organisation where many people click but the first report arrives within five minutes and the security team responds immediately may be in a better position than one with a low click rate and no reports.
Management behaviour
Social engineering often exploits authority. Attackers pretend to be directors because employees hesitate to question them. If managers regularly bypass procedures themselves, for example by asking for urgent payments through chat, employees learn that the procedure is optional. Testing should, with the agreement of the board, include scenarios that involve senior roles, and the results should be discussed at that level.
Multiple channels
Realistic scenarios combine channels: an email followed by a phone call, a chat message that refers to a meeting, a visitor at reception who mentions a name found on LinkedIn. Generative AI makes it easier for attackers to imitate voices, faces and writing styles, which makes the combination more convincing. The answer is not to teach employees to detect deepfakes. It is to make sure that the verification habit does not depend on recognising someone.
How a process-based test is built
A test that looks at processes starts with the question of what an attacker would want to achieve. Typical objectives are a fraudulent payment, access to an account with sensitive data, physical entry to a restricted area or information that enables a later attack. The scenario is then designed backwards from that objective.
A typical sequence looks like this:
- Reconnaissance. Collect what can be found publicly: organisation charts, names of suppliers, job postings that reveal systems in use, posts by employees about events or travel. This alone often shows how easily a credible pretext can be built.
- Scenario design. Choose a small number of realistic scenarios across channels, each aimed at a specific process: password reset, payment change, visitor access, supplier support.
- Agreement. Confirm the scenarios, boundaries and stop criteria with the informed contacts in the organisation.
- Execution. Run the scenarios in a controlled way, recording each step: who was contacted, what was asked, what verification took place, whether it was reported.
- Analysis. For each scenario, identify where the process held and where it did not, and why. Was the procedure unclear, impractical or simply not known?
- Debrief and improvement. Discuss the findings with process owners and management, agree changes and use the scenarios in training.
Social engineering testing is most informative when combined with technical testing. If an attacker obtains a password through a phone call, what can they reach with it? Does multi-factor authentication stop them, or can it be reset through the same service desk? Following the chain from first contact to actual access shows the real impact, and often reveals that the technical and human controls were designed separately.
Scope and ethics
Social engineering testing involves real employees in situations designed to deceive them. That requires care:
- Agree scenarios, boundaries and stop criteria in advance with a small group of informed people in the organisation.
- Avoid scenarios that cause real distress, such as fake medical news or threats of dismissal.
- Do not use test results to sanction individual employees. Report at the level of processes and roles.
- Inform and debrief employees afterwards, so that the test becomes a learning moment and not a source of mistrust.
- Take into account privacy rules and, where applicable, the role of the works council.
A test that damages trust in the organisation undermines the very behaviour it is meant to strengthen.
Metrics that mean something
Useful indicators look at the whole chain:
- the proportion of unusual requests that were verified through an independent channel;
- the time until the first report and until containment;
- the proportion of scenarios that reached their objective, such as access, payment or information;
- the procedural weaknesses found and whether they were fixed;
- changes in these indicators between rounds of testing.
These figures are more difficult to produce than a click rate. They are also far more useful for deciding what to improve.
Three things to do this quarter
- Review the password reset procedure at your service desk. Check which identity questions an outsider could answer using public information.
- Agree one simple verification rule for payment changes and urgent requests, such as "always call back on a number you already have", and make sure managers follow it too.
- Measure reporting in your next phishing exercise: how fast the first report came in and what happened next. Report that number alongside the click rate.
Conclusion
Social engineering succeeds when a convincing story meets a process that cannot tell the difference. Testing should therefore look at verification, escalation, reporting and management behaviour across several channels, not only at who clicked. The goal is not to catch employees out. It is to find where the organisation's routines let an attacker through, and to fix them.
The Social Engineering Testing page describes how FLODISQ designs and runs multi-channel scenarios. The findings feed directly into practical training built around the routine that matters most: pause, verify and report.