By Amal Ahmed Anda
Internet browsers such as Google Chrome and Firefox inform users about potential
threats by presenting warning messages. However, they always leave the final decision
to the users regarding whether to return to the safe situation or to continue to the
The high number of security warnings that users may face makes it difficult to
distinguish between severe or minor threats. The main issue is that most users click
through security warnings and ignore them.
Design effective security warnings with interfaces that influence users’ attention
and comprehension helping them with decision making.
HOW Is SECURITY WARNING DESIGNED AND EVALUATED?
There are two standard methodologies to develop and evaluate security warnings:
1. Systematic Methodology: based on theories and guidelines such as following instructions:
a) Describe the risk precisely.
b) Be concise and accurate.
c) Offer meaningful options.
d) Present relevant contextual information.
e) Display some auditing notice.
f) Follow a consistent layout.
Figure 1 shows the suggested design for successful security warning dialog ..
Figure 1. “Warning design guideline “ .
2. Neuroscience Methodologies: since the cognitive neuroscience grew as a field study of
cyber security, studying and measuring the electrical activity of the brain in different regions
Neuroscience methodologies have two methods:
Neurological method and Psychophysiological method. a) The neurological method: this method includes two techniques which are:
Electroencephalography (EEG) and Functional magnetic resonance imaging (fMRI).
I. Electroencephalography (EEG): Medical and research fields deployed electrobiological
imaging tools to measure brain activities. Electroencephalography is one of these tools.
a large number of electric dipoles produce electric waves, and the EEG measures the
variation of this waves. EEG signal includes different brain waves that represent brain
activity based on electrodes were located in close to brain regions as shown in
Figure 2. Electrode cap with electrodes placed after 10-20 electrode placement system.
II. Functional magnetic resonance imaging (fMRI): fMRI produces cheap high-quality images with less noise. The fMRI is a useful method for measuring brain
activity. The Human-omputer Interaction (HCI) community have used fMRI to
evaluate system designs. Usually, to support the results tradition methods back
the fMRI, such as surveys, field and lab experiments, interviews . The fMRI
scanner takes two types of image wich are the brain’s anatomical images and
functional images that present variations of the BOLD signal as shown in
Figure 3. “Superimposing the Functional Images on the Structural Brain Images” 
b) The psychophysiological method: this method uses two techniques which are eye-
tracking and mouse-cursor- racking. Eye tracking technique measures the location
of a person’s eye movements. Thus, we can determine where a person is looking
and the sequence of actions. HCI researchers can collect information about the
interesting areas of system interfaces and the reasons that affect the usability of
system interfaces. Figure 4 illustrate how the tracker determines the directions of
person’s eye using “corneal-reflection /pupil-centre” method .
Figure 4. The Corneal reflection position according to point of regard 
We reviewed literature relevant to design and evaluate the effectiveness of browsers’ warnings.
To get the whole picture of the security warning’s issues and achievements, we collected the
contributions of the presented studies and the ultimate effectiveness of their results on users’ decision
as they stated in their articles. Table 1 Illustrates the input of each research, goals, and achievement.
Moreover, the used methods and study’s environment should be considered to address the real
progress of each study.
Table 2. The contributions of each paper
Although most of the studies conducted in laboratories, none of them achieved their goals and
affected users behavior making them heeding the security warnings. It is noticeable that most of
the studies focused on comprehension and attention as separated goals. Some of them
manipulated the content of the security warnings to understand the effects on the user’s
behavior but did not consider the initial warning ignorance when users do not read the warning
at all. On the other hand, some studies manipulated the colors and shapes of security warnings
but did not consider that when users pay attention to the warning, they will click through it
because they did not understand the message. Also, most of the studies that focused on
catching users’ attention, measured their success by the time users spent looking at the
warning. However, the authors in  did not find any relation between exposure to the attack
and the amount of time spent viewing the warnings.
ANALYSIS OF SOME PROPOSED WARNING DESIGNS
In this section, I present insight into the designs of some proposed warnings.
1. Mozilla Firefox’s SSL warning : It is the most powerful warning in the collected studies.
As we see in figure 5, the message described the risks providing a question related to the
presented meaningful options, and the recommended option is the first choice. However,
the warning did not describe the risk and consequences clearly. Not all the options have specific
meaning such as “I understand the risk”. Is that means “I want to continue to the page” or
“I do not want more details
information are missed although it still has a structure that is very close to the advanced design
for effective security warning dialog figure 1.
Figure 5. The SSL warning of Mozilla Firefox.
2. Silic et al.  suggested alternative ways to increase secure attitudes, beliefs, and behavior
in warning messages as shown in Figure 6.
Figure 6. “Warning message from pilot study” 
However, the advice did not provide meaningful options the only “Exit” and “Continue”
without any “Help” or “Detail” choices, and even it did not offer any information about the threat
at all. Users can not take the right decision based on this information. The most worth noticeable
feature are the two clear visible buttons “Exit” and “Continue”. Users will click on the continue
button even before reading the text.
3. Bravo-Lillo et al.  presented warning to lead users to read the warning words as shown in
figure 7, and they evaluated the influence of different warning design elements on guiding users
to change their attitudes.
Figure 7 “.Installation dialogs used in Experiment 1. Only the suspicious publisher (`Miicr0soft') is shown. The top left dialog is the control (no attractors applied).”
The authors did not focus on choosing the warnings’ words, which could affect users’ behaviors
and decisions. Moreover, all the suggested warning did not meet the criteria. First of all, the
alerts did not describe the threat apparently using some ambiguous words. Second, the provided
options do not offer useful meaning. Finally, The close icons are still enabled in all the warnings,
so users can close the warning dialog before even read the words.
We compared the proposed warnings of the last studies against the warning guidelines.
We presented the result of the comparison in Table 2.
Table2. The warning designs comparison against security warning design guidelines.
Based on this result, all the presented warnings did not meet the requirements of both
comprehension and attention. Maybe that is why they did not succeed in leading users to the
right decision. Mozilla Firefox’s SSL warning  was the only exception; it reached the
highest heeding rate of 67% in a semi-real environment . It integrated the two goals together
without satisfying all the criteria in . Therefore, future research should focus on designing
security warnings that satisfy most of the criteria focused on both comprehension and
attention. Then, if they do not succeed in catching users’ attention and in guiding them
to make the right decision, enhancement of the warning methods and techniques will be
Published accounts of the harmful effects of malware, spyware, phishing and denial of service
have raised awareness of Internet threats, along with the need for users at all levels to keep their
information and systems secure. We gathered different studies about one of the most critical
related subjects in warning usability. That is browser’s security warning used to alert Internet
users to potential threats. The state of the art design presented in each of the study was
investigated. We provided deep insights into the presented designs and results. Then, we
compared them with each other and against the warning design guidelines to highlight the
related challenges. The main issue was that, although some studies partially reached their goal,
none of them affected users’ behavior making them heed the warnings. Systematic methods can
make a difference. The effectiveness of Mozilla Firefox SSL warning was evidence. It guided
67% of the participants to make the right decision. Also, we determined that the related
challenges indicated to the further studies areas such as combining the efforts of catching user
attention in the first phase. Then reaching an understandable message is the second phase.
Indeed, this challenge needs to be solved before browser’s security warnings become a part of
Akhawe, D., & Felt, A. P. (2013, August). Alice in Warningland:
Effectiveness. In Usenix security (pp. 257-272)
Anderson, B. B., Kirwan, C. B., Jenkins, J. L., Eargle, D.,
Howard, S., & Vance, A. (2015, April). How polymorphic
Anderson, B. B., Kirwan, C. B., Eargle, D., Jensen, S. R., &
Vance, A. (2015). Neural correlates of gender differences and
websites: a neurosecurity study. Journal of Cybersecurity.
Bauer, L., Bravo-Lillo, C., Cranor, L. F., & Fragkaki, E. (2013).
Warning Design Guidelines (CMU-CyLab-13-002).
Bravo-Lillo, C., Cranor, L. F., Downs, J., Komanduri, S., Reeder,
R. W., Schechter, S., & Sleeper, M. (2013, July). Your Attention
Darwish, A., & Bataineh, E. (2012, December). Eye tracking
on (pp. 1-6). IEE
Dimoka, A. (2012). How to conduct a functional magnetic
resonance (fMRI) study in social science research. MIS Quarterly.
Egelman, S., Cranor, L. F., & Hong, J. (2008, April). You've been
phishing warnings. InProceedings of the SIGCHI Conference on
Human Factors in Computing Systems (pp. 1065-1074). ACM.
Egelman, S., & Schechter, S. (2013). The importance of being
Earnest [in security warnings]. In Financial cryptography and data
security (pp. 52-59). Springer Berlin Heidelberg.
Felt, A. P., Ainslie, A., Reeder, R. W., Consolvo, S., Thyagaraja, S.
, Bettes, A., .. & Grimes, J. (2015, April). Improving SSL
warnings: comprehension and adherence. In Proceedings of the
33rd Annual ACM Conference on Human Factors
in Computing Systems (pp. 2893-2902). ACM.
Kolb, N., Bartsch, S., Volkamer, M., & Vogt, J. (2014). Capturing attention for
security intervention. In Human Aspects of Information Security, Privacy, and
Trust (pp. 172-182). Springer International Publishing.
Silic, J. B. M. (2015). Warning! A Comprehensive Model of the Effects of
Sunshine, J., Egelman, S., Almuhimedi, H., Atri, N., & Cranor, L. F.
(2009, August). Crying Wolf: An Empirical Study of SSL Warning
Effectiveness. InUSENIX Security Symposium (pp. 399-416).
Strandvall, T. (2009). Eye tracking in human-computer interaction and
usability research. In Human-Computer Interaction–INTERACT 2009
(pp. 936-937). Springer Berlin Heidelberg.
Teplan, M. (2002). Fundamentals of EEG measurement.
Measurement science review, 2(2), 1-11.