A better understanding of machine learning malware misclassifcation

Nada Alruhaily*, Tom Chothia, Behzad Bordbar

*Corresponding author for this work

Research output: Chapter in Book/Report/Conference proceedingConference contribution


Machine learning-based malware detection systems have been widely suggested and used as a replacement for signature-based detection methods. Such systems have shown that they can provide a high detection rate when recognising non-previously seen malware samples. However, when classifying malware based on their behavioural features, some new malware can go undetected, resulting in a misclassification. Our aim is to gain more understanding of the underlying causes of malware misclassification; this will help to develop more robust malware detection systems. Towards this objective, several questions have been addressed in this paper: Does misclassification increase over a period of time? Do changes that affect classification occur in malware at the level of families, where all instances that belong to certain families are hard to detect? Alternatively, can such changes be traced back to certain malware variants instead of families? Also, does misclassification increase when removing distinct API functions that have been used only by malware? As this technique could be used by malware writers to evade the detection. Our experiments showed that changes in malware behaviour are mostly due to behavioural changes at the level of variants across malware families, where variants did not behave as expected. It also showed that machine learning-based systems could maintain a high detection rate even in the case of trying to evade the detection by not using distinct API functions, which are uniquely used by malware.

Original languageEnglish
Title of host publicationInformation Systems Security and Privacy - 3rd International Conference, ICISSP 2017, Revised Selected Papers
EditorsPaolo Mori, Steven Furnell, Olivier Camp
PublisherSpringer Verlag
Number of pages24
ISBN (Electronic)9783319933542
ISBN (Print)9783319933535
Publication statusPublished - 1 Jan 2018
Event3rd International Conference on Information Systems Security and Privacy, ICISSP 2017 - Porto, Portugal
Duration: 19 Feb 201721 Feb 2017

Publication series

NameCommunications in Computer and Information Science
ISSN (Print)1865-0929


Conference3rd International Conference on Information Systems Security and Privacy, ICISSP 2017


  • Behavioural analysis
  • Classification
  • Machine learning
  • Malware

ASJC Scopus subject areas

  • Computer Science(all)
  • Mathematics(all)


Dive into the research topics of 'A better understanding of machine learning malware misclassifcation'. Together they form a unique fingerprint.

Cite this