Run-time evaluation of architectures: a case study of diversification in IoT

Research output: Contribution to journalArticle


Colleges, School and Institutes

External organisations

  • Loughborough University
  • Arab Academy for Science & Technology and Maritime Transport, Alexandria, Egypt
  • SEI/CMU, University of Hawaii, Honolulu, USA


Run-time properties of modern software system environments, such as Internet of Things (IoT), are a challenge for existing software architecture evaluation methods. Such systems are largely data-driven, characterized by their dynamism, unpredictability in operation, hyper-connectivity, and scale. Properties, such as performance, delayed delivery, and scalability, are acknowledged to pose great risk and are difficult to evaluate at design-time. Run-time evaluation could potentially be used to complement design-time evaluation, enabling significant deviations from the expected performance values to be captured. However, there are no systematic software architecture evaluation methods that intertwine and interleave design-time and run-time evaluation. This paper addresses this gap by proposing a novel run-time architecture evaluation method suited for systems that exhibit uncertainty and dynamism in their operation. Our method uses machine learning and cost-benefit analysis at run-time to continuously profile the architecture decisions made, to assess their added value. We demonstrate the applicability and effectiveness of this approach in the context of an IoT system architecture, where some architecture design decisions were diversified to meet Quality of Service (QoS) requirements. Our approach provides run-time assessment for these decisions which can inform deployment, refinement, and/or phasing-out decisions.


Original languageEnglish
Article number110428
JournalJournal of Systems and Software
Early online date24 Sep 2019
Publication statusPublished - Jan 2020


  • Design diversity, Internet of things, IoT, Run-time architecture evaluation, Runtime architecture evaluation, Software architectures for dynamic environments