Heartefacts: Augmenting mobile video sharing using wrist-worn heart rate sensors

Jo Vermeulen, Lindsay MacDonald, Johannes Schöning, Russell Beale, Sheelagh Carpendale

Research output: Chapter in Book/Report/Conference proceedingConference contribution

7 Citations (Scopus)


An increasing share of our daily interactions with others is mediated through mobile communication technologies. People communicate via text, emoticons, emojis and rich media such as video. We explore the design of Heartefacts, short video clips composed of highlights determined by heart rate changes while watching videos. Our survey investigated video sharing behaviour, and our feasibility study examined the possibility of detecting highlights in videos by monitoring people's heart rates measured with off-the-shelf wristworn sensors. Our results show that people do indeed have measurable responses with respect to their heartbeat patterns to six different emotions elicited by video clips. We compare video highlights verbally identified by our participants to physiological highlights as indicated by their heart rate data and also discuss and compare the automatically generated Heartefacts with video highlights created by an expert in video art. We close with design considerations for Heartefacts in mobile technology.

Original languageEnglish
Title of host publicationDIS 2016 - Proceedings of the 2016 ACM Conference on Designing Interactive Systems
Subtitle of host publicationFuse
PublisherAssociation for Computing Machinery
Number of pages12
ISBN (Electronic)978-1450340311
Publication statusPublished - 4 Jun 2016
Event11th ACM SIGCHI Conference on Designing Interactive Systems, DIS 2016 - Brisbane, Australia
Duration: 4 Jun 20168 Jun 2016


Conference11th ACM SIGCHI Conference on Designing Interactive Systems, DIS 2016


  • Affective computing
  • HR
  • Mobile computing
  • Smartwatch interaction
  • Video artefacts

ASJC Scopus subject areas

  • Computer Graphics and Computer-Aided Design
  • Human-Computer Interaction
  • Software


Dive into the research topics of 'Heartefacts: Augmenting mobile video sharing using wrist-worn heart rate sensors'. Together they form a unique fingerprint.

Cite this