Usability Evaluation of TV Interfaces: Subjective Evaluation Vs. Objective Evaluation

Cigdem Altin Gumussoy*, Aycan Pekpazar, Mustafa Esengun, Ayse Elvan Bayraktaroglu, Gokhan Ince

*Corresponding author for this work

Research output: Contribution to journalArticlepeer-review

15 Citations (Scopus)


This study aims to design a usability test enhanced with subjective usability evaluation methods (concurrent think-aloud, post-task and posttest questionnaires) and objective usability evaluation methods (eye-tracking, emotion recognition and logging technologies). Furthermore, the relationships between the subjective metrics such as task difficulty level, PSSUQ (Post-Study System Usability Questionnaire) scores and objective metrics such as task completion time, eye tracking metrics (fixation count, fixation duration, average fixation duration, saccade count, saccade duration, scanpath length, blink count), logging metrics (keying and back keying count) and emotion recognition metrics (negative emotions count) are aimed to be analyzed. Therefore, a user testing study with 38 participants was conducted to evaluate the usability level of the TV interface of Digiturk, which is one of the digital TV broadcasting platforms in Turkey. The participants completed ten tasks related to the features of the TV interface, such as VOD (video-on-demand) watching, channel locking, and recording for future watches. Whether there is a significant difference between tasks in terms of subjective and objective metrics is investigated by using one-way ANOVA. The results show that while task completion time and task difficulty increase, values of all objective metrics increase. The relationships between the subjective and objective metrics are measured with Pearson correlation and the results show that there is a significant relationship between every subjective and objective metric except PSSUQ scores, which shows the perceived satisfaction level of the participants. Furthermore, exploratory factor analysis is performed to explore the latent structure of the usability measures. As a result of the factor analysis, a two-factor structure in which objective and subjective metrics load on two separate factors is obtained.

Original languageEnglish
Pages (from-to)661-679
Number of pages19
JournalInternational Journal of Human-Computer Interaction
Issue number7
Publication statusPublished - 2022

Bibliographical note

Publisher Copyright:
© 2021 Taylor & Francis Group, LLC.


This work was supported by The Scientific and Technological Research Council of Turkey (TÜBİTAK) within the 3001 Program [Project number: 217M143, 2018]. The authors would like to thank the Digiturk beIN CONNECT company for their assistance during the project.

FundersFunder number
Türkiye Bilimsel ve Teknolojik Araştirma Kurumu217M143


    Dive into the research topics of 'Usability Evaluation of TV Interfaces: Subjective Evaluation Vs. Objective Evaluation'. Together they form a unique fingerprint.

    Cite this