An audio-visual particle filter for speaker tracking on the CLEAR'06 evaluation dataset

Kai Nickel; Tobias Gehrig; Hazim K. Ekenel; John McDonough; Rainer Stiefelhagen

doi:10.1007/978-3-540-69568-4_4

An audio-visual particle filter for speaker tracking on the CLEAR'06 evaluation dataset

Kai Nickel^*, Tobias Gehrig, Hazim K. Ekenel, John McDonough, Rainer Stiefelhagen

^*Corresponding author for this work

Karlsruhe Institute of Technology

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

6 Citations (Scopus)

Abstract

We present an approach for tracking a lecturer during the course of his speech. We use features from multiple cameras and microphones, and process them in a joint particle filter framework. The filter performs sampled projections of 3D location hypotheses and scores them using features from both audio and video. On the video side, the features are based on foreground segmentation, multi-view face detection and upper body detection. On the audio side, the time delays of arrival between pairs of microphones are estimated with a generalized cross correlation function. In the CLEAR'06 evaluation, the system yielded a tracking accuracy (MOTA) of 71% for video-only, 55% for audio-only and 90% for combined audio-visual tracking.

Original language	English
Title of host publication	Multimodal Technologies for Perception of Humans - First International Evaluation Workshop on Classification of Events, Activities and Relationships, CLEAR 2006 Revised Selected Papers
Publisher	Springer Verlag
Pages	69-80
Number of pages	12
ISBN (Print)	9783540695677
DOIs	https://doi.org/10.1007/978-3-540-69568-4_4
Publication status	Published - 2007
Externally published	Yes
Event	1st International Evaluation Workshop on Classification of Events, Activities and Relationships, CLEAR 2006 - Southhampton, United Kingdom Duration: 6 Apr 2006 → 7 Apr 2006

Publication series

Name	Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Volume	4122 LNCS
ISSN (Print)	0302-9743
ISSN (Electronic)	1611-3349

Conference

Conference	1st International Evaluation Workshop on Classification of Events, Activities and Relationships, CLEAR 2006
Country/Territory	United Kingdom
City	Southhampton
Period	6/04/06 → 7/04/06

Access to Document

10.1007/978-3-540-69568-4_4

Cite this

Nickel, K., Gehrig, T., Ekenel, H. K., McDonough, J., & Stiefelhagen, R. (2007). An audio-visual particle filter for speaker tracking on the CLEAR'06 evaluation dataset. In Multimodal Technologies for Perception of Humans - First International Evaluation Workshop on Classification of Events, Activities and Relationships, CLEAR 2006 Revised Selected Papers (pp. 69-80). (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 4122 LNCS). Springer Verlag. https://doi.org/10.1007/978-3-540-69568-4_4

Nickel, Kai ; Gehrig, Tobias ; Ekenel, Hazim K. et al. / An audio-visual particle filter for speaker tracking on the CLEAR'06 evaluation dataset. Multimodal Technologies for Perception of Humans - First International Evaluation Workshop on Classification of Events, Activities and Relationships, CLEAR 2006 Revised Selected Papers. Springer Verlag, 2007. pp. 69-80 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)).

@inproceedings{3498bb9cf8fe4c43a22113c094a32598,

title = "An audio-visual particle filter for speaker tracking on the CLEAR'06 evaluation dataset",

abstract = "We present an approach for tracking a lecturer during the course of his speech. We use features from multiple cameras and microphones, and process them in a joint particle filter framework. The filter performs sampled projections of 3D location hypotheses and scores them using features from both audio and video. On the video side, the features are based on foreground segmentation, multi-view face detection and upper body detection. On the audio side, the time delays of arrival between pairs of microphones are estimated with a generalized cross correlation function. In the CLEAR'06 evaluation, the system yielded a tracking accuracy (MOTA) of 71% for video-only, 55% for audio-only and 90% for combined audio-visual tracking.",

author = "Kai Nickel and Tobias Gehrig and Ekenel, {Hazim K.} and John McDonough and Rainer Stiefelhagen",

year = "2007",

doi = "10.1007/978-3-540-69568-4_4",

language = "English",

isbn = "9783540695677",

series = "Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)",

publisher = "Springer Verlag",

pages = "69--80",

booktitle = "Multimodal Technologies for Perception of Humans - First International Evaluation Workshop on Classification of Events, Activities and Relationships, CLEAR 2006 Revised Selected Papers",

address = "Germany",

note = "1st International Evaluation Workshop on Classification of Events, Activities and Relationships, CLEAR 2006 ; Conference date: 06-04-2006 Through 07-04-2006",

}

Nickel, K, Gehrig, T, Ekenel, HK, McDonough, J & Stiefelhagen, R 2007, An audio-visual particle filter for speaker tracking on the CLEAR'06 evaluation dataset. in Multimodal Technologies for Perception of Humans - First International Evaluation Workshop on Classification of Events, Activities and Relationships, CLEAR 2006 Revised Selected Papers. Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), vol. 4122 LNCS, Springer Verlag, pp. 69-80, 1st International Evaluation Workshop on Classification of Events, Activities and Relationships, CLEAR 2006, Southhampton, United Kingdom, 6/04/06. https://doi.org/10.1007/978-3-540-69568-4_4

An audio-visual particle filter for speaker tracking on the CLEAR'06 evaluation dataset. / Nickel, Kai; Gehrig, Tobias; Ekenel, Hazim K. et al.
Multimodal Technologies for Perception of Humans - First International Evaluation Workshop on Classification of Events, Activities and Relationships, CLEAR 2006 Revised Selected Papers. Springer Verlag, 2007. p. 69-80 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 4122 LNCS).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

TY - GEN

T1 - An audio-visual particle filter for speaker tracking on the CLEAR'06 evaluation dataset

AU - Nickel, Kai

AU - Gehrig, Tobias

AU - Ekenel, Hazim K.

AU - McDonough, John

AU - Stiefelhagen, Rainer

PY - 2007

Y1 - 2007

N2 - We present an approach for tracking a lecturer during the course of his speech. We use features from multiple cameras and microphones, and process them in a joint particle filter framework. The filter performs sampled projections of 3D location hypotheses and scores them using features from both audio and video. On the video side, the features are based on foreground segmentation, multi-view face detection and upper body detection. On the audio side, the time delays of arrival between pairs of microphones are estimated with a generalized cross correlation function. In the CLEAR'06 evaluation, the system yielded a tracking accuracy (MOTA) of 71% for video-only, 55% for audio-only and 90% for combined audio-visual tracking.

AB - We present an approach for tracking a lecturer during the course of his speech. We use features from multiple cameras and microphones, and process them in a joint particle filter framework. The filter performs sampled projections of 3D location hypotheses and scores them using features from both audio and video. On the video side, the features are based on foreground segmentation, multi-view face detection and upper body detection. On the audio side, the time delays of arrival between pairs of microphones are estimated with a generalized cross correlation function. In the CLEAR'06 evaluation, the system yielded a tracking accuracy (MOTA) of 71% for video-only, 55% for audio-only and 90% for combined audio-visual tracking.

UR - http://www.scopus.com/inward/record.url?scp=38049178208&partnerID=8YFLogxK

U2 - 10.1007/978-3-540-69568-4_4

DO - 10.1007/978-3-540-69568-4_4

M3 - Conference contribution

AN - SCOPUS:38049178208

SN - 9783540695677

T3 - Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)

SP - 69

EP - 80

BT - Multimodal Technologies for Perception of Humans - First International Evaluation Workshop on Classification of Events, Activities and Relationships, CLEAR 2006 Revised Selected Papers

PB - Springer Verlag

T2 - 1st International Evaluation Workshop on Classification of Events, Activities and Relationships, CLEAR 2006

Y2 - 6 April 2006 through 7 April 2006

ER -

Nickel K, Gehrig T, Ekenel HK, McDonough J, Stiefelhagen R. An audio-visual particle filter for speaker tracking on the CLEAR'06 evaluation dataset. In Multimodal Technologies for Perception of Humans - First International Evaluation Workshop on Classification of Events, Activities and Relationships, CLEAR 2006 Revised Selected Papers. Springer Verlag. 2007. p. 69-80. (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)). doi: 10.1007/978-3-540-69568-4_4

An audio-visual particle filter for speaker tracking on the CLEAR'06 evaluation dataset

Abstract

Publication series

Conference

Access to Document

Other files and links

Fingerprint

Cite this