An audio-visual particle filter for speaker tracking on the CLEAR'06 evaluation dataset

Kai Nickel; Tobias Gehrig; Hazim K. Ekenel; John McDonough; Rainer Stiefelhagen

doi:10.1007/978-3-540-69568-4_4

An audio-visual particle filter for speaker tracking on the CLEAR'06 evaluation dataset

Kai Nickel^*
, Tobias Gehrig
, Hazim K. Ekenel
, John McDonough
, Rainer Stiefelhagen

^*Bu çalışma için yazışmadan sorumlu yazar

Karlsruhe Institute of Technology

Araştırma sonucu: Kitap/Rapor/Konferans Bildirisinde Bölüm › Konferans katkısı › bilirkişi

6 Atıf (Scopus)

Özet

We present an approach for tracking a lecturer during the course of his speech. We use features from multiple cameras and microphones, and process them in a joint particle filter framework. The filter performs sampled projections of 3D location hypotheses and scores them using features from both audio and video. On the video side, the features are based on foreground segmentation, multi-view face detection and upper body detection. On the audio side, the time delays of arrival between pairs of microphones are estimated with a generalized cross correlation function. In the CLEAR'06 evaluation, the system yielded a tracking accuracy (MOTA) of 71% for video-only, 55% for audio-only and 90% for combined audio-visual tracking.

Orijinal dil	İngilizce
Ana bilgisayar yayını başlığı	Multimodal Technologies for Perception of Humans - First International Evaluation Workshop on Classification of Events, Activities and Relationships, CLEAR 2006 Revised Selected Papers
Yayınlayan	Springer Verlag
Sayfalar	69-80
Sayfa sayısı	12
ISBN (Basılı)	9783540695677
DOI'lar	https://doi.org/10.1007/978-3-540-69568-4_4
Yayın durumu	Yayınlandı - 2007
Harici olarak yayınlandı	Evet
Etkinlik	1st International Evaluation Workshop on Classification of Events, Activities and Relationships, CLEAR 2006 - Southhampton, United Kingdom Süre: 6 Nis 2006 → 7 Nis 2006

Yayın serisi

Adı	Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Hacim	4122 LNCS
ISSN (Basılı)	0302-9743
ISSN (Elektronik)	1611-3349

???event.eventtypes.event.conference???

???event.eventtypes.event.conference???	1st International Evaluation Workshop on Classification of Events, Activities and Relationships, CLEAR 2006
Ülke/Bölge	United Kingdom
Şehir	Southhampton
Periyot	6/04/06 → 7/04/06

Belgeye Erişim

10.1007/978-3-540-69568-4_4

Diğer Dosyalar ve Linkler

Link to publication in Scopus

Alıntı Yap

Nickel, K., Gehrig, T., Ekenel, H. K., McDonough, J., & Stiefelhagen, R. (2007). An audio-visual particle filter for speaker tracking on the CLEAR'06 evaluation dataset. In Multimodal Technologies for Perception of Humans - First International Evaluation Workshop on Classification of Events, Activities and Relationships, CLEAR 2006 Revised Selected Papers (pp. 69-80). (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Hac. 4122 LNCS). Springer Verlag. https://doi.org/10.1007/978-3-540-69568-4_4

Nickel, Kai ; Gehrig, Tobias ; Ekenel, Hazim K. et al. / An audio-visual particle filter for speaker tracking on the CLEAR'06 evaluation dataset. Multimodal Technologies for Perception of Humans - First International Evaluation Workshop on Classification of Events, Activities and Relationships, CLEAR 2006 Revised Selected Papers. Springer Verlag, 2007. pp. 69-80 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)).

@inproceedings{3498bb9cf8fe4c43a22113c094a32598,

title = "An audio-visual particle filter for speaker tracking on the CLEAR'06 evaluation dataset",

abstract = "We present an approach for tracking a lecturer during the course of his speech. We use features from multiple cameras and microphones, and process them in a joint particle filter framework. The filter performs sampled projections of 3D location hypotheses and scores them using features from both audio and video. On the video side, the features are based on foreground segmentation, multi-view face detection and upper body detection. On the audio side, the time delays of arrival between pairs of microphones are estimated with a generalized cross correlation function. In the CLEAR'06 evaluation, the system yielded a tracking accuracy (MOTA) of 71\% for video-only, 55\% for audio-only and 90\% for combined audio-visual tracking.",

author = "Kai Nickel and Tobias Gehrig and Ekenel, \{Hazim K.\} and John McDonough and Rainer Stiefelhagen",

year = "2007",

doi = "10.1007/978-3-540-69568-4\_4",

language = "English",

isbn = "9783540695677",

series = "Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)",

publisher = "Springer Verlag",

pages = "69--80",

booktitle = "Multimodal Technologies for Perception of Humans - First International Evaluation Workshop on Classification of Events, Activities and Relationships, CLEAR 2006 Revised Selected Papers",

address = "Germany",

note = "1st International Evaluation Workshop on Classification of Events, Activities and Relationships, CLEAR 2006 ; Conference date: 06-04-2006 Through 07-04-2006",

}

Nickel, K, Gehrig, T, Ekenel, HK, McDonough, J & Stiefelhagen, R 2007, An audio-visual particle filter for speaker tracking on the CLEAR'06 evaluation dataset. in Multimodal Technologies for Perception of Humans - First International Evaluation Workshop on Classification of Events, Activities and Relationships, CLEAR 2006 Revised Selected Papers. Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), hac. 4122 LNCS, Springer Verlag, pp. 69-80, 1st International Evaluation Workshop on Classification of Events, Activities and Relationships, CLEAR 2006, Southhampton, United Kingdom, 6/04/06. https://doi.org/10.1007/978-3-540-69568-4_4

An audio-visual particle filter for speaker tracking on the CLEAR'06 evaluation dataset. / Nickel, Kai; Gehrig, Tobias; Ekenel, Hazim K. et al.
Multimodal Technologies for Perception of Humans - First International Evaluation Workshop on Classification of Events, Activities and Relationships, CLEAR 2006 Revised Selected Papers. Springer Verlag, 2007. p. 69-80 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Hac. 4122 LNCS).

Araştırma sonucu: Kitap/Rapor/Konferans Bildirisinde Bölüm › Konferans katkısı › bilirkişi

TY - GEN

T1 - An audio-visual particle filter for speaker tracking on the CLEAR'06 evaluation dataset

AU - Nickel, Kai

AU - Gehrig, Tobias

AU - Ekenel, Hazim K.

AU - McDonough, John

AU - Stiefelhagen, Rainer

PY - 2007

Y1 - 2007

N2 - We present an approach for tracking a lecturer during the course of his speech. We use features from multiple cameras and microphones, and process them in a joint particle filter framework. The filter performs sampled projections of 3D location hypotheses and scores them using features from both audio and video. On the video side, the features are based on foreground segmentation, multi-view face detection and upper body detection. On the audio side, the time delays of arrival between pairs of microphones are estimated with a generalized cross correlation function. In the CLEAR'06 evaluation, the system yielded a tracking accuracy (MOTA) of 71% for video-only, 55% for audio-only and 90% for combined audio-visual tracking.

AB - We present an approach for tracking a lecturer during the course of his speech. We use features from multiple cameras and microphones, and process them in a joint particle filter framework. The filter performs sampled projections of 3D location hypotheses and scores them using features from both audio and video. On the video side, the features are based on foreground segmentation, multi-view face detection and upper body detection. On the audio side, the time delays of arrival between pairs of microphones are estimated with a generalized cross correlation function. In the CLEAR'06 evaluation, the system yielded a tracking accuracy (MOTA) of 71% for video-only, 55% for audio-only and 90% for combined audio-visual tracking.

UR - https://www.scopus.com/pages/publications/38049178208

U2 - 10.1007/978-3-540-69568-4_4

DO - 10.1007/978-3-540-69568-4_4

M3 - Conference contribution

AN - SCOPUS:38049178208

SN - 9783540695677

T3 - Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)

SP - 69

EP - 80

BT - Multimodal Technologies for Perception of Humans - First International Evaluation Workshop on Classification of Events, Activities and Relationships, CLEAR 2006 Revised Selected Papers

PB - Springer Verlag

T2 - 1st International Evaluation Workshop on Classification of Events, Activities and Relationships, CLEAR 2006

Y2 - 6 April 2006 through 7 April 2006

ER -

Nickel K, Gehrig T, Ekenel HK, McDonough J, Stiefelhagen R. An audio-visual particle filter for speaker tracking on the CLEAR'06 evaluation dataset. In Multimodal Technologies for Perception of Humans - First International Evaluation Workshop on Classification of Events, Activities and Relationships, CLEAR 2006 Revised Selected Papers. Springer Verlag. 2007. p. 69-80. (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)). doi: 10.1007/978-3-540-69568-4_4

An audio-visual particle filter for speaker tracking on the CLEAR'06 evaluation dataset

Özet

Yayın serisi

???event.eventtypes.event.conference???

Belgeye Erişim

Diğer Dosyalar ve Linkler

Parmak izi

Alıntı Yap