|
A Comparative Annotator-agreement Analysis of Emotional Speech Corpora |
|---|---|
| รหัสดีโอไอ | |
| Creator | Piyawat Sukhummek |
| Title | A Comparative Annotator-agreement Analysis of Emotional Speech Corpora |
| Contributor | Jessada Karnjana, Sawit Kasuriya, Chai Wutiwiwatchai, Thanaruk Theeramunkong |
| Publisher | Sirindhorn International Institute of Technology, Bangkadi Campus (SIIT-BKD) |
| Publication Year | 2561 |
| Journal Title | Journal of Intelligent Informatics and Smart Technology |
| Journal Vol. | 3 |
| Page no. | 1-6 |
| Keyword | annotator-agreement analysis, inter-annotator reliability measurement, IEMOCAP corpus, EMOLA corpus, HMM-based emotion recognition |
| URL Website | https://ph05.tci-thaijo.org/index.php/JIIST |
| Website title | Journal of Intelligent Informatics and Smart Technology |
| ISSN | 2586-9167 |
| Abstract | This paper proposes three methods for removing or filtering out ambiguous utterances: the filtering based on the first label preference and majority vote, the filtering based on full consensus, and the filtering based on the first label preference and full consensus. We investigate two corpora, which are Interactive Emotional Dyadic Motion Capture Database (IEMOCAP) and Emotional Tagged Corpus on Lakorn (EMOLA). The first corpus is an English language corpus whereas the second one is a Thai language corpus, and both are annotated by six annotators. We primarily study only four emotions, which are anger, happiness, neutral, and sadness. The experimental results show that, once the emotionally ambiguous utterances are removed from a corpus by the proposed methods, and then the corpora are used in training and testing emotion recognition models, the accuracy results improve considerably compared with those of emotion recognition models trained and tested by the original corpora. In the best case, the accuracy improves by 37.47 percents. Also, the proposed methods can considerably improve the reliability of agreement among annotators. |
