Authors: Marc A. Kastner, Ichiro Ide, Yasutomo Kawanishi, Takatsugu Hirayama, Daisuke Deguchi, Hiroshi Murase

Abstract:

Natural Language Processing (NLP) is a key element in many real-world applications. However, the semantic gap is an ongoing problem, leading to unnatural results disconnected from the users. To understand semantics in real-world scenarios, human perception needs to be taken into consideration. Imageability is an approach to quantize human perception of words. Research shows a relationship between language usage and the imageability of words, making it useful for multimodal applications. However, imageability datasets are typically created by hand. In this research, a method using image data mining to estimate the imageability of words is proposed. The main assumption is a relationship between the imageability of concepts and crowd-sourced images. We use visual features from Web-crawled images to train a model to predict imageability. The model is evaluated using a test dataset. The proposed method can be used to increase the corpus of imageability dictionaries.

Type: Talk at 25th Annual Meeting of the Association for Natural Language Processing (言語処理学会第25回年次大会), no. A4-7, pp. 747-750

Publication date: March 2019