Publications
Major contributions are highlighted in yellow.
2025
Towards Visual Storytelling by Understanding Narrative Context through Scene-Graphs
Itthisak Phueaksri, Marc A. Kastner, Yasutomo Kawanishi, Takahiro Komamizu, Ichiro Ide. 31th Intl. Conf. on MultiMedia Modeling (MMM2025), To be published in Jan 2025.
Transformer-Based Audio Generation Conditioned by 2D Latent Maps: A Demonstration
Christian Limberg, Zhe Zhang, Marc A. Kastner. 31th Intl. Conf. on MultiMedia Modeling (MMM2025), To be published in Jan 2025.
Quantifying Image-Adjective Associations by Leveraging Large-Scale Pretrained Models
Chihaya Matsuhira, Marc A. Kastner, Takahiro Komamizu, Takatsugu Hirayama, Ichiro Ide. 31th Intl. Conf. on MultiMedia Modeling (MMM2025), To be published in Jan 2025.
2024
Stable Diffusionを用いた音の印象の可視化
梶友理香， 安冨七海， カストナーマークアウレル， 石光 俊介. 第26回 IEEE広島支部学生シンポジウム, Nov 2024.
Investigating conceptual blending of a diffusion model for improving nonword-to-image generation
Chihaya Matsuhira, Marc A. Kastner, Takahiro Komamizu, Takatsugu Hirayama, Ichiro Ide. Oral presentation at ACM Multimedia (ACMMM) 2024, Oct 2024. doi: 10.1145/3664647.3681202
[ presentation ] [ poster ] [ arXiv ]
歌唱時の口唇形状の類似性を考慮した歌詞の自動翻訳の検討
池田昂太郎， 松平茅隼， 加藤大貴， カストナーマークアウレル， 平山高嗣， 駒水孝裕， 井手一郎. Poster at MIRU Symposium (画像の認識・理解シンポジウム), Aug 2024.
Toward Visual Storytelling using Scene-Graph Contexts
Itthisak Phueaksri, Marc A. Kastner, Yasutomo Kawanishi, Takahiro Komamizu, Ichiro Ide. Poster at MIRU Symposium (画像の認識・理解シンポジウム), Aug 2024.
語感の定量化に向けた 画像に写る概念に対する視覚的印象の推定
松平茅隼， カストナーマークアウレル， 駒水孝裕， 平山高嗣， 井手一郎. Poster at MIRU Symposium (画像の認識・理解シンポジウム), Aug 2024.
MUWS ’2024: The 3rd International Workshop on Multimodal Human Understanding for the Web and Social Media
Marc A. Kastner, Gullal S. Cheema, Sherzod Hakimov, Noa Garcia. Workshop abstract at the 14th ACM International Conference on Multimedia Retrieval (ICMR) 2024, Jun 2024. doi: 10.1145/3652583.3658893
Interpolating the Text-to-Image Correspondence Based on Phonetic and Phonological Similarities for Nonword-to-Image Generation
Chihaya Matsuhira, Marc A. Kastner, Takahiro Komamizu, Takatsugu Hirayama, Keisuke Doman, Yasutomo Kawanishi, Ichiro Ide. Journal paper at IEEE Access, vol. 12, pp. 41299-41316, Mar 2024., Mar 2024. doi: 10.1109/ACCESS.2024.3378095
Computational Measurement of Perceived Pointiness from Pronunciation
Chihaya Matsuhira, Marc A. Kastner, Takahiro Komamizu, Ichiro Ide, Takatsugu Hirayama, Yasutomo Kawanishi, Keisuke Doman, Daisuke Deguchi. Journal paper at Multimedia Tools and Applications (MTAP), 83, 26183–26210, Mar 2024. doi: 10.1007/s11042-023-15732-z
Image-Collection Summarization using Scene-Graph Generation with External Knowledge
Itthisak Phueaksri, Marc A. Kastner, Yasutomo Kawanishi, Takahiro Komamizu, Ichiro Ide. Journal paper at IEEE Access, vol. 12, pp. 17499-17512, Jan 2024. doi: 10.1109/ACCESS.2024.3360113
2023
An Approach to Generate a Caption for an Image Collection using Scene Graph Generation
Itthisak Phueaksri, Marc A. Kastner, Yasutomo Kawanishi, Takahiro Komamizu, Ichiro Ide. Journal paper at IEEE Access, vol. 11, pp. 128245-128260, Nov 2023. doi: 10.1109/ACCESS.2023.3332098
Nonword-to-Image Generation Considering Perceptual Association of Phonetically Similar Words
Chihaya Matsuhira, Marc A. Kastner, Takahiro Komamizu, Takatsugu Hirayama, Keisuke Doman, Ichiro Ide. 1st International Workshop on Multimedia Content Generation and Evaluation (co-located at ACM Multimedia 2023), Oct 2023. doi: 10.1145/3607541.3616818
MUWS ’2023: The 2nd International Workshop on Multimodal Human Understanding for the Web and Social Media
Gullal S. Cheema, Sherzod Hakimov, Marc A. Kastner, Noa Garcia. Workshop proposal at the 32nd ACM International Conference on Information and Knowledge Management (CIKM) 2023, Oct 2023. doi: 10.1145/3583780.3615310
[ workshop website ] [ workshop proceedings ]
Image Collection Scene Graph Summarization Enhancing Relation Predictor with External Knowledge
Itthisak Phueaksri, Marc A. Kastner, Yasutomo Kawanishi, Takahiro Komamizu, Ichiro Ide. Poster at MIRU Symposium (画像の認識・理解シンポジウム), Jul 2023.
Leverage semantic alignment of object relations for image captioning
Da Huo, Marc A. Kastner, Takatsugu Hirayama, Takahiro Komamizu, Ichiro Ide. Poster at MIRU Symposium (画像の認識・理解シンポジウム), Jul 2023.
類音語の連想性を考慮した未知語の発音に対する画像生成
松平茅隼， カストナーマークアウレル， 駒水孝裕， 平山高嗣， 道満恵介， 井手一郎. Poster at MIRU Symposium (画像の認識・理解シンポジウム), Awarded with MIRU Interactive Session Award (MIRUインタラクティブ発表賞 受賞), Jul 2023.
Small object detection for bird with swin transformer
Da Huo, Marc A. Kastner, Tingwei Liu, Yasutomo Kawanishi, Takatsugu Hirayama, Takahiro Komamizu, Ichiro Ide. 18th International Conference on Machine Vision and Applications (MVA) Workshop, Jul 2023. doi: 10.23919/MVA57639.2023.10216093
MVA2023 Small Object Detection Challenge for Spotting Birds: Dataset, Methods, and Results
Yuki Kondo, Norimichi Ukita, Takayuki Yamaguchi, Hao-Yu Hou, Mu-Yi Shen, Chia-Chi Hsu, En-Ming Huang, Yu-Chen Huang, Yu-Cheng Xia, Chien-Yao Wang, Chun-Yi Lee, Da Huo, Marc A. Kastner, Tingwei Liu, Yasutomo Kawanishi, Takatsugu Hirayama, Takahiro Komamizu, Ichiro Ide, Yosuke Shinya, Xinyao Liu, Guang Liang, Syusuke Yasui. 18th International Conference on Machine Vision and Applications (MVA), Jul 2023. doi: 10.23919/MVA57639.2023.10215935
固定視点映像からの会話注意行動理解
松田有登， カストナーマークアウレル， 延原章平， 西野恒. Bachelor Thesis Spotlight Session, CVIM (コンピュータビジョンとイメージメディア研究会), May 2023.
Evoked emotion distribution learning through analysis of temporal user comments in social media videos
Yiming Wang, Marc A. Kastner, Da Huo, Takahiro Komamizu, Takatsugu Hirayama, Ichiro Ide. Talk at Meeting of the Technical Committee on Media Experience and Virtual Environment, MVE (メディアエクスペリエンス・バーチャル環境基礎研究会), Mar 2023.
IPA-CLIP: Integrating Phonetic Priors into Vision and Language Pretraining
Chihaya Matsuhira, Marc A. Kastner, Takahiro Komamizu, Takatsugu Hirayama, Keisuke Doman, Yasutomo Kawanishi, Ichiro Ide. arXiv pre-print 2303.03144, Mar 2023. doi: 10.48550/arXiv.2303.03144
Towards Captioning an Image Collection from a Combined Scene Graph Representation Approach
Itthisak Phueaksri, Marc A. Kastner, Yasutomo Kawanishi, Takahiro Komamizu, Ichiro Ide. 29th Intl. Conf. on MultiMedia Modeling (MMM2023), Jan 2023. doi: 10.1007/978-3-031-27077-2_14
2022
Detection of birds in a 3D environment referring to audio-visual information
Yasutomo Kawanishi, Ichiro Ide, Baidong Chu, Chihaya Matsuhira, Marc A. Kastner, Takahiro Komamizu, Daisuke Deguchi. 18th IEEE Intl. Conf. on Advanced Video and Signal-based Surveillance (AVSS2022), Nov 2022. doi: 10.1109/AVSS56176.2022.9959510
Action Semantic Alignment for Image Captioning
Da Huo, Marc A. Kastner, Takahiro Komamizu, Ichiro Ide. 5th IEEE International Conference on Multimedia Information Processing and Retrieval (MIPR2022), Aug 2022. doi: 10.1109/MIPR54900.2022.00041
Towards captioning an image collection using scene graph
Itthisak Phueaksri, Marc A. Kastner, Yasutomo Kawanishi, Takahiro Komamizu, Ichiro Ide. Poster at MIRU Symposium (画像の認識・理解シンポジウム), Jul 2022.
画像生成を介した語感から受ける印象の可視化の検討
松平茅隼， カストナーマークアウレル， 駒水孝裕， 平山高嗣， 道満恵介， 川西康友， 井手一郎. Poster at MIRU Symposium (画像の認識・理解シンポジウム), Jul 2022.
On Estimating Evoked Emotions of Social Media Videos through User Comments Analysis
Yiming Wang, Marc A. Kastner, Takahiro Komamizu, Yasutomo Kawanishi, Takatsugu Hirayama, Ichiro Ide. Poster at MIRU Symposium (画像の認識・理解シンポジウム), Jul 2022.
MUWS ’2022: The 1st International Workshop on Multimodal Understanding for the Web and Social Media
Sherzod Hakimov, Gullal S. Cheema, Marc A. Kastner, Rajiv Shah, Karan Sikka. Workshop proposal at The WebConf (ACM WWW) 2022, Apr 2022.
Personalized Fashion Recommendation using Pairwise Attention
Donnaphat Trakulwaranont, Marc A. Kastner, Shin'ichi Satoh. MultiMedia Modelling (MMM) 2022, Apr 2022. doi: 10.1007/978-3-030-98355-0_19
[ slides ] [ preprint ] [ poster ]
On Identifying Pareidolia Phenomenon by Emulating Patient Behavior
Zhaohui Zhu, Marc A. Kastner, Shin'ichi Satoh. MultiMedia Modelling (MMM) 2022, Apr 2022. doi: 10.1007/978-3-030-98358-1_24
Towards detecting birds from panorama video aided by Sound Source Localization
Baidong Chu, Chihaya Matsuhira, Yasutomo Kawanishi, Marc A. Kastner, Takahiro Komamizu, Ichiro Ide, Daisuke Deguchi. Talk at Meeting of the Technical Committee on Media Experience and Virtual Environment, MVE (メディアエクスペリエンス・バーチャル環境基礎研究会), Mar 2022.
2021
Pose-aware Outfit Transfer between Unpaired in-the-wild Fashion Images
Donnaphat Trakulwaranont, Marc A. Kastner, Shin'ichi Satoh. Short paper at ACM Multimedia Asia (MMAsia) 2021, Dec 2021. doi: 10.1145/3469877.3490569
Imageability- and length-controllable image captioning
Marc A. Kastner, Kazuki Umemura, Ichiro Ide, Yasutomo Kawanishi, Takatsugu Hirayama, Keisuke Doman, Daisuke Deguchi, Hiroshi Murase, Shin'ichi Satoh. Journal paper at IEEE Access, vol. 9, pp. 162951-162961, Nov 2021. doi: 10.1109/ACCESS.2021.3131393
Reproducibility Companion Paper: Describing Subjective Experiment Consistency by p-Value P-P Plot
Jakub Nawała, Lucjan Janowski, Bogdan Ćmiel, Krzysztof Rusek, Marc A. Kastner, Jan Zahálka. Reproducibility Companion Paper at ACM Multimedia (ACMMM) 2021, Oct 2021. doi: 10.1145/3474085.3477935
Reproducibility Companion Paper: Kalman Filter-Based Head Motion Prediction for Cloud-Based Mixed Reality
Serhan Gül, Sebastian Bosse, Dimitri Podborski, Thomas Schierl, Cornelius Hellge, Marc A. Kastner, Jan Zahálka. Reproducibility Companion Paper at ACM Multimedia (ACMMM) 2021, Oct 2021. doi: 10.1145/3474085.3477936
A multi-modal dataset for analyzing the imageability of concepts across modalities
Marc A. Kastner, Chihaya Matsuhira, Ichiro Ide, Shin'ichi Satoh. 4th IEEE International Conference on Multimedia Information Processing and Retrieval (MIPR2021), Sep 2021. doi: 10.1109/MIPR51284.2021.00039
[ slides ] [ github ] [ supplemental visualizations ]
Pointedness of an image: Measuring how pointy an image is perceived
Chihaya Matsuhira, Marc A. Kastner, Ichiro Ide, Yasutomo Kawanishi, Takatsugu Hirayama, Keisuke Doman, Daisuke Deguchi, Hiroshi Murase. Poster at 23rd Int. Conf. on Human-Computer Interaction (HCII2021), Jul 2021. doi: 10.1007/978-3-030-78635-9_20
Tell as You Imagine: Sentence Imageability-aware Image Captioning
Kazuki Umemura, Marc A. Kastner, Ichiro Ide, Yasutomo Kawanishi, Takatsugu Hirayama, Keisuke Doman, Daisuke Deguchi, Hiroshi Murase. MultiMedia Modelling (MMM) 2021. Lecture Notes in Computer Science, vol 12573, Jan 2021. doi: 10.1007/978-3-030-67835-7_6
FashionGraph: understanding fashion data using scene graph generation
Shabnam Sadegharmaki, Marc A. Kastner, Shin'ichi Satoh. Poster at International Conference on Pattern Recognition (ICPR) 2020, Jan 2021. doi: 10.1109/ICPR48806.2021.9412662
2020
On modelling viewer sentiment of social media videos for attractive computing
Marc A. Kastner, Shin'ichi Satoh. Talk at Meeting of the Technical Committee on Media Experience and Virtual Environment, MVE (メディアエクスペリエンス・バーチャル環境基礎研究会), Sep 2020.
俯瞰支援のための文書彩色方法の検討 —単語から連想される色に基づく文書内容の可視化—
松平茅隼， カストナーマークアウレル， 井手一郎， 川西康友， 平山高嗣， 道満恵介， 出口大輔， 村瀬洋. Talk at Meeting of the Technical Committee on Media Experience and Virtual Environment, MVE (メディアエクスペリエンス・バーチャル環境基礎研究会), Sep 2020.
A preliminary study on viewer sentiment analysis of social media videos
Marc A. Kastner, Shin'ichi Satoh. Poster at MIRU Symposium (画像の認識・理解シンポジウム), Aug 2020.
心像性を考慮した画像キャプショニングの検討
梅村和紀，カストナーマークアウレル，井手一郎，川西康友，平山高嗣，道満恵介，出口大輔，村瀬洋. Poster at MIRU Symposium (画像の認識・理解シンポジウム), Aug 2020.
Imageability estimation using visual and language features
Chihaya Matsuhira, Marc A. Kastner, Ichiro Ide, Yasutomo Kawanishi, Takatsugu Hirayama, Keisuke Doman, Daisuke Deguchi, Hiroshi Murase. Short paper and poster at ACM International Conference on Multimedia Retrieval (ICMR) 2020, Jun 2020. doi: 10.1145/3372278.3390731
On the Quantification of the Mental Image of Visual Concepts for Multi-modal Applications
Marc A. Kastner, Ichiro Ide, Yasutomo Kawanishi, Takatsugu Hirayama, Daisuke Deguchi, Hiroshi Murase. Technical Report at CVIM (情報処理学会研究報告) Special Session on Doctoral Theses, May 2020.
[ slides ] [ techreport ] [ poster ]
視覚特徴と言語特徴を用いた単語の心像性推定の検討
松平茅隼， カストナーマークアウレル， 井手一郎， 川西康友， 平山高嗣， 道満恵介， 出口大輔， 村瀬洋. Talk at 26th Annual Meeting of the Association for Natural Language Processing (言語処理学会第26回年次大会), Mar 2020.
心像性に基づく画像キャプショニングの検討
梅村和紀， カストナーマークアウレル， 井手一郎， 川西康友， 平山高嗣， 道満恵介， 出口大輔， 村瀬洋. Talk at Meeting of the Technical Committee on Media Experience and Virtual Environment, MVE (メディアエクスペリエンス・バーチャル環境基礎研究会), Awarded with MVE Award (MVE賞受賞), Mar 2020.
Estimating the imageability of words by mining visual characteristics from crawled image data
Marc A. Kastner, Ichiro Ide, Frank Nack, Yasutomo Kawanishi, Takatsugu Hirayama, Daisuke Deguchi, Hiroshi Murase. Journal paper at Multimedia Tools and Applications (MTAP), 79(25), 18167-18199, Feb 2020. doi: 10.1007/s11042-019-08571-4
Browsing Visual Sentiment Datasets using Psycholinguistic Groundings
Marc A. Kastner, Ichiro Ide, Yasutomo Kawanishi, Takatsugu Hirayama, Daisuke Deguchi, Hiroshi Murase. Demo at MultiMedia Modelling (MMM) 2020, Jan 2020. doi: 10.1007/978-3-030-37734-2_56
2019
On quantizing the mental image of concepts for visual semantic analyses
Marc A. Kastner. Paper at ACM Multimedia (ACMMM) 2019 Doctoral Symposium, Oct 2019. doi: 10.1145/3343031.3352589
On visualizing psycholinguistic groundings for sentiment image datasets
Marc A. Kastner, Ichiro Ide, Yasutomo Kawanishi, Takatsugu Hirayama, Daisuke Deguchi, Hiroshi Murase. Demo at MIRU Symposium (画像の認識・理解シンポジウム), Jul 2019.
Estimating the visual variety of concepts by referring to Web popularity
Marc A. Kastner, Ichiro Ide, Yasutomo Kawanishi, Takatsugu Hirayama, Daisuke Deguchi, Hiroshi Murase. Journal paper at Multimedia Tools and Applications (MTAP), 78(7), 9463-9488, Apr 2019. doi: 10.1007/s11042-018-6528-x
A preliminary study on estimating word imageability labels using Web image data mining
Marc A. Kastner, Ichiro Ide, Yasutomo Kawanishi, Takatsugu Hirayama, Daisuke Deguchi, Hiroshi Murase. Talk at 25th Annual Meeting of the Association for Natural Language Processing (言語処理学会第25回年次大会), no. A4-7, pp. 747-750, Mar 2019.
画像キャプションの質的評価に向けた文の心像性推定手法の検討
梅村和紀， カストナーマークアウレル， 井手一郎， 川西康友， 平山高嗣， 道満恵介， 出口大輔， 村瀬洋. Talk at 25th Annual Meeting of the Association for Natural Language Processing (言語処理学会第25回年次大会), no. A4-9, pp. 755-758, Mar 2019.
2018
On understanding visual relationships of concepts by visualizing bag-of-visual-words models
Marc A. Kastner, Ichiro Ide, Yasutomo Kawanishi, Takatsugu Hirayama, Daisuke Deguchi, Hiroshi Murase. Poster at MIRU Symposium (画像の認識・理解シンポジウム), Aug 2018.
A Survey on Psychology - Connecting Perception, Language, and Memory studies with Computer Vision
Marc A. Kastner, Yukitaka Tsuchiya, Kosuke Osumi, Ryo Akiyama, Lisa Kawai, Hiroya Ikeda. Poster and oral presentation at MIRU Young Researchers Program (画像の認識・理解シンポジウム 若手プログラム), Aug 2018.
Dependable Non-Volatile Memory
Arthur Martens, Rouven Scholz, Phil Lindow, Niklas Lehnfeld, Marc A. Kastner, Rüdiger Kapitza. Paper at ACM International Systems and Storage Conference (SYSTOR) 2018, Jun 2018. doi: 10.1145/3211890.3211898
Web画像の分布に基づく単語概念の視覚的な多様性の推定
カストナーマークアウレル， 井手一郎， 川西康友， 平山高嗣， 出口大輔， 村瀬洋. Technical report at CVIM (情報処理学会研究報告), Mar 2018.
[ paper ] [ slides ] [ poster ]
2017
DeNoVo malloc = Validating Data in Persistent Memory
Arthur Martens, Rouven Scholz, Phil Lindow, Marc A. Kastner, Rüdiger Kapitza. Poster at Eurosys, Apr 2017.
2016
Video Streaming for Foveated High-resolution Rendering
Marc A. Kastner. Master thesis, Feb 2016.
2015
Robust binocular pupil detection for a mobile computing platform
Marc A. Kastner. Project thesis, Aug 2015.
2014
Using sparse optical flow for two-phase gas flow capturing with multiple kinect
Kai Berger, Marc A. Kastner, Yannic Schroeder, Stefan Guthe. Computer Vision and Machine Learning with RGB-D Sensors, Adv. Comp. Vision and Pat. Rec., Jul 2014. doi: 10.1007/978-3-319-08651-4_8
The Split Grid - A Hierarchical 1D-Grid-based Acceleration Data Structure for Ray Tracing
Pablo Bauszat, Marc A. Kastner, Martin Eisemann, Marcus Magnor. Poster at Eurographics, Mar 2014. doi: 10.2312/egp.20141067
2013
