An Interactive Image Editing System Using an Uncertainty-Based Confirmation Strategy (bibtex)
by Shinagawa, Seitaro, Yoshino, Koichiro, Alavi, Seyed Hossein, Georgila, Kallirroi, Traum, David, Sakti, Sakriani and Nakamura, Satoshi
Abstract:
We propose an interactive image editing system that has a confirmation dialogue strategy using an entropy-based uncertainty calculation on its generated images with Deep Convolutional Generative Adversarial Networks (DCGAN). DCGAN is an image generative model that learns an image manifold of a given dataset and enables continuous change of an image. Our proposed image editing system combines DCGAN with a natural language interface that accepts image editing requests in natural language. Although such a system is helpful for human users, it often faces uncertain requests to generate acceptable images. A promising approach to solve this problem is introducing a dialogue process that shows multiple candidates and confirms the user’s intention. However, confirming every editing request creates redundant dialogues. To achieve more efficient dialogues, we propose an entropy-based dialogue strategy that decides when the system should confirm, and enables effective image editing through a dialogue that reduces redundant confirmations. We conducted image editing dialogue experiments using an avatar face illustration dataset for editing by natural language requests. Through quantitative and qualitative analysis, our results show that our entropy-based confirmation strategy achieved an effective dialogue by generating images desired by users.
Reference:
An Interactive Image Editing System Using an Uncertainty-Based Confirmation Strategy (Shinagawa, Seitaro, Yoshino, Koichiro, Alavi, Seyed Hossein, Georgila, Kallirroi, Traum, David, Sakti, Sakriani and Nakamura, Satoshi), In IEEE Access, volume 8, 2020.
Bibtex Entry:
@article{shinagawa_interactive_2020,
	title = {An {Interactive} {Image} {Editing} {System} {Using} an {Uncertainty}-{Based} {Confirmation} {Strategy}},
	volume = {8},
	issn = {2169-3536},
	url = {https://ieeexplore.ieee.org/document/9099288/},
	doi = {10.1109/ACCESS.2020.2997012},
	abstract = {We propose an interactive image editing system that has a confirmation dialogue strategy using an entropy-based uncertainty calculation on its generated images with Deep Convolutional Generative Adversarial Networks (DCGAN). DCGAN is an image generative model that learns an image manifold of a given dataset and enables continuous change of an image. Our proposed image editing system combines DCGAN with a natural language interface that accepts image editing requests in natural language. Although such a system is helpful for human users, it often faces uncertain requests to generate acceptable images. A promising approach to solve this problem is introducing a dialogue process that shows multiple candidates and confirms the user’s intention. However, confirming every editing request creates redundant dialogues. To achieve more efficient dialogues, we propose an entropy-based dialogue strategy that decides when the system should confirm, and enables effective image editing through a dialogue that reduces redundant confirmations. We conducted image editing dialogue experiments using an avatar face illustration dataset for editing by natural language requests. Through quantitative and qualitative analysis, our results show that our entropy-based confirmation strategy achieved an effective dialogue by generating images desired by users.},
	journal = {IEEE Access},
	author = {Shinagawa, Seitaro and Yoshino, Koichiro and Alavi, Seyed Hossein and Georgila, Kallirroi and Traum, David and Sakti, Sakriani and Nakamura, Satoshi},
	month = may,
	year = {2020},
	keywords = {Virtual Humans},
	pages = {98471--98480}
}
Powered by bibtexbrowser