Skip to main navigation Skip to search Skip to main content

Research on Image Caption Method Based on Mixed Image Features

  • Qilu University of Technology

Research output: Chapter in Book/Report/Conference proceedingConference contributionpeer-review

Abstract

With the continuous development of deep learning in the field of image caption, the effect of it has improved. However, the traditional method for feature extraction is based on the whole picture, ignoring the local features and the relationships of global and local features. Another problem is that the text description is broad and not targeted. Considering the above problems, we propose a new method based on mixed image features. The method uses an improved ResNet to extract global features. Local features are extracted using a deep RetinaNet. The global and the local features of the image are merged by an attention mechanism, which are mapped into embedding vectors. To obtain the mapping relationship between images and descriptions, a long short-term memory (LSTM) based on an attention mechanism is used as a language generation model. Image features and semantic features are combined to generate content description of images.

Original languageEnglish
Title of host publicationProceedings of 2019 IEEE 4th Advanced Information Technology, Electronic and Automation Control Conference, IAEAC 2019
EditorsBing Xu, Kefen Mou
PublisherInstitute of Electrical and Electronics Engineers Inc.
Pages1572-1576
Number of pages5
ISBN (Electronic)9781728119076
DOIs
StatePublished - Dec 2019
Externally publishedYes
Event4th IEEE Advanced Information Technology, Electronic and Automation Control Conference, IAEAC 2019 - Chengdu, China
Duration: 20 Dec 201922 Dec 2019

Publication series

NameProceedings of 2019 IEEE 4th Advanced Information Technology, Electronic and Automation Control Conference, IAEAC 2019

Conference

Conference4th IEEE Advanced Information Technology, Electronic and Automation Control Conference, IAEAC 2019
Country/TerritoryChina
CityChengdu
Period20/12/1922/12/19

Keywords

  • image caption
  • mixed feature
  • text generation attention

Fingerprint

Dive into the research topics of 'Research on Image Caption Method Based on Mixed Image Features'. Together they form a unique fingerprint.

Cite this