用于多模态语义分析的嵌入共识自动编码器

Cross-modal retrieval has become a topic of popularity,since multi-data is heterogeneous and the similarities between different forms of information are worthy of attention.Traditional single-modal methods reconstruct the original information and lacks of considering the semantic similarity between different data.In this work,an Embedding Consensus Autoencoder for Cross-Modal Semantic Analysis is proposed,which maps the original data to a low-dimensional shared space to retain semantic information.Considering the similarity between the modalities,an automatic encoder is utilized to associate the feature projection to the semantic code vector.In addition,regularization and sparse constraints are applied to low-dimensional matrices to balance reconstruction errors.The high dimentional data is transformed into semantic code vector.Different models are constrained by parameters to achieve denoising.The experiments on four multi-modal data sets show that the query results are improved and effective cross-modal retrieval is achieved.Further,ECA-CMSA can also be applied to fields related to computer and network such as deep and subspace learning.The model breaks through the obstacles in traditional methods,and uses deep learning methods innovatively to convert multi modal data into abstract expression,which can get better accuracy and achieve better results in recognition.