Direct Inter-Intra View Association for Light Field Super-Resolution

Da Yang; Hao Sheng; Shuai Wang; Rongshan Chen; Zhang Xiong

doi:10.1007/978-981-99-8073-4_21

Direct Inter-Intra View Association for Light Field Super-Resolution

Da Yang
, Hao Sheng^*
, Shuai Wang
, Rongshan Chen
, Zhang Xiong

^*Corresponding author for this work

School of Computer Science and Engineering

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

2 Scopus citations

Abstract

Light field (LF) cameras record both intensity and directions of light rays in a scene with a single exposure. However, due to the inevitable trade-off between spatial and angular dimensions, the spatial resolution of LF images is limited which makes LF super-resolution (LFSR) a research hotspot. The key of LFSR is the complementation across views and the extraction of high-frequency information inside each view. Due to the high-dimensinality of LF data, previous methods usually model these two processes separately, which results in insufficient inter-view information fusion. In this paper, LF Transformer is proposed for comprehensive perception of 4D LF data. Necessary inter-intra view correlations can be directly established inside each LF Transformer block. Therefore it can handle complex disparity variations of LF. Then based on LF Transformers, 4DTNet is designed which comprehensively performs inter-intra view high-frequency information extraction. Extensive experiments on public datasets demonstrate that 4DTNet outperforms the current state-of-the-art methods both numerically and visually.

Original language	English
Title of host publication	Neural Information Processing - 30th International Conference, ICONIP 2023, Proceedings
Editors	Biao Luo, Long Cheng, Zheng-Guang Wu, Hongyi Li, Chaojie Li
Publisher	Springer Science and Business Media Deutschland GmbH
Pages	265-278
Number of pages	14
ISBN (Print)	9789819980727
DOIs	https://doi.org/10.1007/978-981-99-8073-4_21
State	Published - 2024
Event	30th International Conference on Neural Information Processing, ICONIP 2023 - Changsha, China Duration: 20 Nov 2023 → 23 Nov 2023

Publication series

Name	Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Volume	14451 LNCS
ISSN (Print)	0302-9743
ISSN (Electronic)	1611-3349

Conference

Conference	30th International Conference on Neural Information Processing, ICONIP 2023
Country/Territory	China
City	Changsha
Period	20/11/23 → 23/11/23

Keywords

Light field
Super-resolution
Transformer

Access to Document

10.1007/978-981-99-8073-4_21

Cite this

Yang, D., Sheng, H., Wang, S., Chen, R., & Xiong, Z. (2024). Direct Inter-Intra View Association for Light Field Super-Resolution. In B. Luo, L. Cheng, Z.-G. Wu, H. Li, & C. Li (Eds.), Neural Information Processing - 30th International Conference, ICONIP 2023, Proceedings (pp. 265-278). (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 14451 LNCS). Springer Science and Business Media Deutschland GmbH. https://doi.org/10.1007/978-981-99-8073-4_21

Yang, Da ; Sheng, Hao ; Wang, Shuai et al. / Direct Inter-Intra View Association for Light Field Super-Resolution. Neural Information Processing - 30th International Conference, ICONIP 2023, Proceedings. editor / Biao Luo ; Long Cheng ; Zheng-Guang Wu ; Hongyi Li ; Chaojie Li. Springer Science and Business Media Deutschland GmbH, 2024. pp. 265-278 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)).

@inproceedings{5582633cecd144de89041c8ea4ec36e3,

title = "Direct Inter-Intra View Association for Light Field Super-Resolution",

abstract = "Light field (LF) cameras record both intensity and directions of light rays in a scene with a single exposure. However, due to the inevitable trade-off between spatial and angular dimensions, the spatial resolution of LF images is limited which makes LF super-resolution (LFSR) a research hotspot. The key of LFSR is the complementation across views and the extraction of high-frequency information inside each view. Due to the high-dimensinality of LF data, previous methods usually model these two processes separately, which results in insufficient inter-view information fusion. In this paper, LF Transformer is proposed for comprehensive perception of 4D LF data. Necessary inter-intra view correlations can be directly established inside each LF Transformer block. Therefore it can handle complex disparity variations of LF. Then based on LF Transformers, 4DTNet is designed which comprehensively performs inter-intra view high-frequency information extraction. Extensive experiments on public datasets demonstrate that 4DTNet outperforms the current state-of-the-art methods both numerically and visually.",

keywords = "Light field, Super-resolution, Transformer",

author = "Da Yang and Hao Sheng and Shuai Wang and Rongshan Chen and Zhang Xiong",

note = "Publisher Copyright: {\textcopyright} 2024, The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd.; 30th International Conference on Neural Information Processing, ICONIP 2023 ; Conference date: 20-11-2023 Through 23-11-2023",

year = "2024",

doi = "10.1007/978-981-99-8073-4\_21",

language = "英语",

isbn = "9789819980727",

series = "Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)",

publisher = "Springer Science and Business Media Deutschland GmbH",

pages = "265--278",

editor = "Biao Luo and Long Cheng and Zheng-Guang Wu and Hongyi Li and Chaojie Li",

booktitle = "Neural Information Processing - 30th International Conference, ICONIP 2023, Proceedings",

address = "德国",

}

Yang, D, Sheng, H, Wang, S, Chen, R & Xiong, Z 2024, Direct Inter-Intra View Association for Light Field Super-Resolution. in B Luo, L Cheng, Z-G Wu, H Li & C Li (eds), Neural Information Processing - 30th International Conference, ICONIP 2023, Proceedings. Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), vol. 14451 LNCS, Springer Science and Business Media Deutschland GmbH, pp. 265-278, 30th International Conference on Neural Information Processing, ICONIP 2023, Changsha, China, 20/11/23. https://doi.org/10.1007/978-981-99-8073-4_21

Direct Inter-Intra View Association for Light Field Super-Resolution. / Yang, Da; Sheng, Hao; Wang, Shuai et al.
Neural Information Processing - 30th International Conference, ICONIP 2023, Proceedings. ed. / Biao Luo; Long Cheng; Zheng-Guang Wu; Hongyi Li; Chaojie Li. Springer Science and Business Media Deutschland GmbH, 2024. p. 265-278 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 14451 LNCS).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

TY - GEN

T1 - Direct Inter-Intra View Association for Light Field Super-Resolution

AU - Yang, Da

AU - Sheng, Hao

AU - Wang, Shuai

AU - Chen, Rongshan

AU - Xiong, Zhang

PY - 2024

Y1 - 2024

N2 - Light field (LF) cameras record both intensity and directions of light rays in a scene with a single exposure. However, due to the inevitable trade-off between spatial and angular dimensions, the spatial resolution of LF images is limited which makes LF super-resolution (LFSR) a research hotspot. The key of LFSR is the complementation across views and the extraction of high-frequency information inside each view. Due to the high-dimensinality of LF data, previous methods usually model these two processes separately, which results in insufficient inter-view information fusion. In this paper, LF Transformer is proposed for comprehensive perception of 4D LF data. Necessary inter-intra view correlations can be directly established inside each LF Transformer block. Therefore it can handle complex disparity variations of LF. Then based on LF Transformers, 4DTNet is designed which comprehensively performs inter-intra view high-frequency information extraction. Extensive experiments on public datasets demonstrate that 4DTNet outperforms the current state-of-the-art methods both numerically and visually.

AB - Light field (LF) cameras record both intensity and directions of light rays in a scene with a single exposure. However, due to the inevitable trade-off between spatial and angular dimensions, the spatial resolution of LF images is limited which makes LF super-resolution (LFSR) a research hotspot. The key of LFSR is the complementation across views and the extraction of high-frequency information inside each view. Due to the high-dimensinality of LF data, previous methods usually model these two processes separately, which results in insufficient inter-view information fusion. In this paper, LF Transformer is proposed for comprehensive perception of 4D LF data. Necessary inter-intra view correlations can be directly established inside each LF Transformer block. Therefore it can handle complex disparity variations of LF. Then based on LF Transformers, 4DTNet is designed which comprehensively performs inter-intra view high-frequency information extraction. Extensive experiments on public datasets demonstrate that 4DTNet outperforms the current state-of-the-art methods both numerically and visually.

KW - Light field

KW - Super-resolution

KW - Transformer

UR - https://www.scopus.com/pages/publications/85178566978

U2 - 10.1007/978-981-99-8073-4_21

DO - 10.1007/978-981-99-8073-4_21

M3 - 会议稿件

AN - SCOPUS:85178566978

SN - 9789819980727

T3 - Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)

SP - 265

EP - 278

BT - Neural Information Processing - 30th International Conference, ICONIP 2023, Proceedings

A2 - Luo, Biao

A2 - Cheng, Long

A2 - Wu, Zheng-Guang

A2 - Li, Hongyi

A2 - Li, Chaojie

PB - Springer Science and Business Media Deutschland GmbH

T2 - 30th International Conference on Neural Information Processing, ICONIP 2023

Y2 - 20 November 2023 through 23 November 2023

ER -

Yang D, Sheng H, Wang S, Chen R, Xiong Z. Direct Inter-Intra View Association for Light Field Super-Resolution. In Luo B, Cheng L, Wu ZG, Li H, Li C, editors, Neural Information Processing - 30th International Conference, ICONIP 2023, Proceedings. Springer Science and Business Media Deutschland GmbH. 2024. p. 265-278. (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)). doi: 10.1007/978-981-99-8073-4_21