Sketchyscene: Richly-annotated scene sketches

Changqing Zou; Qian Yu; Ruofei Du; Haoran Mo; Yi Zhe Song; Tao Xiang; Chengying Gao; Baoquan Chen; Hao Zhang

doi:10.1007/978-3-030-01267-0_26

Sketchyscene: Richly-annotated scene sketches

Changqing Zou
, Qian Yu
, Ruofei Du
, Haoran Mo
, Yi Zhe Song
, Tao Xiang
, Chengying Gao
, Baoquan Chen^*
, Hao Zhang

^*Corresponding author for this work

University of Maryland, College Park
Queen Mary University of London
Sun Yat-Sen University
Shandong University
Simon Fraser University

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

28 Scopus citations

Abstract

We contribute the first large-scale dataset of scene sketches, SketchyScene, with the goal of advancing research on sketch understanding at both the object and scene level. The dataset is created through a novel and carefully designed crowdsourcing pipeline, enabling users to efficiently generate large quantities of realistic and diverse scene sketches. SketchyScene contains more than 29,000 scene-level sketches, 7,000+ pairs of scene templates and photos, and 11,000+ object sketches. All objects in the scene sketches have ground-truth semantic and instance masks. The dataset is also highly scalable and extensible, easily allowing augmenting and/or changing scene composition. We demonstrate the potential impact of SketchyScene by training new computational models for semantic segmentation of scene sketches and showing how the new dataset enables several applications including image retrieval, sketch colorization, editing, and captioning, etc. The dataset and code can be found at https://github.com/SketchyScene/SketchyScene.

Original language	English
Title of host publication	Computer Vision – ECCV 2018 - 15th European Conference, 2018, Proceedings
Editors	Yair Weiss, Vittorio Ferrari, Cristian Sminchisescu, Martial Hebert
Publisher	Springer Verlag
Pages	438-454
Number of pages	17
ISBN (Print)	9783030012663
DOIs	https://doi.org/10.1007/978-3-030-01267-0_26
State	Published - 2018
Externally published	Yes
Event	15th European Conference on Computer Vision, ECCV 2018 - Munich, Germany Duration: 8 Sep 2018 → 14 Sep 2018

Publication series

Name	Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Volume	11219 LNCS
ISSN (Print)	0302-9743
ISSN (Electronic)	1611-3349

Conference

Conference	15th European Conference on Computer Vision, ECCV 2018
Country/Territory	Germany
City	Munich
Period	8/09/18 → 14/09/18

Keywords

Scene sketch
Sketch dataset
Sketch segmentation

Access to Document

10.1007/978-3-030-01267-0_26

Cite this

Zou, C., Yu, Q., Du, R., Mo, H., Song, Y. Z., Xiang, T., Gao, C., Chen, B., & Zhang, H. (2018). Sketchyscene: Richly-annotated scene sketches. In Y. Weiss, V. Ferrari, C. Sminchisescu, & M. Hebert (Eds.), Computer Vision – ECCV 2018 - 15th European Conference, 2018, Proceedings (pp. 438-454). (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 11219 LNCS). Springer Verlag. https://doi.org/10.1007/978-3-030-01267-0_26

Zou, Changqing ; Yu, Qian ; Du, Ruofei et al. / Sketchyscene : Richly-annotated scene sketches. Computer Vision – ECCV 2018 - 15th European Conference, 2018, Proceedings. editor / Yair Weiss ; Vittorio Ferrari ; Cristian Sminchisescu ; Martial Hebert. Springer Verlag, 2018. pp. 438-454 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)).

@inproceedings{3c5ac4bcd4cd4b95828234ea672aba21,

title = "Sketchyscene: Richly-annotated scene sketches",

abstract = "We contribute the first large-scale dataset of scene sketches, SketchyScene, with the goal of advancing research on sketch understanding at both the object and scene level. The dataset is created through a novel and carefully designed crowdsourcing pipeline, enabling users to efficiently generate large quantities of realistic and diverse scene sketches. SketchyScene contains more than 29,000 scene-level sketches, 7,000+ pairs of scene templates and photos, and 11,000+ object sketches. All objects in the scene sketches have ground-truth semantic and instance masks. The dataset is also highly scalable and extensible, easily allowing augmenting and/or changing scene composition. We demonstrate the potential impact of SketchyScene by training new computational models for semantic segmentation of scene sketches and showing how the new dataset enables several applications including image retrieval, sketch colorization, editing, and captioning, etc. The dataset and code can be found at https://github.com/SketchyScene/SketchyScene.",

keywords = "Scene sketch, Sketch dataset, Sketch segmentation",

author = "Changqing Zou and Qian Yu and Ruofei Du and Haoran Mo and Song, \{Yi Zhe\} and Tao Xiang and Chengying Gao and Baoquan Chen and Hao Zhang",

note = "Publisher Copyright: {\textcopyright} Springer Nature Switzerland AG 2018.; 15th European Conference on Computer Vision, ECCV 2018 ; Conference date: 08-09-2018 Through 14-09-2018",

year = "2018",

doi = "10.1007/978-3-030-01267-0\_26",

language = "英语",

isbn = "9783030012663",

series = "Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)",

publisher = "Springer Verlag",

pages = "438--454",

editor = "Yair Weiss and Vittorio Ferrari and Cristian Sminchisescu and Martial Hebert",

booktitle = "Computer Vision – ECCV 2018 - 15th European Conference, 2018, Proceedings",

address = "德国",

}

Zou, C, Yu, Q, Du, R, Mo, H, Song, YZ, Xiang, T, Gao, C, Chen, B & Zhang, H 2018, Sketchyscene: Richly-annotated scene sketches. in Y Weiss, V Ferrari, C Sminchisescu & M Hebert (eds), Computer Vision – ECCV 2018 - 15th European Conference, 2018, Proceedings. Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), vol. 11219 LNCS, Springer Verlag, pp. 438-454, 15th European Conference on Computer Vision, ECCV 2018, Munich, Germany, 8/09/18. https://doi.org/10.1007/978-3-030-01267-0_26

Sketchyscene: Richly-annotated scene sketches. / Zou, Changqing; Yu, Qian; Du, Ruofei et al.
Computer Vision – ECCV 2018 - 15th European Conference, 2018, Proceedings. ed. / Yair Weiss; Vittorio Ferrari; Cristian Sminchisescu; Martial Hebert. Springer Verlag, 2018. p. 438-454 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 11219 LNCS).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

TY - GEN

T1 - Sketchyscene

T2 - 15th European Conference on Computer Vision, ECCV 2018

AU - Zou, Changqing

AU - Yu, Qian

AU - Du, Ruofei

AU - Mo, Haoran

AU - Song, Yi Zhe

AU - Xiang, Tao

AU - Gao, Chengying

AU - Chen, Baoquan

AU - Zhang, Hao

N1 - Publisher Copyright: © Springer Nature Switzerland AG 2018.

PY - 2018

Y1 - 2018

N2 - We contribute the first large-scale dataset of scene sketches, SketchyScene, with the goal of advancing research on sketch understanding at both the object and scene level. The dataset is created through a novel and carefully designed crowdsourcing pipeline, enabling users to efficiently generate large quantities of realistic and diverse scene sketches. SketchyScene contains more than 29,000 scene-level sketches, 7,000+ pairs of scene templates and photos, and 11,000+ object sketches. All objects in the scene sketches have ground-truth semantic and instance masks. The dataset is also highly scalable and extensible, easily allowing augmenting and/or changing scene composition. We demonstrate the potential impact of SketchyScene by training new computational models for semantic segmentation of scene sketches and showing how the new dataset enables several applications including image retrieval, sketch colorization, editing, and captioning, etc. The dataset and code can be found at https://github.com/SketchyScene/SketchyScene.

AB - We contribute the first large-scale dataset of scene sketches, SketchyScene, with the goal of advancing research on sketch understanding at both the object and scene level. The dataset is created through a novel and carefully designed crowdsourcing pipeline, enabling users to efficiently generate large quantities of realistic and diverse scene sketches. SketchyScene contains more than 29,000 scene-level sketches, 7,000+ pairs of scene templates and photos, and 11,000+ object sketches. All objects in the scene sketches have ground-truth semantic and instance masks. The dataset is also highly scalable and extensible, easily allowing augmenting and/or changing scene composition. We demonstrate the potential impact of SketchyScene by training new computational models for semantic segmentation of scene sketches and showing how the new dataset enables several applications including image retrieval, sketch colorization, editing, and captioning, etc. The dataset and code can be found at https://github.com/SketchyScene/SketchyScene.

KW - Scene sketch

KW - Sketch dataset

KW - Sketch segmentation

UR - https://www.scopus.com/pages/publications/85055415138

U2 - 10.1007/978-3-030-01267-0_26

DO - 10.1007/978-3-030-01267-0_26

M3 - 会议稿件

AN - SCOPUS:85055415138

SN - 9783030012663

T3 - Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)

SP - 438

EP - 454

BT - Computer Vision – ECCV 2018 - 15th European Conference, 2018, Proceedings

A2 - Weiss, Yair

A2 - Ferrari, Vittorio

A2 - Sminchisescu, Cristian

A2 - Hebert, Martial

PB - Springer Verlag

Y2 - 8 September 2018 through 14 September 2018

ER -

Zou C, Yu Q, Du R, Mo H, Song YZ, Xiang T et al. Sketchyscene: Richly-annotated scene sketches. In Weiss Y, Ferrari V, Sminchisescu C, Hebert M, editors, Computer Vision – ECCV 2018 - 15th European Conference, 2018, Proceedings. Springer Verlag. 2018. p. 438-454. (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)). doi: 10.1007/978-3-030-01267-0_26

Sketchyscene: Richly-annotated scene sketches

Abstract

Publication series

Conference

Keywords

Access to Document

Other files and links

Fingerprint

Cite this