Propose-and-Complete: Auto-regressive Semantic Group Generation for Personalized Scene Synthesis

Our research goal is to build a novel scene synthesis framework enabling the flexible generation of individualized indoor virtual environments. Current deep methods only learn the layout patterns from training scene samples, affording only partial co-occurrence possibilities while ignoring any user intent. In contrast, this paper devises a novel framework by flexibly combining and generating function-oriented semantic object groups while accommodating strong user intent. Conforming to this group-centric design paradigm, we consider different strategies for proposing group-level locations and completing semantic clusters with intra-group relationships. The entire framework hinges upon two technical innovations. First, we design a conditional normalizing flow-based ProposeNet to learn the exact distribution of semantic groups, by which we sample potentially plausible group-level locations constrained by user-desirable room functionalities. Second, we design a conditional graph variational auto-encoder, CompleteNet, to instantiate each semantic group with the user-specific complexity (e.g., graph size). With the complete groups readily available, we then recursively select the most plausible proposals and optimize the final layout subject to a collision-free, accessible room space and an arbitrary floor plan. Comprehensive experiments have confirmed that our new framework can produce personalized and versatile unseen 3D scenes from a more expansive design space than conventional domains delimited by training data.