SC-COO: A feedback-based service composition algorithm combining offline and online reinforcement learning

Faced with the current dynamic service environment, rapid and efficient service composition has attracted much attention in recent years. The service composition could complete the reuse of existing services and its ultimate goal is to better satisfy users. However, it is challenging to interact with the service environment to collect data in practical applications due to factors such as high cost and risk. To overcome this limitation, this paper proposes the SC-COO method: A feedback-based service composition algorithm combining offline and online reinforcement learning. The SC-COO method mainly consists of two stages: the offline training module (SC-COO-offline) is the main stage, and the online update module (SC-COO-online) is the auxiliary stage. The SC-COO-offline model is trained through collected offline data, avoiding the drawback of online learning requiring multiple iterations to converge. And online training (SC-COO-online) serves as an auxiliary stage to jointly make decisions and recommend services to users to better adapt to dynamic environments. Furthermore, our SC-COO method offers users’ score preferences in service composition by designing a feedback-based reward mechanism. Continuous interactive feedback with humans can significantly improve the robustness of the service composition system. Finally, some experiments on the RapidAPI dataset demonstrate that SC-COO outperforms other baselines in accuracy, scalability, and convergence. And some results of the ablation experiment also verify the efficiency and applicability of SC-COO.