Modular electric units for first-and-last-mile reservation services considering uncertainty

This study explores the day-ahead management of a fleet of modular electric units (MEUs) providing reservation-based first-and-last-mile services (FLMRS). Given limited fleet resources, the operator seeks to strategically select requests to accept and deploy optimal MEU configurations, aiming to maximize service revenue. This process involves pre-determining MEU configurations and travel itineraries, including routing and charging plans, and providing passengers with timely feedback. Furthermore, the operational environment is variable, affected by uncertain congestion levels, with request information emerging over time. Consequently, the FLMRS problem is modeled as a stochastic, time-dependent, and dynamic routing problem, formulated by a semi-Markov decision process (SMDP). To address this, we develop a multi-agent deep hierarchical reinforcement learning (MADHRL) approach to solve the distributed SMDP model through a reshaped reward function. A tailored MEU assembly rule is introduced to manage complex interactions among agents and reduce the action space for heterogeneous MEUs with varying battery levels. A mean-field fleet state representation helps to mitigate the curse of dimensionality. Additionally, an adjustable rolling-horizon strategy is applied to balance the trade-off between potential request cancellation and profitable request collection, taking into account the distribution of passengers’ patience times. Extensive numerical experiments, based on real-world data from Singapore, validate the efficacy of our methodology. Results offer insights into effective capacity management, including optimal MEU combinations for request acceptance and response timing control, indicating a 3.22% increase in service profit by an MEU fleet compared to traditional vehicles without assembled operations.