Abstract

Efficient and low-carbon berth allocation and quay crane scheduling are crucial for enhancing the competitiveness and sustainability of container ports. This study addresses the berth allocation and quay crane assignment and scheduling problem (BACASP) by incorporating carbon emission costs and uncertainties in vessel arrival times and quay crane processing times. A novel hierarchical reinforcement learning (HRL)-based scheduling framework is proposed, employing three cooperative agents to support real-time berth allocation and quay crane scheduling in dynamic environments. The upper-level agent determines whether to release waiting vessels, while two lower-level agents allocate berth locations and assign quay cranes. Numerical experiments demonstrate the effectiveness of the HRL framework compared to a mixed integer programming (MIP) approach with perfect information, highlighting its capability to achieve near-optimal solutions under sequentially observed information. The study also investigates the impact of uncertainty on operational and carbon emission costs, providing practical managerial insights for port operators. These findings underscore the potential of leveraging well-structured HRL frameworks to address complex and dynamic port operation problems.