Structured Reinforcement Learning for Air Traffic Flow Management

• Patrice Zombré
• Daniel Delahaye
• Innocent O. Davidson
• Rafael Lima de Carvalho
• Felix Mora-Camino

ASECNA, Dakar, Sénégal · Ecole Nationale de l’Aviation Civile-ENAC, France · Cape Peninsula University of Technology-CPUT, South Africa · Universidade Federal do Tocantins-UFT · Universidade Federal Fluminense-UFF and Durban University of Technology-DUT

Event

XX SITRAER – Joinville, SC, Brasil, October 18–20, 2023

Volume

Proceedings of XX SITRAER – Air Transportation Symposium · ISSN 2448-1912

Pages

187–197

Publisher

Universidade Federal de Santa Catarina (UFSC)

Editors

Renata Cavion, Rafael Gigena Cuenca

Abstract

This study considers mainly the pre-tactical planning level where the global balance of traffic flows is already fixed (flows structures and levels, ATC sectors and capacity) and where flights are treated on an individual basis in the space between the different departure/arrival airports. Then, for each flight, decisions such as ground delayed departure (resulting in flight rescheduling) and/or rerouting may be taken to cope with predicted traffic congestion situations. In this study, the adopted objective is the global reduction of interactions between flights considered by ATC while maintaining as much as possible the original efficiency of flight plans computed by airlines, taking into account aircraft performances and short-term weather forecasts. For that, a multidirectional flight centric complexity index is adopted to assess the degree of interaction between flights. A flight scheduling policy based on the global degree of asymmetry of the complexity indexes of the flights is designed to process the whole set of considered flights. For each flight, the effect on traffic smothness of applying a delayed departure is assessed based on the level of asymmetry of the longitudinal complexity indexes. Then, local lateral complexity indexes are computed along each flight in a synchronous way, so that the need of lateral deviations can be characterized. A global optimization problem is considered with two main objectives: reducing the overall traffic complexity and minimizing deviations from the original flight trajectories. Equivalent directional separations are introduced to quantify locally the threat of surrounding traffic in each direction and propose directions of improvement of separations when necessary. Then, for each direction a proportionality factor has to be chosen to minimize the global optimality criterion, this can be performed even for large instances of this problem by using Reinforcement Learning techniques. The proposed solution approach is qualified of ‘structured” since it avoids the Black Box image of many IA techniques by directing the search process for a better traffic solution.

Keywords: Air Traffic Flow Management; Air Traffic Control; Traffic Complexity Measures; Reinforcement Learning; Flight Scheduling