Bibliography

In high school, I wanted to be a professional soccer player. I was so passionate about that sport that I neglected school and got in trouble with my father, an engineering Professor, who forbade me from playing soccer unless I finished the year at the top of my class. This punishment turned my passion for sports into an obsession with science. I was forced to improve in a competitive environment, which introduced me to the beauty of mathematics and physics, and led me to discussions with my father about his readings, which included J. Tarjemanov’s book “The Silver Horse-shoe” on non-Euclidian geometry. This book covered the mathematical underpinning of general relativity and replaced my soccer dreams with space science and engineering.

These events led me to pursue joint Bachelor and Master degrees in engineering at “Polytechnique Montréal” and “National Higher French Institute of Aeronautics and Space” (SUPAERO) in Toulouse, France. I studied Applied Mathematics of Complex Systems at grande école SUPAERO and specialized in Engineering Physics at Polytechnique. I thus had the chance to acquire a solid grounding in advanced PDE's, multi-scale modeling, and optimization. Among others, I learned about the use of Navier-Stokes equations and conformal mapping methods in advanced aerodynamics, Hamilton-Jacobi and Gauss perturbation equations in astrodynamics, and some differential geometric concepts in general relativity.

During my last year at SUPAERO, I had the opportunity to be involved in an international research collaboration that aims to probe the internal structure of the planet Mars, and that is related to NASA’s upcoming InSight mission. My project was to simulate the temperature field in a seismometer and my tasks were to estimate the spectral power density of the temperature signal and then obtain an accurate numerical solution of the dynamical heat equation that arises in this specific problem. I’m happy to say that this work is currently being extended by another SUPAERO-NASA research team.

After graduating from SUPAERO, I worked as a research assistant at the Oncopole medical center in Toulouse in the field of medical image processing. My role was to evaluate the performance of automatic organ identification algorithms for adaptive radiotherapy by investigating discrete and continuous optimization methods such as the Levenberg-Marquardt algorithm, graph-based, and belief propagation methods. This was my first contact with machine learning and it inspired me to undertake graduate studies in this field.

My experience at the Oncopole encouraged me to start, in 2016, graduate studies in Data Science at the Applied Mathematics Department of Polytechnique Montréal under the co-supervision of Professor Dominique Orban and Professor Andrea Lodi, where I had the chance to initiate two research projects. The first one grew-out of a term project for an optimization course “Numerical Methods of Optimization and Control” that I had taken with Professor Dominique Orban, and the results were published in the University of Montreal’s Operations Research “Cahiers du GERAD” letters. In this joint work with my former classmate C. Neal, we were inspired by an approach proposed by Professor R. J. Vanderbei (Princeton) for the prediction of local warming. The main idea was to incorporate temperature correlations into the main model using a first-order Markov chain, and we also presented a forward algorithm which samples from the Laplace distribution to fill in the present missing data, before ending the paper with temperature predictions up to the year 2040 using a Neural Network approach. I am currently preparing this work for submission to the Optimization Letters journal.

I also had the chance to work with Professor Dominique Orban on a second project that piqued my curiosity related to the simulation of black holes and was inspired by Professor K. Thorne’s (Nobel Prize-winning physicist) paper on the movie “Interstellar.” The purpose was to use techniques from optimal control to find photon trajectories, instead of working with discretized equations of motion as is traditionally done by physicists. More precisely, the solution of the discretized Euler-Lagrange equations do not necessarily satisfy the least action principle, so instead, I discretized the optimization problem and used Pontryagin’s Minimum Principle theorem to obtain the geodesics describing the photon trajectories. After writing to Professor Kip Thorne (Caltech) and his colleague Professor Richard Price (MIT) about this approach, they expressed interest and asked for a copy of the draft paper.

Currently, I am pursuing a Ph.D. degree in Operations Research and Artificial Intelligence in the Department of Mechanical and Industrial Engineering (MIE) at the University of Toronto (UofT). My thesis topic is “High-dimensional continuous reinforcement learning for finance.” My goal is to use the financial application domain as a challenging real-world environment in which to advance reinforcement learning. My research involves the application of theoretical mathematical principles such as the theory of dynamical systems, abstract algebra and group theory, representation theory and topology, and also involves developing and testing machine learning algorithms using Python.

Machine learning (ML) is a statistical data analysis method that automates analytical model building. It is based on the idea that computer systems can learn from data, identify patterns and make decisions with minimal human intervention. ML systems have been applied successfully to many real-world problems that previously required manually developed algorithms. An area of ML, called reinforcement learning (RL) is the most appropriate ML approach for the portfolio management (PM) problem because of its ability to solve sequential decision-making problems, which is a key aspect of PM systems. Specifically, RL aims to create software agents that interact with their environment and learn how to optimally perform a series of decisions so as to maximize some notion of cumulative reward, and it is considered an active research area with many applications, including finance.

RL has been proven useful in domains such as game playing and robotics. However, the application of RL to PM is a challenging task due to several characteristics of the financial domain which include: 1) the relaxation of the Markov property which is not suitable for path-dependent problems, 2) the design of a reward function that is financially meaningful, 3) the treatment of transaction costs, where the RL agent has to always incur a penalty regardless of the action taken (good or bad), 4) the incorporation of financial constraints, 5) the multi-type and multi-frequency nature of financial data and the challenge of their feature combination, 6) the lack of a RL framework supporting cross-sectional analysis needed to complete the trend analysis point of view, 7) the importance of the interpretability of the investment policy, 8) dealing with the non-stationarity and the partial observability nature of financial markets, 9) the state-space visitation and the lack of exploration, 10) the non-stationary character of the policy in the time-horizon setting unlike the infinite-horizon one commonly used in game playing applications, and 11) the risk-awareness of the RL policy. These problem characteristics have not been studied sufficiently in the general framework of RL and, more specifically, in relation to RL in finance. We aim to leverage operation research and mathematical finance methodologies to tackle these issues and help to make intelligent, optimal RL decisions. My first published paper “Continuous Control with Stacked Deep Dynamic Recurrent Reinforcement Learning for Portfolio Optimization” addresses issues 1 to 4. A second paper “What is the Value of Cross-Sectional Approach to Deep Reinforcement Learning?” in preparation addresses issues 5 to 10. These two papers lay the foundation for RL in finance. We found issue 8 to be the most intriguing and it became the focus of our work. Our next goal is to dig deeper into exploiting topological characteristics such us symmetry in financial time series data.

One-dimensional time series (1D-TS) representations do not expose the co-occurrent events and the latent states of the data in a way that ML can easily recognize. For financial data, there are patterns at various scales that can be learned to improve performance. We will examine the hypothesis that the concept of symmetry augmentation is fundamentally linked to learning. Our research focus is on the augmentation of symmetry embedded in 1D-TS. Motivated by the duality between 1D-TS and networks, we will augment the symmetry by converting 1D-TS into three 2-dimensional representations: temporal correlation (GAF), transition dynamics (MTF), and recurrent events (RP). This conversion does not require a priori knowledge of the types of symmetries hidden in the 1D-TS. We will then exploit the group equivariance property of CNNs to learn the hidden symmetries in the augmented 2-dimensional data. We will show that such conversion only increases the amount of symmetry, which may lead to more efficient learning. Specifically, we will prove that a direct sum based augmentation will never decrease the amount of symmetry. We will also attempt to measure the amount of symmetry in the original 1D-TS and augmented representations using the notion of persistent homology, which should reveal symmetry increases after augmentation.

In addition to my thesis research, I am interested in an econophysics problem, which involves using quantum field theory and quantum computing to study financial networks and systemic risk. Modern financial markets are highly interdependent. These interdependencies consist of loans, holding shares, and other liabilities between institutions in the financial network. The failure of an organization can create a shock that can propagate through the network, spreading financial contagion and leading to additional failures. In order to understand and manage this systemic risk, we need a very large network model. However, handling large realistic networks with many interactions is a NP-hard combinatorial optimization problem. Last summer, I participated in the Creative Destruction Lab Quantum Bootcamp at the Rotman Business School at UofT, where I developed an algorithm for systemic risk evaluation. We used a mixed-integer linear program (MILP) to optimize the financial network structure to reduce the number of cascade failures due to shocks and hence better mitigate systemic risk. We optimized a seventy-node network with a D-Wave quantum computer. This size of problem is challenging with standard computers and quantum computing offers the possibility of scaling to much larger problems in the future. I plan to continue working in this area and want to develop a theory of quantum systemic risk.