Descripció del projecte
Deep learning, the study of neural networks with multiple layers and non-linearities, has seen upsurge in recent years thanks to new findings and state-of-the-art performances in fields like computer vision [1], machine translation [2], speech processing [3] and complex game playing [4].
While in recent years the algorithmic aspects have advanced rapidly, the research has mostly concentrated on data that can be transformed easily into matrices or sequences (such as image matrices, real-valued series or sequences of discrete tokens), and on artificial intelligence tasks of perception (vision, reading, listening).
While this is boosting technologies such as self-driving cars or automatic chatbots, it only represents a very small fraction of all the industries currently dealing with massive datasets and in need for artificial intelligence breakthroughs. For example, less efforts have been made in (a) applications of machine learning where the data is more complex, such as multi-relation, multi-attribute, spatio-temporal data, such as data of cities, telecom providers, ecommerce or financial institutions. Here, AI applications which require a reasoning beyond perception, for example recommender systems, forecasting, complex decision-making. “Solving” deep learning for such cases could broaden the impact of AI in society.
We propose to build on recent advances of deep learning to study and develop techniques beyond perception which use complex data. As a use case of complex reasoning we will work with the ability to generate financial advisory for an individual given a history of financial activities provided by the individual. We have massive datasets of financial activities that we believe are unprecedented in previous machine learning literature.
The grand goal of generating financial advisory would require a modeling of massive relational datasets containing multiple entities of different types (e.g. customers, payments, shops, accounts etc), and possible many-to-many relations between them; where the entities or relations can evolve in time, be described by multi-dimensional attributes and may be spatially-bounded. We will start by investigating the non-trivial aspect on how to build deep learning models for:
1. Forecasting expenses and events: Given a history of individual expenses, their relations, and the expenses of other users, accurately forecast the times and impact of expenses or financial events which can be anticipated. Here, we would start building on deep models such as long short-term memory networks, which are obtaining state-of-the-art results for sequential/temporal tasks (e.g. [2]). But we will then extend to construct forecasting models which take into account the “network” of users or taking advantage of multiple correlated signals simultaneously.
2. Uncertainty modeling: Deep learning systems for regression or forecasting provide point estimates but not uncertainty intervals or scores; thus would have no mechanism to assess the uncertainty of a prediction. We will study this problem, propose mechanisms to yield uncertainty scores, and compare to a recent baseline [5]. We will also try to study the interplay between these models and other machine learning models that deal with uncertainty in a principled way, such as Gaussian processes.
3. Data synthesis: While there have been advances in data synthesis using deep learning, with approaches like PixelCNN, WaveNet or Generative Adversarial Networks (see e.g. [6]), this is addressing signals such as images or speech which (despite challenging) are relatively “uniform” (a matrix or time series). Generating data that is relational, contextual to human behavior and external factors, and with long-term dependencies would be more challenging. An application of data generation could be synthesizing personalized data for what-if analysis, to aid in complex decision-taking or for assessing the long-term financial health and providing recommendations.
We also note that the conclusions extracted from this research could be extended to other areas with the same degree of data complexity, such as transport networks or e-commerce.
[1] Krizhevsky et al., ImageNet Classification with Deep Convolutional Neural Networks, NIPS 2012
[2] Sutskever et al., Sequence to Sequence Learning with Neural Networks, NIPS 2014
[3] Hinton et al., Deep neural networks for acoustic modeling in speech recognition. IEEE Signal Processing Magazine, 2012
[4] Silver et al., Mastering the game of Go with deep neural networks and tree search, Nature, 2016
[5] Gal and Gharhamani, Dropout as a Bayesian Approximation: Insights and applications, ICML 2015
[6] Van de Oord et al., Conditional Image Generation with PixelCNN Decoders, NIPS 2016
