Deep Networks and Knowledge: from Rule Learning to Neural-Symbolic Argument Mining

Deep Learning has revolutionized the whole discipline of machine learning, heavily impacting fields such as Computer Vision, Natural Language Processing, and other domains concerned with the processing of raw inputs. Nonetheless, Deep Networks are still difficult to interpret, and their inference process is all but transparent. Moreover, there are still challenging tasks for Deep Networks: contexts where the success depends on structured knowledge that can not be easily provided to the networks in a standardized way. We aim to investigate the behavior of Deep Networks, assessing whether they are capable of learning complex concepts such as rules and constraints without explicit information, and then how to improve them by providing such symbolic knowledge in a general and modular way. We start by addressing two tasks: learning the rule of a game and learning to construct the solution to Constraint Satisfaction Problems. We provide the networks only with examples, without encoding any information regarding the task. We observe that the networks are capable of learning to play by the rules and to make feasible assignments in the CSPs. Then, we move to Argument Mining, a complex NLP task which consists of finding the argumentative elements in a document and identifying their relationships. We analyze Neural Attention, a mechanism widely used in NLP to improve networks' performance and interpretability, providing a taxonomy of its implementations. We exploit such a method to train an ensemble of deep residual networks and test them on four different corpora for Argument Mining, reaching or advancing the state of the art in most of the datasets we considered for this study. Finally, we realize the first implementation of neural-symbolic argument mining. We use the Logic Tensor Networks framework to introduce logic rules during the training process and establish that they give a positive contribution under multiple dimensions.