Ensemble Learning Based Stock Market Prediction Enhanced with Sentiment Analysis

Besides technical and fundamental analysis, machine learning and sentiment analysis obtained from non-structural news and comments have been studied extensively in financial market prediction in recent years. It is still uncertain how to combine predictions from news, sentiment scores or financial data. In this study, we provide a methodology to achieve this issue. Besides the methodology, this study differs from previous studies in terms of data coverage and used models in both sentiment analysis and prediction. Our study consists of weekly predictions by ensemble learning and feature selection methods using 683 variables for stocks traded in the Borsa Istanbul 30 index. In addition, we predicted sentiment scores from news of 18 different sectors and combined both predictions with weighted normalized returns. We used Random Forests, Extreme Gradient Boosting and Light Gradient Boosting Machines of ensemble learning methods for predictions. From the parameters such as training set length, estimation methods, variable selection methods, number of variables, and the number of models in the prediction method, we took the combination that gives the best result. For sentiment scores, tests were performed using BERT, Word2Vec, XLNet and Flair methods. Then, we extracted final sentiment scores from the news. With the proposed trade system, we combined the results obtained from these financial variables and the news sentiment scores. Final results show that we achieved a better performance than both predictions made by using sentiment scores and financial data in terms of weekly return and accuracy.