Social media and clickstream analysis in Turkish news with apache spark

Text analysis in the processing of deriving information from social media channels has become more crucial in recent years. These studies can be applied to social media data to answer wide variety of questions about consumers, brands, products or any other campaign strategies for the content producers. In this context, understand the sentiment or specific emotions expressed during the social media communication channels, mostly on Twitter, identify to intent about the corresponding content and any other stages of the consumer interest. In this study, one of the biggest Turkish news provider, Hürriyet, and its top-rated authors’ tweets are analyzed by using advanced natural language processing techniques. Due to the limitation of supported libraries for the Turkish language, preprocessing steps for unstructured datasets are studied with self-developed classes using machine learning techniques. Combining Twitter circulation together with the company’s clickstream data, we motivate to find the unique patterns for user’s attention on specific categories on the news content. By measuring the full value of digital media attention data through Twitter and click datasets, we will provide the ideal scenario for using controlled marketing experiments at the digital media sector in Turkey.