Machine Learning Approaches to Predict the Selectivity of Compounds against HDAC1 and HDAC6

The design of compounds selectively binding to specific isoforms of histone deacetylases (HDACs) is ongoing research to prevent adverse side effects. Two of the most studied isoforms are HDAC1 and HDAC6 which are important targets in various disease conditions. Here, various machine learning (ML) approaches were built and tested to predict the bioactivity and selectivity towards specific isoforms. The selectivities of compounds were precited using two different approaches: selectivity profiling and selectivity window approaches. The bioactivity models of HDAC1 and HDAC6 were used to determine the selectivities of compounds in the selectivity profiling approach. In the selectivity window approach, models were developed by directly training on the bioactivity differences of tested compounds against HDAC1 and HDAC6. In this study, firstly all available classification and regression ML algorithms in Python package were tested to find suitable algorithms. Five ML algorithms were selected based on their performances and algorithm differences. These models were compared to each other by using traditional evaluation metrics. Then, a consensus approach of the selected ML models was employed to compare the selectivity window and selectivity profiling approaches regarding their ability to distinguish HDAC1- and HDAC6-selective compounds from others. Here, the performances were also tested against an external set and it was seen that the selectivity window approach was performing slightly better in categorizing selective compounds than the selectivity profiling approach. The approaches presented in this study could be an important step to utilize for screening molecular libraries to discover selective inhibitors for targets.