Efficiency Analysis and Classification of an Airline’s Email Campaigns Using DEA and Decision Trees

Campaigns significantly impact overall company performance, making the measurement and prediction of campaign efficiency essential. This study proposes an integrated methodology that combines efficiency measurement with efficiency prediction for airline email campaigns. In the first part of the methodology, Data Envelopment Analysis (DEA) was applied to real airline campaign data to evaluate efficiency; this is the first study to analyze email campaign efficiency in this context. In the second part of the methodology, decision tree algorithms were employed to classify historical campaign data based on DEA scores, with the aim of predicting the efficiency of future campaigns—a novel approach in this context. A core dataset of 76 airline email campaigns with six inputs and two outputs was analyzed using output-oriented CCR (Charnes, Cooper, Rhodes) and BCC (Banker, Charnes, Cooper) models; 26 and 46 campaigns were identified as efficient, respectively. The analysis was further segmented by group size, seasonality, and route type. Efficient campaigns were then ranked via super-efficiency, and sensitivity analysis assessed variable and campaign effects. For prediction, decision tree algorithms (J48 (C4.5), C5.0, and CART (Classification and Regression Trees)) were employed to classify campaigns as efficient or inefficient, using DEA efficiency scores as the target variable and DEA inputs as attributes, with classification performed for both BCC and CCR core models. Class imbalance was addressed with SMOTE, and models were evaluated under stratified 10-fold cross-validation. After balancing, the BCC core model (BCC_C) yielded the most reliable predictions (overall accuracy 76.3%), with J48 providing the most balanced results, whereas the CCR core model (CCR_C) remained weak across algorithms.