Redefining clustered federated learning for system identification: The path of ClusterCraft

This paper addresses the System Identification (SYSID) problem within the framework of federated learning. We introduce a novel algorithm, an Incremental Clustering-based federated learning method for SYSID (IC-SYSID), designed to tackle SYSID challenges across multiple data sources without requiring structural prior knowledge of the dataset. IC-SYSID utilizes an incremental clustering method, ClusterCraft (CC), to eliminate the dependency on the prior knowledge of the dataset. CC starts with a single cluster model and assigns similar local workers to the same clusters by dynamically increasing the number of clusters. To reduce the number of clusters generated by CC, we develop ClusterMerge, where similar cluster models are merged. We also introduce enhanced ClusterCraft (eCC) to minimize the generation of similar cluster models during the training. Moreover, IC-SYSID addresses cluster model instability by integrating a regularization term into the loss function and initializing cluster models with scaled Glorot initialization. It utilizes a mini-batch deep learning approach to manage large SYSID datasets during local training. Extensive results on a synthetic benchmark and a realistic vehicle fleet dataset show that CC consistently outperforms the baseline C-SYSID, achieving improvements of up to 44% in SYSID accuracy while effectively eliminating unstable cluster models. Compared to CC, CC-CM reduces the average number of learned clusters by approximately 75%, with no statistically significant degradation in SYSID accuracy. The eCC further improves scalability by reducing both communication rounds and cluster counts by up to 53% compared to CC-CM, while maintaining competitive performance. Statistical validation confirms the robustness and efficiency of the IC-SYSID.