Breast Cancer Prediction using Stacking Models & Hyperparameter Tuning

Abstract

This paper explores the application of stacking models for breast cancer detection, integrating key techniques such as data balancing, hyperparameter tuning, and feature selection. We implemented five different stacking configurations. Initially, Logistic Regression (LR) was used as the meta-classifier, while the base estimators included Decision Tree (DT), Support Vector Machine (SVM), K-Nearest Neighbor (KNN), and Random Forest (RF) classifiers. In the second configuration, we reversed the roles: DT acted as the meta-classifier, with SVM, KNN, RF, and LR serving as the base estimators. In a third setup, SVM was used as the meta-classifier, with DT, LR, KNN, and RF as the base learners. Fourth, we implemented KNN as the stacking classifier, with LR, DT, SVM, and RF as the base estimators. Finally, in the fifth configuration, RF was the meta-classifier, supported by LR, DT, KNN, and SVM as base learners. The evaluation of stacking models was conducted in five phases, starting with a baseline with no adjustments, followed by applying data balancing alone, then adding hyperparameter tuning, applying Chi-square feature selection with data balancing, and finally using correlation-based feature selection with data balancing, all systematically excluding certain elements to analyze their individual impact. Among all cases, the stacking model with LR delivers the best performance, achieving an accuracy of 97.63%, precision of 97.68%, recall of 97.63%, and an F-measure of 97.63%, showcasing its exceptional reliability and balanced effectiveness. All models were evaluated using 10-fold cross-validation.