Selection of informative regressors for models constructed using the maximum consistency criterion
Abstract
This paper addresses the timely problem of selecting the most significant independent variables (regressors) in a linear regression model, whose parameters are estimated using the maximum consistency method (MCM). Unlike classical regression analysis, where model quality is traditionally assessed by minimizing the mean squared error, the sum of squared residuals, or other loss functions, MMC focuses on maximizing the number of observation pairs for which the signs of the increments of the observed and estimated values of the dependent variable coincide. This criterion, known as the consistency of behavior criterion (CBC), is highly robust to outliers and anomalous values, does not require assumptions about the normality of the error distribution, and allows for the reproduction of structural patterns in the data, including trend directions and inflection points. However, until now, there have been no regressor selection procedures adapted to the specifics of this discrete criterion, which has limited the application of MMS in problems with a large number of potential factors. This paper proposes, for the first time, an original algorithm for selecting regressors based on the sequential inclusion of variables that maximize the contribution to the increase in the value of the consistency criterion. This criterion, known as the consistency of behavior criterion (CBC), is highly robust to outliers and anomalous values, does not require assumptions about the normality of the error distribution, and allows for the reproduction of structural patterns in the data, including the direction of trends and points of trend reversal. However, until now, there have been no regressor selection procedures adapted to the specifics of this discrete criterion, which has limited the application of MMS in problems with a large number of potential factors. This paper proposes, for the first time, an original algorithm for selecting regressors based on the sequential inclusion of variables that maximize the contribution to the increase in the value of the consistency criterion. A step-by-step procedure for direct selection is described; for each candidate, a semi-integer linear programming problem is solved, allowing the maximum value of the consistency criterion to be precisely computed for a given set of regressors. Formal formulations of the problems are provided, and the solvability conditions (existence of a solution) and the monotonicity property of the criterion with respect to the inclusion of regressors are proven. An illustrative example using synthetic data, comprising 10 observations and 10 predictors, demonstrates the effectiveness of the proposed approach: the algorithm correctly identifies all significant variables and is robust to the addition of non-informative factors. The results can be used in building regression models for complex technical and transportation systems, in tasks involving the forecasting of telecommunications network load, the analysis of passenger flows, and other applied areas characterized by noisy data and the importance of correctly reproducing the direction of changes in the dependent variable.