An improved support vector regression using least squares method

Due to the good performance in terms of accuracy, sparsity and flexibility, support vector regression (SVR) has become one of the most popular surrogate models and has been widely researched and applied in various fields. However, SVR only depends on a subset of the training data, because the -insensitive loss function ignores any training data that is within the threshold. Therefore, some extra information may be extracted from these training data to improve the accuracy of SVR. By using the least squares method, a new improved SVR (ISVR) is developed in this paper, which combines the characteristics of SVR and traditional regression methods. ISVR is based on a two-stage procedure. The principle of ISVR is to treat the response of SVR obtained in the first stage as feedback, and then add some highly nonlinear ingredients and extra linear ingredients accordingly in the second stage by utilizing a correction function. Particularly, three types of ISVR are constructed by selecting different correction functions. Additionally, the performance of ISVR is investigated through eight mathematical problems of varying dimensions and one structural mechanics problem. The results show that ISVR has some advantages in accuracy when compared with SVR, even though the number of training points varies.