Regularization of boosted decision stumps using tabu search
Authors:
- Michał Bereta
Abstract
This study proposes a novel method to improve the well-known AdaBoost algorithm by combining it with a procedure inspired by tabu search. After each iteration of AdaBoost, the attribute used by the weak learner is placed on the tabu list, which prevents it from being utilized by the subsequent weak learners. The length of the tabu list becomes a new meta-parameter of the learning process and can be tuned based on the cross-validation procedure. This study shows that the proposed approach can improve the original AdaBoost procedure, preventing it from over-fitting to training data. This study also demonstrates that the novel method can act as a regularization procedure. Finally, the paper presents results for the proposed algorithm for 20 classification problems from the UCI repository and for face verification and gender recognition problems.
- Record ID
- CUT01f6e0cce308451492440caf3973326b
- Publication categories
- ;
- Author
- Journal series
- Applied Soft Computing, ISSN 1568-4946, e-ISSN 1872-9681
- Issue year
- 2019
- Vol
- 79
- Pages
- 424-438
- Other elements of collation
- fot.; rys.; tab.; wykr.; Bibliografia (na s.) - 437-438; Bibliografia (liczba pozycji) - 31; Oznaczenie streszczenia - Abstr.; Numeracja w czasopiśmie - Vol. 79
- Keywords in English
- boosting, AdaBoost, tabu search, regularization, face verification, gender recognition, local binary patterns, Gabor filters, FERET database
- DOI
- DOI:10.1016/j.asoc.2019.04.003 Opening in a new tab
- URL
- https://www.sciencedirect.com/science/article/pii/S1568494619301863 Opening in a new tab
- Language
- eng (en) English
- Score (nominal)
- 200
- Additional fields
- Indeksowana w: Web of Science, Scopus
- Uniform Resource Identifier
- https://cris.pk.edu.pl/info/article/CUT01f6e0cce308451492440caf3973326b/
- URN
urn:pkr-prod:CUT01f6e0cce308451492440caf3973326b
* presented citation count is obtained through Internet information analysis, and it is close to the number calculated by the Publish or PerishOpening in a new tab system.