Visible to Intel only — GUID: GUID-F25EAE61-8EE6-4367-99E6-CB40FA1AE1C9
Visible to Intel only — GUID: GUID-F25EAE61-8EE6-4367-99E6-CB40FA1AE1C9
AdaBoost Multiclass Classifier
AdaBoost (short for “Adaptive Boosting”) is a popular boosting classification algorithm. The AdaBoost algorithm performs well on a variety of data sets except some noisy data ([Friedman98], [Zhu2005]). The library supports two methods for the algorithms:
SAMME, or Stagewise Additive Modeling using a Multi-class Exponential loss function [Zhu2005]
SAMME.R, which is a modification of SAMME method for Real-valued returned probabilities from weak learner
Details
Given n feature vectors of size p, a vector of class labels where in case of binary classification and , where C is a number of classes, describes the class t the feature vector belongs to, and is a weak learner algorithm, the problem is to build an AdaBoost classifier.
Training Stage
SAMME method
The following scheme shows the major steps of the SAMME algorithm:
Initialize weights for
For :
Train the weak learner using weights .
Choose a confidence value , where
Update , where is a normalization factor.
Output the final hypothesis:
SAMME.R method
The following scheme shows the major steps of the SAMME.R algorithm:
Initialize weights for
For :
Train the weak learner using weights .
Receive the weighed class probability estimates from weak learner:
For , set :
For , update :
where is a normalization factor, ,
Output the final hypothesis:
Prediction Stage
SAMME method
Given the AdaBoost classifier and r feature vectors , the problem is to calculate the final class H(x):
SAMME.R method
Given the AdaBoost classifier and r feature vectors , the problem is to calculate the final class H(x):
where is as defined above in Training Stage.
Batch Processing
AdaBoost classifier follows the general workflow described in Classification Usage Model.
Training
For a description of the input and output, refer to Classification Usage Model. At the training stage, an AdaBoost classifier has the following parameters:
Parameter |
Default Value |
Description |
---|---|---|
algorithmFPType |
float |
The floating-point type that the algorithm uses for intermediate computations. Can be float or double. |
method |
defaultDense |
Available methods for computation of the AdaBoost algorithm:
|
weakLearnerTraining |
Pointer to an object of the classification stump training class |
Pointer to the training algorithm of the weak learner. By default, a classification stump weak learner is used. |
weakLearnerPrediction |
Pointer to an object of the classification stump prediction class |
Pointer to the prediction algorithm of the weak learner. By default, a classification stump weak learner is used. |
accuracyThreshold |
0.01 |
AdaBoost training accuracy. |
maxIterations |
100 |
The maximal number of iterations for the algorithm. |
learningRate |
1.0 |
Multiplier for each classifier to shrink its contribution. |
nClasses |
2 |
The number of classes. |
resultsToCompute |
0 |
The 64-bit integer flag that specifies which extra characteristics of AdaBoost to compute. Current version of the library only provides the following option: computeWeakLearnersErrors |
Output
In addition to classifier output, AdaBoost calculates the result described below. Pass the Result ID as a parameter to the methods that access the result of your algorithm. For more details, see Algorithms.
Result ID |
Result |
---|---|
weakLearnersErrors |
A numeric table containing weak learner’s classification errors computed when the computeWeakLearnersErrors option is on.
NOTE:
By default, this result is an object of the HomogenNumericTable class, but you can define the result as an object of any class derived from NumericTable.
|
Prediction
For a description of the input and output, refer to Classification Usage Model. At the prediction stage, an AdaBoost classifier has the following parameters:
Parameter |
Default Value |
Description |
---|---|---|
algorithmFPType |
float |
The floating-point type that the algorithm uses for intermediate computations. Can be float or double. |
method |
defaultDense |
Performance-oriented computation method, the only method supported by the AdaBoost classifier at the prediction stage. |
weakLearnerPrediction |
Pointer to an object of the classification stump prediction class |
Pointer to the prediction algorithm of the weak learner. By default, a classification stump weak learner is used. |
nClasses |
2 |
The number of classes. |
Examples
C++ (CPU)
Batch Processing: