Multi-class Classifier

Intel® oneAPI Data Analytics Library Developer Guide and Reference

Download PDF

ID 772611

Date 7/13/2023

Version

Public

A newer version of this document is available. Customers should click here to go to the newest version.

Visible to Intel only — GUID: GUID-3FFE682F-0CFF-4290-AB28-74CC0F552F39

View Details

Multi-class Classifier

While some classification algorithms naturally permit the use of more than two classes, some algorithms, such as Support Vector Machines (SVM), are by nature solving a two-class problem only. These two-class (or binary) classifiers can be turned into multi-class classifiers by using different strategies, such as One-Against-Rest or One-Against-One.

oneDAL implements a Multi-Class Classifier using the One-Against-One strategy.

Multi-class classifiers, such as SVM, are based on two-class classifiers, which are integral components of the models trained with the corresponding multi-class classifier algorithms.

Details

Given n feature vectors $x_{1} = (x_{11}, \dots, x_{1 p}), \dots, x_{n} = (x_{n 1}, \dots, x_{n p})$ of size p, the number of classes K, and a vector of class labels $y = (y_{1}, \dots, y_{n})$ , where $y_{i} \in {0, 1, \dots, K - 1}$ , the problem is to build a multi-class classifier using a two-class (binary) classifier, such as a two-class SVM.

Training Stage

The model is trained with the One-Against-One method that uses the binary classification described in [Hsu02] as follows: For each pair of classes $(i, j)$ , train a binary classifier, such as SVM. The total number of such binary classifiers is $\frac{K (K - 1)}{2}$ .

Prediction Stage

Given a new feature vector $x_{i}$ , the classifier determines the class to which the vector belongs.

oneDAL provides two methods for class label prediction:

Wu method. According to the algorithm 2 for computation of the class probabilities described in [Wu04]. The library returns the index of the class with the largest probability.
Vote-based method. If the binary classifier predicts the feature vector to be in i-th class, the number of votes for the class i is increased by one, otherwise the vote is given to the j-th class. If two classes have equal numbers of votes, the class with the smallest index is selected.

Usage of Training Alternative

To build a Multi-class Classifier model using methods of the Model Builder class of Multi-class Classifier, complete the following steps:

Create a Multi-class Classifier model builder using a constructor with the required number of features and classes.
Use the setTwoClassClassifierModel method for each pair of classes to add the pre-trained two-class classifiers to the model. In the parameters to the method specify the classes’ indices and the pointer to the pre-trained two-class classifier for this pair of classes. You need to do this for each pair of classes, because the One-Against-One strategy is used.
Use the getModel method to get the trained Multi-class Classifier model.
Use the getStatus method to check the status of the model building process. If DAAL_NOTHROW_EXCEPTIONS macros is defined, the status report contains the list of errors that describe the problems API encountered (in case of API runtime failure).

Examples

oneAPI C++

Batch Processing

svm_two_class_thunder_dense_batch.cpp

C++ (CPU)

Batch Processing

svm_multi_class_model_builder.cpp

Java*

NOTE:

The Java interfaces are deprecated in the oneDAL library and may no longer be supported in future releases.

Batch Processing

SVMMultiClassModelBuilder.java

Python*

https://github.com/intel/scikit-learn-intelex/tree/master/examples/daal4py/svm_multi_class_model_builder.py

Batch Processing

Multi-class classifier follows the general workflow described in Classification Usage Model.

Training

At the training stage, a multi-class classifier has the following parameters:

Training Parameters for Multi-class Classifier (Batch Processing)
Parameter	Default Value	Description
`algorithmFPType`	`float`	The floating-point type that the algorithm uses for intermediate computations. Can be `float` or `double`.
`method`	`defaultDense`	The computation method used by the multi-class classifier. The only training method supported so far is One-Against-One.
`training`	Pointer to an object of the SVM training class	Pointer to the training algorithm of the two-class classifier. By default, the SVM two-class classifier is used.
`nClasses`	Not applicable	The number of classes. A required parameter.

Prediction

At the prediction stage, a multi-class classifier has the following parameters:

Prediction Parameters for Multi-class Classifier (Batch Processing)
Parameter	Method	Default Value	Description
`algorithmFPType`	`defaultDense` or `voteBased`	`float`	The floating-point type that the algorithm uses for intermediate computations. Can be `float` or `double`.
`pmethod`	Not applicable	`defaultDense`	Available methods for multi-class classifier prediction stage: `defaultDense` - the method described in [Wu04] `voteBased` - the method based on the votes obtained from two-class classifiers.
`tmethod`	`defaultDense` or `voteBased`	training::oneAgainstOne	The computation method that was used to train the multi-class classifier model.
`prediction`	`defaultDense` or `voteBased`	Pointer to an object of the SVM prediction class	Pointer to the prediction algorithm of the two-class classifier. By default, the SVM two-class classifier is used.
`nClasses`	`defaultDense` or `voteBased`	Not applicable	The number of classes. A required parameter.
`maxIterations`	`defaultDense`	100	The maximal number of iterations for the algorithm.
`accuracyThreshold`	`defaultDense`	1.0e-12	The prediction accuracy.
`resultsToEvaluate`	`voteBased`	`computeClassLabels`	The 64-bit integer flag that specifies which extra characteristics of the decision function to compute. Provide one of the following values to request a single characteristic or use bitwise OR to request a combination of the characteristics: `computeClassLabels` for prediction `computeDecisionFunction` for decisionFunction

Output

In addition to classifier output, multiclass classifier calculates the result described below. Pass the Result ID as a parameter to the methods that access the result of your algorithm. For more details, see Algorithms.

Output for Multi-class Classifier (Batch Processing)
Result ID	Result
`decisionFunction`	A numeric table of size $n \times \frac{K (K - 1)}{2}$ containing the results of the decision function computed for all binary models when the `computeDecisionFunction` option is enabled.

NOTE:

If resultsToEvaluate does not contain computeDecisionFunction, the result of decisionFunction table is NULL.

By default, each numeric table of this result is an object of the HomogenNumericTable class, but you can define the result as an object of any class derived from NumericTable except for PackedSymmetricMatrix and PackedTriangularMatrix.

Examples

C++ (CPU)

Batch Processing:

Java*

NOTE:

The Java interfaces are deprecated in the oneDAL library and may no longer be supported in future releases.

Batch Processing:

Python*

Batch Processing:

https://github.com/intel/scikit-learn-intelex/tree/master/examples/daal4py/svm_multiclass_batch.py

Select Your Language

Using Intel.com Search

Quick Links

Recent Searches

Advanced Search

Only search in

Intel® oneAPI Data Analytics Library Developer Guide and Reference

Multi-class Classifier

Details

Usage of Training Alternative

Batch Processing