Visible to Intel only — GUID: GUID-F8FF2400-E0AA-41E3-9853-2D3AE6EBF13D
basic_statistics_dense_batch.cpp
basic_statistics_dense_online.cpp
column_accessor_homogen.cpp
cor_dense_batch.cpp
cor_dense_online.cpp
cov_dense_batch.cpp
cov_dense_online.cpp
csr_accessor.cpp
dbscan_brute_force_batch.cpp
df_cls_hist_batch.cpp
df_cls_hist_batch_random.cpp
df_cls_traverse_model.cpp
df_reg_hist_batch.cpp
df_reg_hist_batch_random.cpp
df_reg_traverse_model.cpp
kmeans_init_dense.cpp
kmeans_lloyd_dense_batch.cpp
knn_cls_brute_force_dense_batch.cpp
knn_reg_brute_force_dense_batch.cpp
knn_search_brute_force_dense_batch.cpp
linear_kernel_dense_batch.cpp
linear_regression_dense_batch.cpp
linear_regression_dense_online.cpp
logistic_regression_dense_batch.cpp
pca_cor_dense_batch.cpp
pca_cor_dense_online.cpp
pca_precomputed_cor_dense_batch.cpp
pca_precomputed_cov_dense_batch.cpp
rbf_kernel_dense_batch.cpp
svm_two_class_thunder_dense_batch.cpp
basic_statistics_dense_batch.cpp
basic_statistics_dense_online.cpp
column_accessor_homogen.cpp
connected_components_batch.cpp
cor_dense_batch.cpp
cor_dense_online.cpp
cov_dense_batch.cpp
cov_dense_online.cpp
csr_accessor.cpp
dbscan_brute_force_batch.cpp
df_cls_dense_batch.cpp
df_reg_dense_batch.cpp
directed_graph.cpp
graph_service_functions.cpp
jaccard_batch.cpp
jaccard_batch_app.cpp
kmeans_init_dense.cpp
kmeans_lloyd_dense_batch.cpp
knn_cls_brute_force_dense_batch.cpp
knn_cls_kd_tree_dense_batch.cpp
knn_search_brute_force_dense_batch.cpp
linear_kernel_dense_batch.cpp
linear_regression_dense_batch.cpp
linear_regression_dense_online.cpp
logloss_dense_batch.cpp
louvain_batch.cpp
pca_dense_batch.cpp
pca_dense_online.cpp
pca_precomputed_dense_batch.cpp
polynomial_kernel_dense_batch.cpp
rbf_kernel_dense_batch.cpp
shortest_paths_batch.cpp
sigmoid_kernel_dense_batch.cpp
subgraph_isomorphism_batch.cpp
svm_multi_class_thunder_csr_batch.cpp
svm_multi_class_thunder_dense_batch.cpp
svm_nu_cls_thunder_csr_batch.cpp
svm_nu_cls_thunder_dense_batch.cpp
svm_nu_reg_thunder_csr_batch.cpp
svm_nu_reg_thunder_dense_batch.cpp
svm_reg_thunder_csr_batch.cpp
svm_reg_thunder_dense_batch.cpp
svm_two_class_smo_csr_batch.cpp
svm_two_class_smo_dense_batch.cpp
svm_two_class_thunder_csr_batch.cpp
svm_two_class_thunder_dense_batch.cpp
triangle_counting_batch.cpp
K-Means Clustering
Glossary
Correlation and Variance-Covariance Matrices
Principal Component Analysis
Principal Components Analysis Transform
Singular Value Decomposition
Association Rules
Kernel Functions
Expectation-Maximization
Cholesky Decomposition
QR Decomposition
Outlier Detection
Distance Matrix
Distributions
Engines
Moments of Low Order
Quantile
Quality Metrics
Sorting
Normalization
Optimization Solvers
Decision Forest
Decision Trees
Gradient Boosted Trees
Stump
Linear and Ridge Regressions
LASSO and Elastic Net Regressions
k-Nearest Neighbors (kNN) Classifier
Implicit Alternating Least Squares
Logistic Regression
Naïve Bayes Classifier
Support Vector Machine Classifier
Multi-class Classifier
Boosting
Visible to Intel only — GUID: GUID-F8FF2400-E0AA-41E3-9853-2D3AE6EBF13D
Classification Gradient Boosted Trees
Gradient boosted trees classification is the special case of gradient boosted trees. For more details, see Gradient Boosted Trees.
Details
Given n feature vectors 
of n p-dimensional feature vectors and a vector of class labels 
, where 
and C is the number of classes, which describes the class to which the feature vector 
belongs, the problem is to build a gradient boosted trees classifier.
Training Stage
Gradient boosted trees classification follows the algorithmic framework of gradient boosted trees training. For a classification problem with K classes, K regression trees are constructed on each iteration, one for each output class. The loss function is cross-entropy (multinomial deviance):
where 
Binary classification is a special case when single regression tree is trained on each iteration. The loss function is
where 
Prediction Stage
Given the gradient boosted trees classifier model and vectors 
, the problem is to calculate labels for those vectors. To solve the problem for each given feature vector , the algorithm finds the leaf node in a tree in the ensemble, and the leaf node gives the tree response. The algorithm computes a sum of responses of all the trees for each class and chooses the label y corresponding to the class with the maximal response value (highest class probability).
Usage of Training Alternative
To build a Gradient Boosted Trees Classification model using methods of the Model Builder class of Gradient Boosted Tree Classification, complete the following steps:
Create a Gradient Boosted Tree Classification model builder using a constructor with the required number of classes and trees.
Create a decision tree and add nodes to it:
Use the createTree method with the required number of nodes in a tree and a label of the class for which the tree is created.
Use the addSplitNode and addLeafNode methods to add split and leaf nodes to the created tree. See the note below describing the decision tree structure.
After you add all nodes to the current tree, proceed to creating the next one in the same way.
Use the getModel method to get the trained Gradient Boosted Trees Classification model after all trees have been created.
NOTE:
Each tree consists of internal nodes (called non-leaf or split nodes) and external nodes (leaf nodes). Each split node denotes a feature test that is a Boolean expression, for example, f < featureValue or f = featureValue, where f is a feature and featureValue is a constant. The test type depends on the feature type: continuous, categorical, or ordinal. For more information on the test types, see Decision Tree.
The inducted decision tree is a binary tree, meaning that each non-leaf node has exactly two branches: true and false. Each split node contains featureIndex, the index of the feature used for the feature test in this node, and featureValue, the constant for the Boolean expression in the test. Each leaf node contains a classLabel, the predicted class for this leaf. For more information on decision trees, see Decision Tree.
Add nodes to the created tree in accordance with the pre-calculated structure of the tree. Check that the leaf nodes do not have children nodes and that the splits have exactly two children.
Examples
C++ (CPU)
Python*
Batch Processing
Gradient boosted trees classification follows the general workflow described in Gradient Boosted Trees and Classification Usage Model
Training
In addition to parameters of the gradient boosted trees described in Batch Processing, the gradient boosted trees classification training algorithm has the following parameters:
Parameter |
Default Value |
Description |
|---|---|---|
algorithmFPType |
float |
The floating-point type that the algorithm uses for intermediate computations. Can be float or double. |
method |
defaultDense |
The computation method used by the gradient boosted trees regression. The only training method supported so far is the default dense method. |
nClasses |
Not applicable |
The number of classes. A required parameter. |
loss |
crossEntropy |
Loss function type. |
Prediction
In addition to the parameters of a classifier, the gradient boosted trees classifier has the following parameters at the prediction stage:
Parameter |
Default Value |
Description |
|---|---|---|
algorithmFPType |
float |
The floating-point type that the algorithm uses for intermediate computations. Can be float or double. |
method |
defaultDense |
The computation method used by the gradient boosted trees regression. The only training method supported so far is the default dense method. |
nClasses |
Not applicable |
The number of classes. A required parameter. |
numIterations |
0 |
An integer parameter that indicates how many trained iterations of the model should be used in prediction. The default value 0 denotes no limit. All the trained trees should be used. |
Examples
C++ (CPU)
Batch Processing:
Python*
Batch Processing: