Visible to Intel only — GUID: GUID-0846E4AB-9044-4528-8E24-14476E09C2E3
basic_statistics_dense_batch.cpp
basic_statistics_dense_online.cpp
column_accessor_homogen.cpp
cor_dense_batch.cpp
cor_dense_online.cpp
cov_dense_batch.cpp
cov_dense_biased_batch.cpp
cov_dense_biased_online.cpp
cov_dense_online.cpp
csr_accessor.cpp
csr_table.cpp
dbscan_brute_force_batch.cpp
df_cls_hist_batch.cpp
df_cls_hist_batch_random.cpp
df_cls_traverse_model.cpp
df_reg_hist_batch.cpp
df_reg_hist_batch_random.cpp
df_reg_traverse_model.cpp
heterogen_table.cpp
homogen_table.cpp
kmeans_init_dense.cpp
kmeans_lloyd_dense_batch.cpp
knn_cls_brute_force_dense_batch.cpp
knn_reg_brute_force_dense_batch.cpp
knn_search_brute_force_dense_batch.cpp
linear_kernel_dense_batch.cpp
linear_regression_dense_batch.cpp
linear_regression_dense_online.cpp
logistic_regression_dense_batch.cpp
pca_cor_dense_batch.cpp
pca_cor_dense_online.cpp
pca_cov_dense_batch.cpp
pca_cov_dense_online.cpp
pca_precomputed_cor_dense_batch.cpp
pca_precomputed_cov_dense_batch.cpp
pca_svd_dense_batch.cpp
rbf_kernel_dense_batch.cpp
svm_two_class_thunder_dense_batch.cpp
basic_statistics_dense_batch.cpp
basic_statistics_dense_online.cpp
column_accessor_homogen.cpp
connected_components_batch.cpp
cor_dense_batch.cpp
cor_dense_online.cpp
cov_dense_batch.cpp
cov_dense_biased_batch.cpp
cov_dense_biased_online.cpp
cov_dense_online.cpp
csr_accessor.cpp
csr_table.cpp
dbscan_brute_force_batch.cpp
df_cls_dense_batch.cpp
df_reg_dense_batch.cpp
directed_graph.cpp
graph_service_functions.cpp
heterogen_table.cpp
homogen_table.cpp
jaccard_batch.cpp
jaccard_batch_app.cpp
kmeans_init_dense.cpp
kmeans_lloyd_dense_batch.cpp
knn_cls_brute_force_dense_batch.cpp
knn_cls_kd_tree_dense_batch.cpp
knn_search_brute_force_dense_batch.cpp
linear_kernel_dense_batch.cpp
linear_regression_dense_batch.cpp
linear_regression_dense_online.cpp
logloss_dense_batch.cpp
louvain_batch.cpp
pca_cor_dense_batch.cpp
pca_cor_dense_online.cpp
pca_cov_dense_batch.cpp
pca_cov_dense_online.cpp
pca_precomputed_dense_batch.cpp
pca_svd_dense_batch.cpp
pca_svd_dense_online.cpp
polynomial_kernel_dense_batch.cpp
rbf_kernel_dense_batch.cpp
shortest_paths_batch.cpp
sigmoid_kernel_dense_batch.cpp
subgraph_isomorphism_batch.cpp
svm_multi_class_thunder_csr_batch.cpp
svm_multi_class_thunder_dense_batch.cpp
svm_nu_cls_thunder_csr_batch.cpp
svm_nu_cls_thunder_dense_batch.cpp
svm_nu_reg_thunder_csr_batch.cpp
svm_nu_reg_thunder_dense_batch.cpp
svm_reg_thunder_csr_batch.cpp
svm_reg_thunder_dense_batch.cpp
svm_two_class_smo_csr_batch.cpp
svm_two_class_smo_dense_batch.cpp
svm_two_class_thunder_csr_batch.cpp
svm_two_class_thunder_dense_batch.cpp
triangle_counting_batch.cpp
K-Means Clustering
Density-Based Spatial Clustering of Applications with Noise
Correlation and Variance-Covariance Matrices
Principal Component Analysis
Principal Components Analysis Transform
Singular Value Decomposition
Association Rules
Kernel Functions
Expectation-Maximization
Cholesky Decomposition
QR Decomposition
Outlier Detection
Distance Matrix
Distributions
Engines
Moments of Low Order
Quantile
Quality Metrics
Sorting
Normalization
Optimization Solvers
Decision Forest
Decision Trees
Gradient Boosted Trees
Stump
Linear and Ridge Regressions
LASSO and Elastic Net Regressions
k-Nearest Neighbors (kNN) Classifier
Implicit Alternating Least Squares
Logistic Regression
Naïve Bayes Classifier
Support Vector Machine Classifier
Multi-class Classifier
Boosting
Visible to Intel only — GUID: GUID-0846E4AB-9044-4528-8E24-14476E09C2E3
LASSO and Elastic Net Computation
Batch Processing
LASSO and Elastic Net algorithms follow the general workflow described in Regression Usage Model.
Training
For a description of common input and output parameters, refer to Regression Usage Model. Both LASSO and Elastic Net algorithms have the following input parameters in addition to the common input parameters:
Input ID |
Input |
|---|---|
weights |
Optional input. Pointer to the By default, all weights are equal to 1. |
gramMatrix |
Optional input. Pointer to the By default, the table is set to an empty numeric table. It is used only when the number of features is less than the number of observations. |
numeric table with weights of samples. The input can be an object of any class derived from NumericTable except for PackedTriangularMatrix, PackedSymmetricMatrix, and CSRNumericTable.
numeric table with pre-computed Gram Matrix. The input can be an object of any class derived from NumericTable except for CSRNumericTable.Chosse the appropriate tab to see the parameters used in LASSO and Elastic Net batch training algorithms:
LASSO
Parameter |
Default Value |
Description |
|---|---|---|
algorithmFPType |
float |
The floating-point type that the algorithm uses for intermediate computations. Can be float or double. |
method |
defaultDense |
The computation method used by the LASSO regression. The only training method supported so far is the default dense method. |
interceptFlag |
True |
A flag that indicates whether or not to compute |
lassoParameters |
A numeric table of size |
A numeric table of size
This parameter can be an object of any class derived from NumericTable, except for PackedTriangularMatrix, PackedSymmetricMatrix, and CSRNumericTable. |
optimizationSolver |
Optimization procedure used at the training stage. |
|
optResultToCompute |
0 |
The 64-bit integer flag that specifies which extra characteristics of the LASSO regression to compute. Provide the following value to request a characteristic:
|
dataUseInComputation |
doNotUse |
A flag that indicates a permission to overwrite input data. Provide the following value to restrict or allow modification of input data:
|
that contains the default LASSO parameter equal to 0.1.
coefficients: 
(where k is the number of dependent variables) or 
for 
.
.Elastic Net
Parameter |
Default Value |
Description |
|---|---|---|
algorithmFPType |
float |
The floating-point type that the algorithm uses for intermediate computations. Can be float or double. |
method |
defaultDense |
The computation method used by the Elastic Net regression. The only training method supported so far is the default dense method. |
interceptFlag |
True |
A flag that indicates whether or not to compute |
penaltyL1 |
A umeric table of size |
L1 regularization coefficient (penaltyL1 is The numeric table of size
This parameter can be an object of any class derived from NumericTable, except for PackedTriangularMatrix, PackedSymmetricMatrix, and CSRNumericTable. |
penaltyL2 |
A numeric table of size |
L2 regularization coefficient (penaltyL2 is The numeric table of size
This parameter can be an object of any class derived from NumericTable, except for PackedTriangularMatrix, PackedSymmetricMatrix, and CSRNumericTable. |
optimizationSolver |
Optimization procedure used at the training stage. |
|
optResultToCompute |
0 |
The 64-bit integer flag that specifies which extra characteristics of the Elastic Net regression to compute. Provide the following value to request a characteristic:
|
dataUseInComputation |
doNotUse |
A flag that indicates a permission to overwrite input data. Provide the following value to restrict or allow modification of input data:
|
as described in 
for 
.
as described in 
for 
.
NOTE:
Common combinations of Elastic Net regularization parameters [Friedman2010] might be computed as shown below:
compromise between L1 (lasso penalty) and L2 (ridge-regression penalty) regularization:
control full regularization:
In addition, both LASSO and Elastic Net algorithms have the following optional results:
Result ID |
Result |
|---|---|
gramMatrix |
Pointer to the computed Gram Matrix with size |
Prediction
For a description of the input and output, refer to Regression Usage Model.
At the prediction stage, LASSO and Elastic Net algorithms have the following parameters:
Parameter |
Default Value |
Description |
|---|---|---|
algorithmFPType |
float |
The floating-point type that the algorithm uses for intermediate computations. Can be float or double. |
method |
defaultDense |
Default performance-oriented computation method, the only method supported by the regression-based prediction. |
Examples
LASSO
C++: lasso_reg_dense_batch.cpp
Elastic Net
Performance Considerations
For better performance when the number of samples is larger than the number of features in the training data set, certain coordinates of gradient and Hessian are computed via the component of Gram matrix. When the number of features is larger than the number of observations, the cost of each iteration via Gram matrix depends on the number of features. In this case, computation is performed via residual update [Friedman2010].
To get the best overall performance for LASSO and Elastic Net training, do the following:
If the number of features is less than the number of samples, use homogenous table.
If the number of features is greater than the number of samples, use SOA layout rather than AOS layout.