Visible to Intel only — GUID: GUID-7CD61AB3-6B64-476A-B21C-2A363FE35CFA
Visible to Intel only — GUID: GUID-7CD61AB3-6B64-476A-B21C-2A363FE35CFA
Coordinate Descent Algorithm
The Coordinate Descent algorithm follows the algorithmic framework of iterative solver with one exception: the default method (defaultDense) of Coordinate Descent algorithm is a case of the iterative solver method with the batch equal to the number of observations in the training data set.
Details
The set of intrinsic parameters is empty. Algorithmic-specific transformation T, algorithm-specific vector U, and power d of Lebesgue space[Adams2003] are defined as follows:
Define the index j to update the component of a coefficient as a remainder in the division of the number of current iteration (t) by the number of features in the training data set (p):
Alternatively, if selection parameter was set to random, generate j randomly.
If stepLengthSequence was not provided by the user, compute the learning rate: (the diagonal element of the Hessian matrix)
Update the j-th component of vector :
Note: for example, if a non-smooth term , where p is the number of features in the training data set, the objective function should compute prox operator as follows:
Convergence check is performed each p iterations:
,
For , the infinity norm () is defined as follows:
Computation
Coordinate Descent algorithm is a special case of an iterative solver. For parameters, input, and output of iterative solvers, see Iterative Solver > Computation.
Algorithm Parameters
In addition to the input of a iterative solver, Coordinate Descent algorithm accepts the following parameters:
Parameter |
Default Value |
Description |
---|---|---|
algorithmFPType |
float |
The floating-point type that the algorithm uses for intermediate computations. Can be float or double. |
method |
defaultDense |
Performance-oriented method. |
engine |
SharePtr< engines:: mt19937:: Batch>() |
Pointer to the random number generator engine that is used internally during each iteration to choose a random component of the minimum result vector to be updated. |
positive |
false |
A boolean value. When set to true, it forces the coefficients to be positive. |
selection |
cyclic |
Value that specifies the strategy of certain coordinate selection on each iteration. Except for default cyclic value, Coordinate Descent also supports:
|
skipTheFirstComponents |
false |
A boolean value. When set to true, Coordinate Descent algorithm will skip the first component from optimization. |