Distributed Processing

Intel® oneAPI Data Analytics Library Developer Guide and Reference

Download PDF

ID 772611

Date 7/13/2023

Version

Public

A newer version of this document is available. Customers should click here to go to the newest version.

Visible to Intel only — GUID: GUID-3A1108A3-F98B-41ED-95E9-13F4C7F49331

View Details

Distributed Processing

This mode assumes that the data set is split into nblocks blocks across computation nodes.

Algorithm Parameters

The correlation and variance-covariance matrices algorithm in the distributed processing mode has the following parameters:

Algorithm Parameters for Correlation and Variance-Covariance Matrices Algorithm (Distributed Processing)
Parameter	Default Valude	Description
`computeStep`	Not applicable	The parameter required to initialize the algorithm. Can be: `step1Local` - the first step, performed on local nodes `step2Master` - the second step, performed on a master node
`algorithmFPType`	`float`	The floating-point type that the algorithm uses for intermediate computations. Can be `float` or `double`.
`method`	`defaultDense`	Available methods for computation of low order moments: defaultDense default performance-oriented method singlePassDense implementation of the single-pass algorithm proposed by D.H.D. West sumDense implementation of the algorithm in the cases where the basic statistics associated with the numeric table are pre-computed sums; returns an error if pre-computed sums are not defined fastCSR performance-oriented method for CSR numeric tables singlePassCSR implementation of the single-pass algorithm proposed by D.H.D. West; optimized for CSR numeric tables sumCSR implementation of the algorithm in the cases where the basic statistics associated with the numeric table are pre-computed sums; optimized for CSR numeric tables; returns an error if pre-computed sums are not defined
`outputMatrixType`	`covarianceMatrix`	The type of the output matrix. Can be: `covarianceMatrix` - variance-covariance matrix `correlationMatrix` - correlation matrix

Computation of correlation and variance-covariance matrices follows the general schema described in Algorithms:

Step 1 - on Local Nodes

In this step, the correlation and variance-covariance matrices algorithm accepts the input described below. Pass the Input ID as a parameter to the methods that provide input for your algorithm. For more details, see Algorithms.

Step 1: Algorithm Input for Correlation and Variance-Covariance Matrices Algorithm (Distributed Processing)
Input ID	Input
`data`	Pointer to the numeric table of size $n_{i} \times p$ that represents the i-th data block on the local node. While the input for `defaultDense`, `singlePassDense`, or `sumDense` method can be an object of any class derived from `NumericTable`, the input for `fastCSR`, `singlePassCSR`, or `sumCSR` method can only be an object of the `CSRNumericTable` class.

In this step, the correlation and variance-covariance matrices algorithm calculates the results described below. Pass the Result ID as a parameter to the methods that access the results of your algorithm. For more details, see Algorithms.

Step 1: Algorithm Output for Correlation and Variance-Covariance Matrices Algorithm (Distributed Processing)
Result ID	Result
`nObservations`	Pointer to the $1 i m e s 1$ numeric table that contains the number of observations processed so far on the local node. NOTE: By default, this result is an object of the `HomogenNumericTable` class, but you can define the result as an object of any class derived from `NumericTable` except `CSRNumericTable`.
`crossProduct`	Pointer to $p i m e s p$ numeric table with the cross-product matrix computed so far on the local node. NOTE: By default, this table is an object of the `HomogenNumericTable` class, but you can define the result as an object of any class derived from `NumericTable` except `PackedSymmetricMatrix`, `PackedTriangularMatrix`, and `CSRNumericTable`.
`sum`	Pointer to $1 i m e s p$ numeric table with partial sums computed so far on the local node. NOTE: By default, this table is an object of the `HomogenNumericTable` class, but you can define the result as an object of any class derived from `NumericTable` except `PackedSymmetricMatrix`, `PackedTriangularMatrix`, and `CSRNumericTable`.

Step 2 - on Master Node

Step 2: Algorithm Input for Correlation and Variance-Covariance Matrices Algorithm (Distributed Processing)
Input ID	Input
`partialResults`	A collection that contains results computed in Step 1 on local nodes (`nObservations`, `crossProduct`, and `sum`). NOTE: The collection can contain objects of any class derived from the `NumericTable` class except `PackedSymmetricMatrix` and `PackedTriangularMatrix`.

In this step, the correlation and variance-covariance matrices algorithm calculates the results described in the following table. Pass the Result ID as a parameter to the methods that access the results of your algorithm. For more details, see Algorithms.

Step 2: Algorithm Output for for Correlation and Variance-Covariance Matrices Algorithm (Distributed Processing)
Result ID	Result
`covariance`	Use when outputMatrixType``=``covarianceMatrix. Pointer to the numeric table with the $p i m e s p$ variance-covariance matrix. NOTE: By default, this result is an object of the `HomogenNumericTable` class, but you can define the result as an object of any class derived from `NumericTable` except `PackedTriangularMatrix` and `CSRNumericTable`.
`correlation`	Use when outputMatrixType``=``correlationMatrix. Pointer to the numeric table with the $p i m e s p$ correlation matrix. NOTE: By default, this result is an object of the `HomogenNumericTable` class, but you can define the result as an object of any class derived from `NumericTable` except `PackedTriangularMatrix` and `CSRNumericTable`.
`mean`	Pointer to the $1 i m e s p$ numeric table with means. NOTE: By default, this result is an object of the `HomogenNumericTable` class, but you can define the result as an object of any class derived from `NumericTable` except `PackedTriangularMatrix`, `PackedSymmetricMatrix`, and `CSRNumericTable`.

Product and Performance Information
Performance varies by use, configuration and other factors. Learn more at www.Intel.com/PerformanceIndex. Notice revision #20201201

Product and Performance Information

Performance varies by use, configuration and other factors. Learn more at www.Intel.com/PerformanceIndex.

Notice revision #20201201

Select Your Language

Using Intel.com Search

Quick Links

Recent Searches

Advanced Search

Only search in

Intel® oneAPI Data Analytics Library Developer Guide and Reference

Distributed Processing

Algorithm Parameters

Step 1 - on Local Nodes

Step 2 - on Master Node