Pre-Installation Requirements for the Intel® Ethernet Fabric Suite Software

Documentation

Install & Setup

000059166

09/10/2021

For software applications,  Intel® Ethernet Fabric Suite (Intel® EFS) maintains consistency and compatibility with existing standard APIs through the open source OpenFabrics Alliance* (OFA*) software stack on Linux* distribution releases.

You install the software using one of the following methods:

  • Guided installation using Text User Interface (TUI) menus (recommended)
  • Command Line Interface (CLI) commands
  • Linux* Distribution Software

Installation overview

The standard software installation process takes you through installing the Intel® Ethernet Fabric (Intel® EFS -Basic) or Intel® Ethernet Fabric Suite (Intel® EFS -FS) software package, configuring the system, and verifying the system settings.

Intel recommends that you install the Intel® EFS software on the Management Node using the Install TUI, and then use FastFabric to configure the Management Node.

After the Management Node has been configured, the Basic software can be installed on all the remaining hosts using either the FastFabric TUI or a provisioning or diskless boot mechanism.

Installation packages

The IntelEth-Basic.DISTRO.VERSION.tgz installation package (where DISTRO.VERSION is the OS distribution and software version) installs the Intel® Ethernet Host Software components needed to set up compute, I/O, and Service nodes with drivers, stacks, and basic tools for local configuration and monitoring.

The installation package includes the following components:

  • Intel® EFS Basic Tools
  • PSM3
  • Eth RV Module
  • Open MPI (gcc)
  • MPI Source
  • Debug Info
  • Fast Fabric (Only in Fabric Suite package; it includes Fast Fabric tool sets and MPI applications and benchmarks.)

The IntelEth-FS.DISTRO.VERSION.tgz installation package (where DISTRO.VERSION is the OS distribution and software version) provides the Intel® Ethernet Host Software package along with the Intel® Ethernet Fabric Suite FastFabric toolset.

The installation package includes the following components:

Pre-installation requirements

1. Fabric design prerequisites

Ensure that the following requirements are met prior to installing the software.

It is important that the design and installation of the hardware be planned carefully prior to the installation and setup of the fabric. The design plan must include the following information:

  • Identification of servers that will function as the administration or Management Nodes, where the Intel® Ethernet Fabric Suite (Intel® EFS) will be installed.
    • Server memory requirements are based on the software being used.
    • Swap disk space allowance should follow recommendations for the given version of Linux*.
    • Intel recommends, but does not require the following:
  • Using Intel® Xeon® Processor dual-socket server.
  • 32 GB or more of ECC memory (ideally fully populating DIMM channels on processor to insure maximum memory bandwidth)
  • 256 GB or more of storage (ideally RAIDed for resiliency)
  • Plan the cabling of the fabric and create a cable planning spreadsheet using the sample .xlsx files installed into /usr/share/eth-tools/samples/ on the management node. After a plan is established, ethxlattopology may be used to convert the spreadsheet into a topology XML file that can be used by FastFabric during fabric verification.
  • Plan the naming conventions for hosts and switches in the fabric. Intel recommends all switches and hosts be given unique names. Having unique names simplifies operations that are performed using host and switch names.

2. Fabric setup prerequisites

Ensure that the following requirements are met prior to installing and setting up the fabric.

  1. Ensure all hardware is installed:
    • Servers
    • Core and edge switches
    • Fabric cables
  2. Ensure an Network Interface Card (NIC) is installed in each server.
  3. The hardware configuration should be reviewed to ensure everything has been installed according to the plan.
    Following the software installation, Intel® Ethernet Fabric Suite FastFabric tools may be used to help verify the installation.
  4. (Linux*) Ensure the required operating system (OS) version (with the same kernel version) is installed on all hosts with the following options:
    • Root user command prompt ends in # or $ with a space after it.
    • Fancy and colored prompts are disabled.
    • TCL and Expect packages are installed on all Fabric Management Nodes.
      The Management Nodes that run FastFabric should have a full installation and must include the TCL and Expect OS RPMs.
      For MPI clusters, install the C and Fortran compilers, along with their associated tools, on each Management Node.
  5. (Linux) Enable remote login as root on each host.
    In order for FastFabric to manage the hosts, the Management Node must be able to securely log in as root to each host. This can be accomplished using SSH.

  6. Resolve the TCP/IP Host Names.

    FastFabric and TCP/IP must resolve host names to the management network.

    Name resolution is accomplished by configuring a DNS server on the management network, with both management network and Ethernet addresses for each host.

    Alternatively, an /etc/hosts file needs to be created on the Management Node; FastFabric can then propagate this /etc/hosts file to all the other hosts.

    If you are using the /etc/hosts file approach and not using Domain Name System (DNS):

    • On the master node, add all the Ethernet* addresses into the /etc/hosts file.
    • The localhost line should not be edited.
    • The /etc/hosts file should not have any node-specific data.
    • Copy the file to every node, as follows:
      ethscpall -p -f hostfile /etc/hosts /etc/hosts
    If you are using Domain Name System (DNS):
    • Refer to the documentation for the domain name system (DNS) server being used. Make sure to edit the /etc/resolv.conf configuration file on the Management Node to use the proper DNS server.
    • Refer to the Linux* OS documentation for more information about configuring the /etc/resolv.conf file. This file is typically configured during OS installation.
    • If /etc/resolv.conf must be manually configured for each host, FastFabric can aid in copying the file to all the hosts.
    • The /etc/resolv.conf file created on the Management Node must not have any node-specific data and must be appropriate for use on all hosts.
    • Copying the /etc/resolv.conf file to all the nodes is accomplished during the OS installation.
    • If the /etc/resolv.conf file was not set up on all the hosts during the OS installation, the FastFabric Copy a file to all hosts operation can be used during the Install Host Software on Remaining Servers procedures to copy the /etc/resolv.conf file from the Management Node to all the other nodes.
  7. Set up a Network Time Protocol (NTP) server. Configure an NTP server for the cluster, and set all Linux* hosts and internally managed chassis to sync to the NTP server.

3. OS RPMs installation prerequisites

If you are using the INSTALL script for installation, OS RPMs must be installed before you can install the Intel® Ethernet Fabric Suite software.

4. MPI Compiler prerequisites

The Open MPI that is downloaded with the Intel® EFS -Basic Software package is selectable in the Intel Intel® EFS Install Menu. This MPI was built with GCC compiler included with the relative OS distribution. To use this MPI for compiling and running your applications and benchmarks, you need to install the appropriate development libraries.

5. Performance tuning prerequisites

Intel recommends that you preconfigure servers and settings to tune fabric performance to meet the needs of your system. These tasks can be performed before or after the installation. If you are interested in benchmarking the performance of your system, these tips may help you obtain better performance.

Related topics
Compatible Hardware and Software for the Intel® Ethernet Fabric Suite Software
Installation Guide for the Intel® Ethernet Fabric Suite Software