Introduction
Intel® Optane™ persistent memory (PMem) is a disruptive technology that creates a new tier between memory and storage. Intel Optane™ PMem modules support two modes: Memory Mode for volatile use cases and App Direct Mode that provides byte-addressable persistent storage.
In this portion of the Quickstart Guide to Provisioning Pmem, we describe some methods of provisioning from within Microsoft Windows Server OS, by providing a tutorial for configuring and managing Intel® Optane™ persistent memory (PMem) modules in Memory Mode, App Direct Mode or mixed-mode, using Windows Server PowerShell commands.
This quick start guide (QSG) has three sections:
- Quick Start Guide Part 1: Persistent Memory Provisioning Introduction
- Quick Start Guide Part 2: Provisioning for Intel® Optane™ Persistent Memory on Linux
- Quick Start Guide Part 3: Provisioning for Intel® Optane™ Persistent Memory on Microsoft Windows
Windows Server Support for Intel® Optane™ PMem
This section provides an overview of the software tools for provisioning PMem on Microsoft Server 2019.
Device Manager View
Device Manager provides a list of PMem devices available in the system.
Software: PowerShell cmdlets
Powershell has the following cmdlets for managing PMem.
- Get-PmemDisk
- Get-PmemPhysicalDevice
- New-PmemDisk
- Remove-PmemDisk
- Get-PmemUnusedRegion
Getting Help
For more information on managing PMem on Windows, see Understanding and Deploy PMem
Tutorial: Provisioning Intel Optane Persistent Memory
The tutorial provides instructions for configuring and managing Intel® Optane™ persistent memory (PMem) modules in App Direct Mode using ipmctl and use the PowerShell utility to finish the configuration.
All the commands described in this section are demonstrated on a two-socket system with six terabytes (TB) of configured PMem, and 384 gigabytes (GB) of DRAM installed.
This tutorial considers that an operating mode is configured already by the Original Device Manufacturer (ODM), which is common in new systems received by customers. Typically, we see systems configured in App Direct Mode or Memory Mode.
Note Ipmctl snippets and screenshots below are for ipmctl 2.0
Prerequisites
- Windows Server 2019
- Intel Optane PMem Modules are installed
- The ipmctl utility is used to configure and manage PMem modules.
- To determine if ipmctl is already installed, type ipmctl into the PowerShell window
- If ipmctl is not installed, please download and refer to the instructions in the ipmctl README Tutorial
Step 1: PMem Discovery
This tutorial considers that an operating mode is pre-configured by the system provider. New systems are commonly pre-configured in App Direct Mode or Memory Mode based on the customer requirement.
- ipmctl show -dimm command displays the PMem modules discovered in the system and verifies that software can communicate with them. Among other information, this command outputs each DIMM ID, capacity, health state, and firmware version. The PMem module firmware and BIOS releases are available from the system provider
- ipmctl show -topology command identifies which memory slots the DDR and PMem modules are installed in
- ipmctl show -memoryresources will report information about the current configuration if it has been configured. If no configuration has been created, ipmctl will return a message similar to "One or more PMem modules have invalid PCD data. A platform reboot is recommended to restore valid PCD data, then try again." In the output below, the MemoryCapacity and AppDirectCapacity values are used to determine if the system was configured in Memory mode, App Direct Mode, or mixed mode, here are a few examples:
- If the system is in Memory Mode, you will see something like this (see Volatile row):
- If the system is in AppDirect, you will see something like this (see App Direct Row):
- If the system is in Mixed Mode (50/50), you will see something like this (see volatile and AppDirect rows):
Step 2: Determine a Configuration Goal
At this point, if your system is configured as you need it, consider the following step as just information. However, if you need to make changes to the configuration, this section explains the basics. If you need to change the platform configuration, such as Memory mode to App Direct Mode, see appendix A on resetting the system.
When setting a goal, we set the Intel Optane PMem modules to be used in one of the three standard modes: Memory Mode, App Direct, and mixed. Once the mode is set, the system needs to be rebooted for the changes to go into effect.
The first thing you need to do is decide which of the three modes you require and run the appropriate configuration command
- Memory Mode - 100% of PMem capacity across sockets can be provisioned in Memory mode, as described below. In this example, 100% of the available PMem capacity is to be provisioned in Memory mode. You can always use the -f option to overwrite any existing goal. The system must comply with recommended DRAM to PMem capacity ratios. At a minimum, a system must have at least a 1:4 ratio and no more than 1:16 per CPU socket. For example, a system with 192GB of PMem should have at least 768GB of PMem. A warning may be reported by ipmctl and the BIOS at boot time if the configuration does not meet the requirements.
- App Direct Mode - PMem can be provisioned in App Direct Mode with interleaving enabled or disabled. As described in the QSG Guide Part 1, interleaving increases the throughput of reads and writes to PMem.
- Mixed Modes - A percentage of PMem capacity can be provisioned in Memory mode, as described below. Any remaining PMem capacity can either be reserved or used to create an AppDirect region.
Step 3: Configure your Platform
At this point, we have three options, choose the sub-tutorial below to configure your system.
Memory Mode Example
In this example, 100% of the available PMem capacity is to be provisioned in Memory Mode.
- Enter the following on the command line ipmctl create -goal memorymode=100
- A reboot is required to process new memory allocation goals.
App Direct Example
In this example, we configure the system into App Direct Mode.
- Enter the following on the command line ipmctl create -goal persistentmemorytype=appdirect
- A reboot is required to process new memory allocation goals.
Mixed Mode Example
In this example, 60% of the available PMem capacity is to be provisioned in Memory Mode.
- Enter the following on the command line ipmctl create -goal memorymode=60
- A reboot is required to process new memory allocation goals.
Step 4: Provisioning of Intel Optane Persistent Memory Namespaces
This step of the tutorial is for continuing the provisioning process of creating PMem disks (equivalent to namespaces in Linux) for App Direct Mode and mixed-mode.
Now we need to create PMem disks to be used by an application(s). The disk size can be any % of the overall GiB capacity of the installed PMem modules. Please run the following commands to configure the system.
- Open Powershell as Administrator and enter the following commands
- Get-PmemUnusedRegion
- This command Is used to list the regions for PowerShell. If the output of this command is empty, no regions exist on the PMem modules. Recall that ipmctl is used to create regions.
- After creating regions using ipmctl and then rebooting, the newly created interleave sets are represented as PMem unused regions. These are the regions that are not assigned to a logical persistent memory device on the system
- New-PmemDisk
- This command allows Windows PowerShell to create a PMem Disk to be created on a region, and the capacity to be visible to the Windows operating system and used by applications. Just as an SSD can be carved into partitions, PMem disks represent the unit of storage that appears as a device that can be used for I/O
- FSDax is the mode used by default
- Creating a new persistent memory disk. This operation may take a few moments
- Initialize each newly created PMem disk by going to the Computer Management > Storage > Disk Management console to view each new disk and initialize it.
- Initialize the disk using MBR or GPT partitioning before the logical disk manager can access it.
-
- Format the disk using NTFS (right-click on disk > Format…)
- Get-pmemdisk
- Verifies that PMemDisks was created and initialized
- Get-PmemUnusedRegion
We have come to the end of the tutorial for provisioning PMEM within a Windows Server 2019 environment. See below for a link to other articles in this series.
- Quick Start Guide Part 1: Persistent Memory Provisioning Introduction
- Quick Start Guide Part 2: Provisioning for Intel® Optane™ Persistent Memory on Linux
Resources:
- Google PMem Group
- ipmctl user guide
- NDCTL User Guide
- pmem.io
- PMem Getting Started Guide
- #pmem Slack Channel – Join Here
Appendix
Appendix A: Misc commands
PS C:\Windows\system32> get-pmemdisk | Remove-PmemDisk
- This command will remove the persistent memory disk(s) from the system and results in data loss.