The scope of the steps below is to install native client drivers for IBM SpectrumScale (gpfs)
for 3rd party generic CentOS systems and other supported linux distros. It is divided into 3 parts :
- GPFS client driver install
- Network, Host Names, SSH Access setup
- Adding Client to a GPFS Cluster
GPFS client driver install
1.) On the Client - Find out the version and kernel levels :
a.) cat /etc/redhat-release
CentOS Linux release 7.9.2009 (Core)
b.) uname -a
Linux venice-s414-101420 3.10.0-1160.76.1.el7.x86_64
Please note down Kernel Info "3.10.0-1160.76.1.el7.x86_64" , since if there is any missing packages, these have to match for the Kernel Version
2.) FTP or SCP Copy the IBM Linux Installer rpms to /usr/local/src/gpfs/ or copy from another existing Linux Client
Example : [root@exampleclient ]# scp -r /usr/local/src/gpfs/ root@newclient:/usr/local/src/gpfs/
3.) Install the rpms from the folder made above :
yum -y install gpfs.base*.rpm gpfs.docs*.rpm gpfs.gpl*.rpm gpfs.gskit*.rpm gpfs.msg.en_US*.rpm gpfs.license.std*.rpm
4.) Building the GPFS Portability Layer .
a.) Precheck 1 : Before you run the build , ensure the Kernel Development package is installed.
How to find if it is already installed : Example : rpm -qa | grep -i "kernel-devel"
This kernel-devel provides kernel headers and makefiles sufficient to build modules against the kernel level.
You may download the Kernel-devel package from the matching Kernel level :
Example : https://rpmfind.net/linux/RPM/centos/updates/7.9.2009/x86_64/Packages/kernel-devel-3.10.0-1160.76.1.el7.x86_64.html
Install step : yum install kernel-devel-3.10.0-1160.76.1.el7.x86_64.rpm
b.) Precheck 2 : Some mmbuild of ss is broken for 5.1.4 and 5.1.2 or install 5.1.5.0 and 5.1.2.7 where the OS has a retbleed kernel patch . There is a temporary workaround till fixed in a future version of IBM SS : To see if you are impacted , please see :
https://www.ibm.com/support/pages/node/6612669?myns=s033&mynp=OCSTXKQY&mync=E&cm_sp=s033-_-OCSTXKQY-_-E
Workaround:
Edit /usr/lpp/mmfs/src/gpl-linux/Kbuild on the cluster node
Find the line
$(KBHOSTPROGS) := lxtrace
Above that line, add a new one with:
CFLAGS_kdump-kern.o += -mfunction-return=keep
Save the file and run mmbuildgpl again.
c.) Finally run the mmbuild step : /usr/lpp/mmfs/bin/mmbuildgpl
5.) [ OPTIONAL ] Set Environment PATH (so that you can run mm-command directly from / , without full path)
[root@xxx]# touch /etc/profile.d/gpfs.sh
[root@xxx]# vim /etc/profile.d/gpfs.sh
Add line : export PATH=/usr/lpp/mmfs/bin/:$PATH
[root@xxx]# chmod 777 /etc/profile.d/gpfs.sh
Reload Profile [root@xxx]# . /etc/profile
Verify : [root@xxx]# echo "$PATH"
Network, Host Names, SSH Access setup
6.) [OPTIONAL ] Disable Firewall on the client
[root@xxx]# systemctl stop firewalld
[root@xxx]# mmnetverify (if you want to verify the network readiness before adding node)
[root@xxx]# systemctl disable firewalld
7.) Hostname resolved:
Make sure the client linux hostname (aka client node name) is resolved by your DNS or by making
entries to the /etc/hosts on the SNO and SMGW nodes at minimum.
8.) SSH Keyless Access:
Copy Keys from SNO Nodes to the Client : Example :
[root@sno-190XXX-a ~]# scp /root/.ssh/* root@newclient:/root/.ssh
Adding Client to a GPFS Cluster
9.) From Any SNO Controller Node do as below :
mmaddnode -N newclient
mmchlicense client --accept -N newclient
mmchconfig pagepool=16G -N newclient
10.) Startup GPFS on the Client
From the Client :
[root@newclient ~]# mmstartup
[root@newclient ~]# mmgetstate (---wait for status to be active)
Your SNO volume should be mounted now and verify that by df -h or mmdf
Footnotes : The GPFS portability layer is specific to both the current kernel and the GPFS version. If either the kernel or the GPFS version changes, a new GPFS portability layer needs to be built.
0 Comments