Check Runlevel on UNIX Servers

There are a bunch of runlevel for UNIX or Linux which is also called init level or mode of operation. Usually there are seven init or runlevels (0 to 6). Only one runlevel can be running at a single time.

Standard Runlevel:

0 : Halt
1 : Single user mode or Maintenance mode
6 : Reboot

Linux Runlevel:

0 : Halt
1 : Single user mode or Maintenance mode
2 : Multi user mode without networking
3 : Multi user mode with networking
4 : User definable
5 : Multi user mode with GUI
6 : Reboot

Solaris Runlevel:

0 : Halt (Brings server to OBP)
S : Single user mode with root filesystem as read-only
    (Sol10: svc:/milestone/single-user)
1 : Single user mode with all filesystems mounted
2 : Multi user mode without networking (Sol10: svc:/milestone/multi-user)
3 : Multi user mode with networking (Sol10: svc:/milestone/multi-user-server)
4 : User definable
5 : Shutdown or Power-off Hardware
6 : Reboot

HP-UX Runlevel

0 : Halt (Brings server to OBP)
S : Single user mode with root filesystem as read-only
1 : Single user mode with all filesystems mounted
2 : Multi user mode with most daemons running + CDE
3 : Multi user mode with NFS and Common Desktop Environment (CDE)
4 : Multi user mode with VUE
5 : User definable
6 : User definable

AIX Runlevel

0 : Reserved
1 : Reserved
2 : Multi user mode

Now that we know the different runlevels for each flavor – here are the two ways to get the runlevel:

Checking runlevel using “who” command :

[email protected]# who -r
.       run-level 3  Feb  4 16:14     3      0  S

Checking system runlevel using “runlevel” command :

[email protected]# runlevel
N 3

Add EMC Storage in Solaris under VxVM

New task at hand fellow UNIX Administrators. Now we need to discover the new (EMC) LUNs presented by the Storage team and make it available into Veritas Volume Manager (VxVM) and extend one of the filesystem under Veritas Filesystem (VxFS).

Display and save your current EMC LUN list:

[email protected]# /etc/powermt display dev=all

To discover the LUNs here are the simple steps:

[email protected]# /etc/powercf -q
[email protected]# /etc/powermt config

Verify the new LUNs and cross check it with the LUN number given by the Storage team:

[email protected]# /etc/powermt display dev=all

Prepare the newly discovered LUN for Veritas:

[email protected]# format
> label the new disk

Let VxVM discover the EMC PowerPath:

[email protected]# /usr/sbin/vxdctl enable
[email protected]# /usr/sbin/vxdisk list

Setup the disk and do what-ever you like with the new disk:

[email protected]# /etc/vx/bin/vxdisksetup -i emcpower11a

There you have it! From discovery to importing of a new LUN in Solaris under VxVM.

See you on my next note!!!

This entry was posted in Uncategorized on by .

Extend Filesystem under VxVM

We have already discussed extending a filesystem in AIX under JFS as well as the basic of softpartion where we could also extend its filesystem size. This time we have a task which is under Veritas Volume Manager (VxVM). We need to extend a partition that is being managed by VxVM.

To check if there is available space

[email protected]# vxassist -g ora_dg maxgrow fs_oracle
or
[email protected]# vxassist -g ora_dg maxsize layout=concat

To increase size

[email protected]# /etc/vx/bin/vxresize -g ora_dg fs_oracle +10g

There you have it folks! Very simple indeed.

Change EMC Powerpath Failover Policy on UNIX Systems

As UNIX system administrators – we need our Operating Systems running in optimal state every time. The same is what we want to our storage systems. In EMC Powerpath there are a couple of failover policies that we can enforce to our storage devices but we must always use the one that is best suitable. We have here a LUN that is currently under Basic Failover policy even though the storage is CLARiiON:

[email protected]#  powermt display dev=emcpower1a
Pseudo name=emcpower1a
CLARiiON ID=APM00012345678 [solaris/polaris]
Logical device ID=60060160EABCDEFGHIJKL00MNOPQRS12 [LUN 155]
state=alive; policy=BasicFailover; priority=0; queued-IOs=0
Owner: default=SP B, current=SP B
==============================================================================
---------------- Host ---------------   - Stor -   -- I/O Path -  -- Stats ---
### HW Path                 I/O Paths    Interf.   Mode    State  Q-IOs Errors
==============================================================================
2304 [email protected][email protected] c3t0d1s0  SP A3     active  alive      0      1
2304 [email protected][email protected] c3t1d1s0  SP B3     active  alive      0      0
2305 [email protected][email protected] c4t0d1s0  SP B1     active  alive      0      0
2305 [email protected][email protected] c4t1d1s0  SP A1     active  alive      0      1

To fix this issue – we need to set the failover policy back to CLARiiON:

[email protected]#  powermt set policy=co dev=emcpower1a

[email protected]#  powermt display dev=emcpower1a
Pseudo name=emcpower1a
CLARiiON ID=APM00012345678 [solaris/polaris]
Logical device ID=60060160EABCDEFGHIJKL00MNOPQRS12 [LUN 155]
state=alive; policy=CLAROpt; priority=0; queued-IOs=0
Owner: default=SP B, current=SP B
==============================================================================
---------------- Host ---------------   - Stor -   -- I/O Path -  -- Stats ---
### HW Path                 I/O Paths    Interf.   Mode    State  Q-IOs Errors
==============================================================================
2304 [email protected][email protected] c3t0d1s0  SP A3     active  alive      0      1
2304 [email protected][email protected] c3t1d1s0  SP B3     active  alive      0      0
2305 [email protected][email protected] c4t0d1s0  SP B1     active  alive      0      0
2305 [email protected][email protected] c4t1d1s0  SP A1     active  alive      0      1

Here are the other policies that you can use to set the correct one for your storage array:

powermt set policy=rr|io|lb|so [dev=power#|all]

The dev parameter is optional. If you do not include the dev parameter,
the powermt set policy command changes the policy of all PowerPath devices.

Parameters
· so = Symmetrix Optimization (default)
· co = Clariion Optimization
· li = Least I/Os (queued)
· lb = Least Blocks (queued)
· rr = Round Robin (one path after another)
· re = Request (failover only)
· nr = No Redirect (no load-balancing or failover)

Import Disk Group (DG) in VxVM

Have you ever encounter rebooting a server and your Veritas filesystems are missing? Most cases these filesystems were not imported during boot up and is not available for OS to use. Importing a disk groups in VxVM does not need any reboot, follow these simple steps and you will be able to get back your disk groups in no time.

List missing disk groups:

[email protected]# vxdisk -o alldgs list
DEVICE       TYPE            DISK         GROUP        STATUS
c0t0d0s2     auto:none       -            -            online invalid
c1t0d0s2     auto:none       -            -            online invalid
c8t0d3s2     auto:sliced     -           (datadg)      online
c8t0d9s2     auto:sliced     -           (datadg)      online
c8t0d10s2    auto:sliced     -           (datadg)      online
c8t0d11s2    auto:sliced     -           (datadg)      online
c8t0d12s2    auto:sliced     -           (datadg)      online
c8t0d13s2    auto:sliced     -           (datadg)      online
c8t0d14s2    auto:sliced     -           (datadg)      online
c8t0d15s2    auto:sliced     -           (datadg)      online
c8t0d16s2    auto:sliced     -           (datadg)      online
c8t0d17s2    auto:sliced     -           (datadg)      online

Import the missing disk groups:

[email protected]# vxdg import datadg

List disk groups:

[email protected]# vxdisk list
DEVICE       TYPE            DISK         GROUP        STATUS
c0t0d0s2     auto:none       -            -            online invalid
c1t0d0s2     auto:none       -            -            online invalid
c8t0d3s2     auto:sliced     c8t0d3       datadg       online
c8t0d9s2     auto:sliced     c8t0d9       datadg       online
c8t0d10s2    auto:sliced     c8t0d10      datadg       online
c8t0d11s2    auto:sliced     c8t0d11      datadg       online
c8t0d12s2    auto:sliced     c8t0d12      datadg       online
c8t0d13s2    auto:sliced     c8t0d13      datadg       online
c8t0d14s2    auto:sliced     c8t0d14      datadg       online
c8t0d15s2    auto:sliced     c8t0d15      datadg       online
c8t0d16s2    auto:sliced     c8t0d16      datadg       online
c8t0d17s2    auto:sliced     c8t0d17      datadg       online

How to use snap command on AIX

For any software or hardware issues with AIX, logging a case with IBM – the support team will always ask for snap files. Snap files will collect all system configuration that will help and guide the IBM support team to track whatever issue there is on the system.

From the man pages of snap(1M):

The snap command gathers system configuration information and compresses
the information into a pax file. The file may then be written to a device
such as tape or DVD, or transmitted to a remote system. The information
gathered with the snap command might be required to identify and resolve
system problems. Note: Root user authority is required to execute the
snap command. Use the snap -o /dev/cd0 command to copy the compressed
image to DVD. Use the snap -o /dev/rmt0 command to copy the image to tape.

To generate the snap file:

snap -r -> removes the previous snap files.
snap -gc -> create the snap file

The default directory for snap command output:

/tmp/ibmsupt

Sample output of the snap command:

[email protected]:/ > snap -gc
Checking space requirement for general information.....................
.......................................................................
.......................................................................
........................................ done.
.********Checking and initializing directory structure
Directory /tmp/ibmsupt/svCollect already exists... skipping
Directory /tmp/ibmsupt/client_collect already exists... skipping
Directory /tmp/ibmsupt/general already exists... skipping
Directory /tmp/ibmsupt/general/diagnostics already exists... skipping
Directory /tmp/ibmsupt/pcixscsi already exists... skipping
Directory /tmp/ibmsupt/sissas already exists... skipping
Directory /tmp/ibmsupt/testcase already exists... skipping
Directory /tmp/ibmsupt/other already exists... skipping
********Finished setting up directory /tmp/ibmsupt

Checking Space requirement for svCollect
Checking Space requirement for client_collect
Checking for enough free space in filesystem... done.

Gathering general system information...................................
.......................................................................
.......................................................................
.......................... done.
Gathering platform/scanout information.done.
Gathering svCollect data
Gathering client_collect data
Gathering pcixscsi system information..................................
........ done.
Gathering sissas system information...... done.

Creating compressed pax file...
Starting pax/compress process... Please wait... done.

-rw-------    1 0        0           1370809 Jun  6 02:59 snap.pax.Z

Fix Garbled Solaris Console

If ever you encounter a console with output such as below, it can be easily be remedied by fixing the /etc/ttydefs file.

ttymon: unable to find <console> in "/etc/ttydefs"
+----------------------------------------------------------------------------+
ü ×AÒNÉNG                                                                    ü
ü Ôèe ðroçráíó áîä äátá ótoreä oî tèió óùóteí áre liceîóeä to¬ or áre        ü
ü ðriöáte ðroðertù of¬ ÔÅÌUS áîä áre láwfõllù áöáiláâle oîlù to áõtèoriúeä   ü
ü õóeró for áððroöeä ðõrðoóeó.  Uîáõtèoriúeä ácceóó to áîù ðroçráí or äátá   ü
ü oî tèió óùóteí ió îot ðeríitteä¬
+----------------------------------------------------------------------------+
ü ×AÒNÉNG                                                                    ü
ü Ôèe ðroçráíó áîä äátá ótoreä oî tèió óùóteí áre liceîóeä to¬ or áre        ü
ü ðriöáte ðroðertù of¬ ÔÅÌUS áîä áre láwfõllù áöáiláâle oîlù to áõtèoriúeä   ü
ü õóeró for áððroöeä ðõrðoóeó.  Uîáõtèoriúeä ácceóó to áîù ðroçráí or äátá   ü
ü oî tèió óùóteí ió îot ðeríitte䬠áîä áîù õîáõtèoriúeä ácceóó âeùoîä tèió   ü
ü ðoiît íáù leáä to ðroóecõtioî áîä¯or äióciðliîárù áctioî.                  ü
ü Ôèió óùóteí íáù  âe íoîitoreä át áîù tiíe for oðerátioîál reáóoîó.         ü
ü Éf ùoõ áre îot áî áõtèoriúeä õóer¬ äo îot átteíðt to loç oî.               ü
+----------------------------------------------------------------------------+
SÅÏS: Òeótoreä ótrrèeáä rðõt×AÒNÉNG: locëä: cáîîot coîtáct ótátä
(error ´©¬ coîtiîõiîç×AÒNÉNG: óöc_tli_ëcreáte:
xðrt_reçióter fáileä×AÒNÉNG: locëä: cáîîot coîtáct ótátä

To fix the console, we need to login to the Solaris server via SSH and edit the /etc/ttydefs and add the following line:

console:9600 hupcl opost onlcr crtscts:9600::console

After adding the following line, your /etc/ttydefs would look like this:

[email protected]# cat /etc/ttydefs
# VERSION=1
460800:460800 hupcl:460800 hupcl::307200
307200:307200 hupcl:307200 hupcl::230400
230400:230400 hupcl:230400 hupcl::153600
153600:153600 hupcl:153600 hupcl::115200
115200:115200 hupcl:115200 hupcl::76800
76800:76800 hupcl:76800 hupcl::57600
57600:57600 hupcl:57600 hupcl::38400
38400:38400 hupcl:38400 hupcl::19200
19200:19200 hupcl:19200 hupcl::9600
9600:9600 hupcl:9600 hupcl::4800
4800:4800 hupcl:4800 hupcl::2400
2400:2400 hupcl:2400 hupcl::1200
1200:1200 hupcl:1200 hupcl::300
300:300 hupcl:300 hupcl::460800

460800E:460800 hupcl evenp:460800 evenp::307200
307200E:307200 hupcl evenp:307200 evenp::230400
230400E:230400 hupcl evenp:230400 evenp::153600
153600E:153600 hupcl evenp:153600 evenp::115200
115200E:115200 hupcl evenp:115200 evenp::76800
76800E:76800 hupcl evenp:76800 evenp::57600
57600E:57600 hupcl evenp:57600 evenp::38400
38400E:38400 hupcl evenp:38400 evenp::19200
19200E:19200 hupcl evenp:19200 evenp::9600
9600E:9600 hupcl evenp:9600 evenp::4800
4800E:4800 hupcl evenp:4800 evenp::2400
2400E:2400 hupcl evenp:2400 evenp::1200
1200E:1200 hupcl evenp:1200 evenp::300
300E:300 hupcl evenp:300 evenp::19200

auto:hupcl:sane hupcl:A:9600

console1:1200 hupcl opost onlcr:1200::console2
console2:300 hupcl opost onlcr:300::console3
console3:2400 hupcl opost onlcr:2400::console4
console4:4800 hupcl opost onlcr:4800::console5
console5:19200 hupcl opost onlcr:19200::console

contty:9600 hupcl opost onlcr:9600 sane::contty1
contty1:1200 hupcl opost onlcr:1200 sane::contty2
contty2:300 hupcl opost onlcr:300 sane::contty3
contty3:2400 hupcl opost onlcr:2400 sane::contty4
contty4:4800 hupcl opost onlcr:4800 sane::contty5
contty5:19200 hupcl opost onlcr:19200 sane::contty

4800H:4800:4800 sane hupcl::9600H
9600H:9600:9600 sane hupcl::19200H
19200H:19200:19200 sane hupcl::38400H
38400H:38400:38400 sane hupcl::2400H
2400H:2400:2400 sane hupcl::1200H
1200H:1200:1200 sane hupcl::300H
300H:300:300 sane hupcl::4800H

conttyH:9600 opost onlcr:9600 hupcl sane::contty1H
contty1H:1200 opost onlcr:1200 hupcl sane::contty2H
contty2H:300 opost onlcr:300 hupcl sane::contty3H
contty3H:2400 opost onlcr:2400 hupcl sane::contty4H
contty4H:4800 opost onlcr:4800 hupcl sane::contty5H
contty5H:19200 opost onlcr:19200 hupcl sane::conttyH

console:9600 hupcl opost onlcr crtscts:9600::console

 

We Sun Solve! solves SunSolve

Missed the old SunSolve where you can check and download the latest patches and fixes for Sun Solaris? We all know that SunSolve is now decommisioned by Oracle and has replaced it with Oracle Support and thanks to it’s Flash interface its one hell of sluggish support site.

Look no further because We Sun Solve! is here!!! Their patch database are always updated and easily seachable.

So if you are like me that missed the old SunSolve, go to We Solve Sun!

Integrated Lights Out Manager (ILOM) CLI Cheat Sheet

Learning the Integrated Lights Out Manager (ILOM) will be a big help when a System Administrator cannot login anymore to the server. ILOM can be accessed via IP network or serial console. Here are some of the most commonly used commands that could get anyone on track on managing the ILOM:

Display SP firmware version:

-> version

Show available CLI commands:

-> show /SP/cli/commands

Display help information about commands and targets

-> help

Connect to host serial console:

-> start /SP/console

List users with open sessions to the system console (who is using the console?):

-> show /SP/sessions

Terminate a server console session started by another user:

-> stop /SP/console

Power on host:

-> start /SYS

Power off host:

-> stop /SYS

Reset host:

-> reset /SYS

Reset ILOM:

-> reset /SP

Enable locate LED:

-> set /SYS LOCATE=on

Disable locate LED:

-> set /SYS LOCATE=off

Add a local user account:

-> create /SP/users/username password=password role=[administrator|operator]

Displaying Users:

-> show /SP/users

Unix or Linux Crontab Headers

Even though cron is our usual lifesaver for redundant tasks on a Unix or Linux Systems, there are times that we really forget the syntax that is needed for our cronjobs. Here is a simple yet effective headers that we could place on top of the crontab as comments:

###############################################################
#.---------------- minute (0 - 59)
#|  .------------- hour (0 - 23)
#|  |  .---------- day of month (1 - 31)
#|  |  |  .------- month (1 - 12) OR jan,feb,mar,apr ...
#|  |  |  |  .---- day of week (0 - 6) (Sunday=0 or 7)  OR
#|  |  |  |  |         sun,mon,tue,wed,thu,fri,sat
#|  |  |  |  |
*  *  *  *  *  command to be executed