FAQ: Error Resizing a Logical Volume in AIX

"There’s No Such Thing As A Silly Question" -- does the cliche sound familiar? In this part of pimp-my-rig reloaded, technical questions are answered. Mail them to me and I will post the answers here. If you have a better answer, by all means share it with us.

The beauty of a volume manager is that the backing device, or meta device as what other operating systems would call it, is made independent from the operating system. The storage device itself is controlled by another layer that is put in place by the volume manager. Thereby, the operating system does not have direct control over this storage. This way the volume group and its underlying logical volumes can be resized on demand.

Several operating systems implement this kind of obscurity including but not limited to Linux and Unix. Today we take a look at this functionality in AIX. However, resizing volumes sometimes has its own set of headaches and we need to be prepared to get around the configuration and the issues encountered each time.

Q. A colleague was trying to resize a logical volume since the application has demanded increased disk space and more breathing room. However, what he thought was a simple change to execute resulted in an error. And thus opened additional learning opportunities for us both. The change and its error was below:

root@host:/root => chfs -a size=+24G /dev/appdata01lv
0516-787 extendlv: Maximum allocation for logical volume appdata01lv
        is 3804.
root@host:/root => lslv appdata01lv
LOGICAL VOLUME:     appdata01lv            VOLUME GROUP:   appvg
LV IDENTIFIER:      00c175f4004c00635401257a0b3fe3.9 PERMISSION:     read/write
VG STATE:           active/complete        LV STATE:       opened/syncd
TYPE:               jfs2                   WRITE VERIFY:   off
MAX LPs:            3804                   PP SIZE:        32 megabyte(s)
COPIES:             1                      SCHED POLICY:   parallel
LPs:                3804                   PPs:            3804
STALE PPs:          0                      BB POLICY:      relocatable
INTER-POLICY:       maximum                RELOCATABLE:    yes
INTRA-POLICY:       middle                 UPPER BOUND:    1024
MOUNT POINT:        /mount/point/info      LABEL:          /label/has/been/hidden
MIRROR WRITE CONSISTENCY: on/ACTIVE
EACH LP COPY ON A SEPARATE PV ?: yes
Serialize IO ?:     NO
DEVICESUBTYPE : DS_LVZ
COPY 1 MIRROR POOL: None
COPY 2 MIRROR POOL: None
COPY 3 MIRROR POOL: None

A. This change can be done online without losing data nor incurring any downtime. And that is the beauty of a volume manager.

As seen from the output of the commands above, he was trying to add 24GB of space to an existing volume "appdatal01lv" but the logical volume itself seems to deny the adding of additional space to it saying that the "Maximum allocation for logical volume appdata01lv" has been hit.

He got a bit stuck here for a bit, while I was reviewing my notes from changes that I executed before. I knew I had hit this roadblock before.

From a previous change I implemented, I modified the property "MAX LPs" for the target logical volume before adding space to that volume. That step, though sometimes no longer necessary, was missing in his procedure and this time it is not optional.

We implemented the additional step to his procedure and it executed with success.

root@host:/root => chlv -x 7500 appdata01lv
root@host:/root => chfs -a size=+24G /dev/appdata01lv
Filesystem size changed to 299630592
root@host:/root => lslv appdata01lv
LOGICAL VOLUME:     appdata01lv            VOLUME GROUP:   appvg
LV IDENTIFIER:      00c175f4004c00635401257a0b3fe3.9 PERMISSION:     read/write
VG STATE:           active/complete        LV STATE:       opened/syncd
TYPE:               jfs2                   WRITE VERIFY:   off
MAX LPs:            7500                   PP SIZE:        32 megabyte(s)
COPIES:             1                      SCHED POLICY:   parallel
LPs:                4572                   PPs:            4572
STALE PPs:          0                      BB POLICY:      relocatable
INTER-POLICY:       maximum                RELOCATABLE:    yes
INTRA-POLICY:       middle                 UPPER BOUND:    1024
MOUNT POINT:        /mount/point/info      LABEL:          /label/has/been/hidden
MIRROR WRITE CONSISTENCY: on/ACTIVE
EACH LP COPY ON A SEPARATE PV ?: yes
Serialize IO ?:     NO
DEVICESUBTYPE : DS_LVZ
COPY 1 MIRROR POOL: None
COPY 2 MIRROR POOL: None
COPY 3 MIRROR POOL: None

To change the "MAX LPs" property, chfs -x NEW_VALUE has to be executed to the logical volume.

For this particular error message, the solution is to modify "MAX LPs" to cater to the additional storage requirement. You can check the current allocation from the value of "LPs" just another line below "MAX LPs". The change should not make the value of LPs go over its set MAX.

If you have changes involving increase in disk space using the AIX volume manager, better check for the value of "MAX LPs" before executing the change. Or you can note the additional step above so when implementation time comes, it will proceed without further delays.

Read More

INFO: What is a "Sticky Bit"?

At a recent job interview, I was asked this question. I was able to satisfactorily answer the question but failed to answer the follow-up question -- what is the difference between the sticky bit "t" (small letter) and the sticky bit "T" (bold letter)? This experience led me to improve my knowledge further. This post is dedicated to the experience I had, and to share the knowledge to those who quest for it.

What is the "Sticky Bit"? ("t" and "T"). The sticky bit is primarily used on shared directories with open permissions, such as /var/tmp and /tmp. Directories with open permissions (or publicly writable) present a lot of problems because users **CAN CREATE** files, **READ/EXECUTE** files owned by other users, and at the same time are **ALLOWED EDIT/REMOVE** privileges to files owned by other users.

With sticky bit set on publicly writable directories, users **CAN CREATE** files, **READ/EXECUTE** those files as well as files owned by other users, but are **NOT ALLOWED EDIT/REMOVE** privileges to files they don't own. The only exception to this is the super-user root which can edit or remove files.

In simple terms, sticky bit is used to indicate special treatment of certain directories. A directory for which the sticky bit is set restricts deletion or modification of files it contains. A file in a sticky bit set directory may only be removed, modified or renamed by a user who has write permission on the directory, and either owns the file, owns the directory, or is the super-user. This is useful for directories such as /tmp, which must be publicly writable, but should deny users permission to arbitrarily delete or rename the files of others.

It is worthy to note here that the directories with open permissions (or publicly writable) can be any arbitrary directory, not just limited to /tmp and /var/tmp. Its just that /tmp and /var/tmp exist by default on the system.

Which directory has sticky permissions? How do you identify a directory that has sticky bit set? To illustrate further see the example below.

user@host# ls -ld /sticky/directory
drwxrwxrwt  2   user   group   512   Jun 20 11:02  /sticky/directory

On the output of the "ls" command, see the "t" (on drwxrwxrwt)? That tells us that the sticky bit is set.

A "T" refers to when the execute permissions are OFF.
A "t" refers to when the execute permissions are ON.

How to set sticky bit permissions. The leading "1" in the "chmod" command sets the sticky bit.

user@host# chmod 1777 /sticky/directory (permissions for "other" have execute bit ON) 
drwxrwxrwt
user@host# chmod 1776 /sticky/directory (permissions for "other" have execute bit OFF)
drwxrwxrwT

The equivalent of the above commands in another syntax.

user@host# chmod o+t /sticky/directory
user@host# chmod o+x /sticky/directory
drwxrwxrwt        
user@host# chmod o+t /sticky/directory
user@host# chmod o-x /sticky/directory
drwxrwxrwT

Or..

user@host# chmod o=rwxt /sticky/directory
drwxrwxrwt        
user@host# chmod o=rwt /sticky/directory
drwxrwxrwT

There you go, more information regarding sticky bit. To my knowledge, sticky bit for files are not supported by any modern implementation of Unix or Linux. It is only significant and applicable to directories.

Read More

HOW-TO: Tabbed Putty with PuttyCM (Windows 7)

If you work as a system administrator and your workstation has windows (particularly Windows 7), chances are you connect to the Linux, Solaris, AIX or whatever flavor of *nix using putty. If by any chance you don't you might want to try putty, instead of the current program you're using.

I have used putty since I started working as a Unix sysad. And I can tell that putty in itself is a superb and versatile tool. But work starts getting complicated once you have several putty sessions open. And it becomes even challenging once you open more putty sessions. PuttyCM (or Putty Connection Manager) organizes putty sessions into tabs or split windows (by dragging a tab). This way, working becomes organized and much faster. You may want to download PuttyCM to try it out.

However, PuttyCM doesn't work out-of-the-box in Windows 7. So here's a pimp-my-rig feature designed to guide you in configuring PuttyCM to make it work in Windows 7. To give you a better understanding of what I mean by it doesn't work, here is the screenshot of PuttyCM connecting to my Popcorn Hour C-200 NMJ.

As you can see, it doesn't work well. You are able to use the connection manager or putty sessions but it doesn't display the connection properly. And you can't do your job with PuttyCM. So it is just back to plain Putty or so it seems.

Here's how to make PuttyCM work in Windows 7.

Explore to the directory containing the PuttyCM executable (its filename is: puttycm.exe). Right-click the executable file and go to Compatibility tab. Choose compatibility for Windows Vista. You may choose Vista SP1 or SP2 as these work as well. Toward the bottom of the Compatibility tab, tick "Run this program as an administrator". You will be seeing a window similar to the one below.

Now try to launch PuttyCM and connect to a host.

How sweet it is to see the terminal connection display.

So there you go folks, working tabbed putty with PuttyCM in Windows 7. As of this writing, PuttyCM version 0.7.1 build 136 (beta) works. And the current alpha 0.7.1 build 233 works in Windows 7 and Windows 7 SP1.

Read More

FAQ: Error Running 64bit Commands

"There’s No Such Thing As A Silly Question" -- does the cliche sound familiar? In this part of pimp-my-rig reloaded, technical questions are answered. Mail them to me and I will post the answers here. If you have a better answer, by all means share it with us.

Q. I recently encountered an error that made me scratch my head. I knew I had encountered it before and had resolved the problem. Unfortunately, my notes were not available at work due to the security policies in place. But when I got home, the putty logs that I saved from before proved really useful.

Encountering weird errors like this one really makes its mark on your memory. To give you an idea what it is, see below. Simple commands like uptime will return errors, and will not run. In fact, all 64-bit executables refuse to run and will give the same error as below.

user@host > uptime
exec(): 0509-036 Cannot load program uptime because of the following errors:
        0509-033 Cannot run a 64-bit program until the 64-bit
        environment has been configured. See the system administrator.

A. Fortunately, I enabled logging on my putty and saved them for future review. And the time for reaping the benefits of that move paid out.

This was how I resolved that problem above. The command to run is "smitty load64bit" or "smit load64bit" (should be executed as root or a privileged user). After smit loads, select "Enable Now".

Once executed, 64bit commands will now run without hitch. No need to reboot the machine.

To my curiosity, I checked the output of smit.log and below is how it looks.

user@host > cat smit.log

[Apr 28 2010, 07:31:21]
    Starting SMIT

(Menu screen selected as FastPath,
        id          = "system",
        id_seq_num  = "220",
        next_id     = "load64bit",
        title       = "Enable 64-bit Application Environment".)

(Menu screen selected,
        FastPath    = "load64bit",
        id_seq_num  = "220",
        next_id     = "load64bit",
        title       = "Enable 64-bit Application Environment".)

(Dialogue screen selected,
        FastPath    = "run64now",
        id          = "run64now",
        title       = "".)

[Apr 28 2010, 07:31:33]

    Command_to_Execute follows below:
>> /etc/methods/cfg64


    Output from Command_to_Execute follows below:

---- start ----


---- end ----

[Apr 28 2010, 07:31:37]
    Exiting SMIT

From the output of smit.log above, you can deduce that if you will not be using smit you can instead use "/etc/methods/cfg64" to accomplish the same results.

Read More

FAQ: Powermt Returns Initialization Error

"There’s No Such Thing As A Silly Question" -- does the cliche sound familiar? In this part of pimp-my-rig reloaded, technical questions are answered. Mail them to me and I will post the answers here. If you have a better answer, by all means share it with us.

Q. I was working on an AIX host when I encountered "Initialization Error." when running "powermt display". This error is present when running as sudo. But I tried running as root, hoping things would be different as super-user but that was not the case. The errors I encountered are below.

user@host > sudo powermt display
Initialization error.
user@host > su
Password:
root@host > powermt display
Initialization error.

A. This puzzled me even more. Even as the super-user "root", the error was persistent. I'm not that very well versed with AIX, so I tried using a command that was always useful in Solaris -- truss. I then hit jackpot!

This is the output of the command.

root@host > truss powermt display

execve("/usr/sbin/powermt", 0x2FF22C94, 0x2000FB38)  argc: 2
__loadx(0x03480000, 0x2FF22A70, 0x00000140, 0x10000000, 0x2000033D) = 0x00000000
__loadx(0x0C000000, 0xF020AABC, 0x00000000, 0xF0317004, 0xF020A460) = 0x00000000
thread_init(0x0000000000000000, 0x00000000D0117CCC) = 
sbrk(0x00000000)                                = 0x20018404
vmgetinfo(0x2FF22978, 7, 16)                    = 0
sbrk(0x00000000)                                = 0x20018404
vmgetinfo(0x2FF22440, 7, 16)                    = 0
sbrk(0x00000000)                                = 0x20018404
sbrk(0x0000000C)                                = 0x20018404
__libc_sbrk(0x00000000)                         = 0x20018410
getrpid(-1, -1, -265185236)                     = 253984
_getpid()                                       = 253984
getprocs(0x20018ED8, 920, 0x00000000, 0, 0xF03173B0, 1) = 1
appulimit(1005, 0)                              = 0x20029000
_thread_self()                                  = 610395
thread_setmystate(0x00000000, 0x2FF225D0)       = 0
thread_setmystate(0x2FF222C0, 0x2FF225C8)       = 0
_sigaction(3, 0x2FF228F8, 0x2FF22908)           = 0
_sigaction(4, 0x2FF228F8, 0x2FF22908)           = 0
_sigaction(5, 0x2FF228F8, 0x2FF22908)           = 0
_sigaction(6, 0x2FF228F8, 0x2FF22908)           = 0
_sigaction(7, 0x2FF228F8, 0x2FF22908)           = 0
_sigaction(8, 0x2FF228F8, 0x2FF22908)           = 0
_sigaction(10, 0x2FF228F8, 0x2FF22908)          = 0
_sigaction(11, 0x2FF228F8, 0x2FF22908)          = 0
_sigaction(12, 0x2FF228F8, 0x2FF22908)          = 0
_sigaction(36, 0x2FF228F8, 0x2FF22908)          = 0
_sigaction(39, 0x2FF228F8, 0x2FF22908)          = 0
open("/unix", O_RDONLY|O_LARGEFILE)             = 3
kioctl(3, 22528, 0x00000000, 0x00000000)        Err#25 ENOTTY
kioctl(3, 22528, 0x00000000, 0x00000000)        Err#25 ENOTTY
kread(3, "01F7\005 IFC aE6\0\0\0\0".., 4096)    = 4096
klseek(3, 0, 0, 0x00000001)                     = 0
klseek(3, 0, 13942134, 0x00000000)              = 0
kread(3, "\0\0\0\0\0\0 1\0\0\0\004".., 4096)    = 4096
klseek(3, 0, 0, 0x00000001)                     = 0
klseek(3, 0, 0, 0x00000001)                     = 0
klseek(3, 0, 15041308, 0x00000000)              = 0
kread(3, " _ _ m u l h\0 _ _ m u l".., 4096)    = 4096
klseek(3, 0, 0, 0x00000001)                     = 0
klseek(3, 0, 13942170, 0x00000000)              = 0
kread(3, "\0\0\0\0\0\0 180\0\0\0\v".., 4096)    = 4096
klseek(3, 0, 8, 0x00000001)                     = 0
kread(3, "\0\0\0\001 XF1B8\0\0028E".., 4096)    = 4096
klseek(3, 0, 8, 0x00000001)                     = 0
kread(3, "\0\0\0\001 XF5 H\0\0\tE6".., 4096)    = 4096
klseek(3, 0, 8, 0x00000001)                     = 0
kread(3, "\0\0\0\001 XF8D8\0\0028E".., 4096)    = 4096
klseek(3, 0, 8, 0x00000001)                     = 0
kread(3, "\0\0\0\001 XFC h\0\00F12".., 4096)    = 4096
klseek(3, 0, 8, 0x00000001)                     = 0
kread(3, "\0\0\0\001 XFFF8\0\0028E".., 4096)    = 4096
klseek(3, 0, 8, 0x00000001)                     = 0
kread(3, "\0\0\0\001 Y0388\0\0149F".., 4096)    = 4096
klseek(3, 0, 8, 0x00000001)                     = 0
kread(3, "\0\0\0\001 Y0718\0\0028E".., 4096)    = 4096
klseek(3, 0, 8, 0x00000001)                     = 0
kread(3, "\0\0\0\001 Y\nA8\0\0028E".., 4096)    = 4096
klseek(3, 0, 8, 0x00000001)                     = 0
kread(3, "\0\0\0\001 Y0E 8\0\019AC".., 4096)    = 4096
klseek(3, 0, 8, 0x00000001)                     = 0
kread(3, "\0\0\0\001 Y11C8\0\01E <".., 4096)    = 4096
klseek(3, 0, 8, 0x00000001)                     = 0
kread(3, "\0\0\0\001 Y15 X\0\0 " W".., 4096)    = 4096
klseek(3, 0, 8, 0x00000001)                     = 0
kread(3, "\0\0\0\001 Y18E8\0\0 '99".., 4096)    = 4096
klseek(3, 0, 8, 0x00000001)                     = 0
..
(some output omitted for brevity..)
..
close(3)                                        = 0
open("/etc/emc/mpaa.lams", O_RDWR|O_CREAT, S_IRUSR|S_IWUSR|S_IRGRP|S_IWGRP) = 3
kfcntl(3, F_SETLKW, 0x2FF224B8)                 = 0
open("/etc/emc/mpaa.lams", O_RDONLY)            = 4
kioctl(4, 22528, 0x00000000, 0x00000000)        Err#25 ENOTTY
kioctl(4, 22528, 0x00000000, 0x00000000)        Err#25 ENOTTY
kread(4, " g l o b a l : v e r s i".., 4096)    = 122
kread(4, " g l o b a l : v e r s i".., 4096)    = 0
close(4)                                        = 0
kfcntl(3, F_SETLK, 0x2FF224B8)                  = 0
close(3)                                        = 0
open("/etc/emcp_registration", O_RDONLY)        = 3
kioctl(3, 22528, 0x00000000, 0x00000000)        Err#25 ENOTTY
kioctl(3, 22528, 0x00000000, 0x00000000)        Err#25 ENOTTY
klseek(3, 0, 0, 0x00000000)                     = 0
close(3)                                        = 0
statx("CuDv", 0x2FF21710, 76, 0)                Err#2  ENOENT
statx("CuDv", 0x2FF21710, 76, 0)                Err#2  ENOENT
open("CuDv", O_RDONLY)                          Err#2  ENOENT
Initialization error.
kwrite(2, " I n i t i a l i z a t i".., 22)     = 22
__loadx(0x04000000, 0x2FF20A50, 0x00002000, 0x00000000, 0x00000000) = 0x00000000
kfcntl(1, F_GETFL, 0x1009505B)                  = 67110914
kfcntl(2, F_GETFL, 0x1009505B)                  = 67110914
_exit(103)

As seen from the error above, powermt was looking for the file "CuDv" which is an ODM object. So I tried a trick that I previously noted in post FAQ: Path Errors on AIX ODM Commands.

root@host > cd /etc/objrepos
root@host > powermt display
Symmetrix logical device count=138
CLARiiON logical device count=0
Hitachi logical device count=0
Invista logical device count=0
HP xp logical device count=0
Ess logical device count=0
HP HSx logical device count=0
==============================================================================
----- Host Bus Adapters ---------  ------ I/O Paths -----  ------ Stats ------
###  HW Path                       Summary   Total   Dead  IO/Sec Q-IOs Errors
==============================================================================
   0 fscsi0                        optimal     138      0       -     0      0
   1 fscsi2                        optimal     138      0       -     0      0

This was deja vu all over again. But I now know that when facing these errors, it is useful to just change directory to /etc/objrepos before executing the privileged commands.

Read More

FAQ: Path Errors on AIX ODM Commands

"There’s No Such Thing As A Silly Question" -- does the cliche sound familiar? In this part of pimp-my-rig reloaded, technical questions are answered. Mail them to me and I will post the answers here. If you have a better answer, by all means share it with us.

Q. I was working on an AIX host when I encountered weird errors when invoking commands involving the ODM (short for Object Data Manager). The errors I encountered are below.

root@host > bootlist -m normal -o
-
-
root@host > bootlist -m normal hdisk0 hdisk2
0514-209 bootlist: Was not able to access the CuDv object class
root@host > bootlist -m normal hdisk0
0514-209 bootlist: Was not able to access the CuDv object class
root@host > odmget -q attribute=TCB_STATE PdAt 
0518-506 odmget: Cannot open object class PdAt
Check path name and permissions.
root@host > rmlv dump02
0518-506 odmget: Cannot open object class PdAt
Check path name and permissions.
Warning, all data contained on logical volume dump02 will be destroyed.
rmlv: Do you wish to continue? y(es) n(o)? y
0518-506 odmget: Cannot open object class SWservAt
Check path name and permissions.

A. To get a background of what ODM is, AIX stores most of the system management information in /etc/objrepos, /usr/lib/objrepos, and /usr/share/lib/objrepos. Files (also referred to as system object classes) in these directories are adminstered by the Object Database Manager which is a set of library routines and programs providing basic object oriented database facilities.

Any changes you make to the configuration reflects or directly changes files on the ODM. Under most circumstances, only SMIT or the commands SMIT call should be used to change the contents of the system object classes. A harmless way to look at the object database is to use odmget <Class> where <Class> is one of the files in /etc/objrepos.

So after taking a look at the environment and checking that the environment ODMDIR correctly points to /etc/objrepos, we got to a dead-end.

root@host > echo $ODMDIR
ODMDIR=/etc/objrepos

To the rescue is a colleague who is very well versed with AIX. In fact, I consider him a guru in AIX. His suggestion was to try excuting the above commands under the /etc/objrepos directory and see if the errors would go away. And we hit paydirt!

root@host > cd /etc/objrepos
root@host > bootlist -m normal -p o
hdisk0 blv=hd5
hdisk2 blv=hd5
root@host > bootlist -m normal hdisk0
root@host > savebase
root@host > bootlist -m normal -o
hdisk0 blv=hd5

The above is a simple workaround to what may seem to be a big problem. Change directory first to /etc/objrepos then execute the commands.

If you have a permanent solution to the problem, share it with us.

Read More

FAQ: Cannot Fork: No Swap Space

"There’s No Such Thing As A Silly Question" -- does the cliche sound familiar? In this part of pimp-my-rig reloaded, technical questions are answered. Mail them to me and I will post the answers here. If you have a better answer, by all means share it with us.

There are times when you run into uncharted territory, when you least expect it. A host was failing to execute commands, giving a no swap space error even though swap space was only 9% used. Commands fails to execute but work intermitently.

This happened on an IBM server running AIX 5.3.

All of our efforts to resolve this issue failed, so we called the vendor -- IBM. The error manifests itself as:

user@host# uptime 
ksh: cannot fork: no swap space 
user@host# lsps -s 
Total Paging Space   Percent Used 
4096MB               9% 

Here is a solution to the problem as provided from IBM. It is a known issue from APAR IY89941 fixed at 5.3 TL5 (bos.mp 5.3.0.53) and above. The root cause is that you are running out of 64K pages in the VMM and there is code bug when this type of scenario occurs.

The work around is to disable 64K page size support (until upgrade to higher levels of AIX). To disable run command:

root@host# vmo -r -o vmm_mpsize_support=0

This change is not dynamic. A reboot is required after making this change.

Read More

FAQ: PRNG is not SEEDED

"There’s No Such Thing As A Silly Question" -- does the cliche sound familiar? In this part of pimp-my-rig reloaded, technical questions are answered. Mail them to me and I will post the answers here. If you have a better answer, by all means share it with us.

Q: I was trying to help out a friend who was in a dilemma he had already resolved. While trying to scp to a host, he got the error message "PRNG is not SEEDED". Only the super-user root was able to scp, but he wanted drill down to the root of the problem. After all, who wants to execute scp as root, right?

He was able to successfully scp on other machines as himself, but on this particular machine he was only able to do it as the super-user, root. It is worthy to note here that ssh works, but scp does not.

A: When trying to run scp and you get an error message "PRNG is not SEEDED", you probably have an issue with the /dev/random and/or /dev/urandom devices on your system. These devices are created by default during system installation.

Check that proper permissions are present on random numbers generators. The correct permissions should be "others" must have "read" access to these devices. Below are the permissions on a working system:

# ls -l /dev/random /dev/urandom
crw-r--r-- 1 root system 39, 0 Jun 22 10:48 /dev/random
crw-r--r-- 1 root system 39, 1 Jun 22 10:48 /dev/urandom

To corrent, change them like below:

# chmod o+r /dev/random /dev/urandom 

Now stop and start the SSH daemon again, and retry if scp works.

# stopsrc -s sshd
# startsrc -s sshd

For unknown reasons, there are times when the files /dev/random and /dev/urandom are missing from the system. If this is the case, you will need to create them. Execute the following commands:

# stopsrc -s sshd
# mknod /dev/random c 39 0
# mknod /dev/urandom c 39 1
# randomctl -l
# ls -ald /dev/random /dev/urandom
# startsrc -s sshd

So far problems with s-commands I have encountered are related to the random generator character devices. I have encountered similar problems while trying to blank a forgotten root password and setting a new one on a Solaris machine.

You may further drill down on the cause of the problem by using strace or truss, whichever is applicable on your *nix system.

Read More

FAQ: Locate's Equivalent Command in AIX

"There’s No Such Thing As A Silly Question" -- does the cliche sound familiar? In this part of pimp-my-rig reloaded, technical questions are answered. Mail them to me and I will post the answers here. If you have a better answer, by all means share it with us.

Q. Just recently I asked a colleague (whom I consider to be a guru of AIX) what the equivalent of the Linux command locate is on this particular operating system. The equivalent Linux and Solaris commands are previously documented here.

A. Unfortunately, his answer was there is none. But the discussion did not end there. He suggested that I use another command that has a similar function and is quite as flexible as locate.

The commands is whereis (and which_fileset). And this whereis is part of the package bos.rte.commands and bos.compat.links. To determine if whereis is installed on the machine, check as outlined below:

On AIX 5.3:

user@host => which_fileset whereis
/usr/ucb/whereis -> /usr/bin/whereis    bos.compat.links 5.3.0.0
/usr/bin/whereis                        bos.rte.commands 5.3.0.0

On AIX 6.1:

user@host => which_fileset whereis
/usr/ucb/whereis -> /usr/bin/whereis    bos.compat.links 6.1.0.0
/usr/bin/whereis                        bos.rte.commands 6.1.0.0

The above command is dependent on another package and without that there will be errors as outlined below.

user@host => which_fileset whereis
The /usr/lpp/bos/AIX_file_list file does not exist.
Please install the "bos.content_list" fileset
and try the "which_fileset" command again.

When all is well. Try it out:

user@host => whereis hosts
hosts: /etc/hosts.allow /etc/hosts.deny /etc/hosts /etc/hosts.bak /usr/share/man/man5/hosts.5.gz

Did I mention the same command works for Solaris and Linux? It does (given the dependent packages are installed), so add this option on those operating systems too!

Read More

FAQ: Change NIC Speed in AIX

"There’s No Such Thing As A Silly Question" -- does the cliche sound familiar? In this part of pimp-my-rig reloaded, technical questions are answered. Mail them to me and I will post the answers here. If you have a better answer, by all means share it with us.

Experience in administering a Solaris Unix box made me think changing the speed of a network interface can be a simple execution of a series of commands -- but not in AIX. My experience was not so simple.

Recently while trying to change the speed of a NIC in an AIX box, I got into errors. Errors which I discovered to be a common discussion among system administrators in the web. The error is below:

Method error (/usr/lib/methods/chgent):
0514-062 Cannot perform the requested function because the
specified device is busy.

This was after the interface was "down"-ed and "detach"-ed, executing "chdev -l ent0 -a media_speed=1000_Full_Duplex". Being an AIX sysad amateur myself, I came to consult a colleague who has more in-depth knowledge and experience in AIX. I am grateful for his insight into the matter and here is the outline of what he did.

Here is a brief background of our objective -- change the NIC speed from Auto_Negotiation to 1000_Full_Duplex in an AIX box. And the workplan in place is:

[1] ifconfig en0 down detach

[2] chdev -l ent0 -a state=detach

[3] chdev -l ent0 -a media_speed=1000_Full_Duplex

[4] smit chinet >> (select interface) and change Current STATE to "up"
(NOTE: using "ifconfig en0 up" here sometimes assigns IP address of 0.0.0.0, so use smit instead)

[5] mkdev -l en0

I never got to execute beyond step [3] due to the error. So I sought for help.

What we did was to modify step [3] and inserted another step in the procedure : rmdev -l ent0. Which when executed outputs:

# rmdev -l ent0
ent0 Defined

After which, succeeding steps in the procedure above went smoothly.

To verify the change, execute entstat on the interface:

# entstat -d ent0 | grep Speed

Media Speed Selected: 1000 Mbps / 1 Gbps, Full Duplex
Media Speed Running: 1000 Mbps / 1 Gbps, Full Duplex

Another remarkable point to note here is that in step [4] of the original procedure, executing ifconfig to change the state of the device to "up" sometimes assigns it an IP address of 0.0.0.0.

As a Solaris sysad, this change in AIX was very much new to me. But hey, it is not bad to learn something new. And I hope that by sharing the inserted step in the above procedure, this little experience might be able to help other sysads stuck in the same error.

Read More

FAQ: More Secure Password-less SSH -- Windows to Linux

"There’s No Such Thing As A Silly Question" -- does the cliche sound familiar? In this part of pimp-my-rig reloaded, technical questions are answered. Mail them to me and I will post the answers here. If you have a better answer, by all means share it with us.

The password-less SSH procedure previously posted outlined the establishment of trusted logins from a Windows machine to a _nix machine. The set-up works and is very convinient to implement. However, it comes at the expense of security.

FAQ: Many would agree that an unprotected private key -- a private key with empty passphrase -- is less secure and thus exposes the account to a security threat. Not that this set-up is totally unsecure but once the private key is compromised, the account is completely vulnerable as access to it is open. Can we make the set-up a more secure then?

Protecting the private key with a passphrase is a very logical thing to do. Not only that, it is highly recommended especially on platforms or systems where security is a major concern. The private key then becomes useless without the passphrase to unlock it.

In order to accomplish this stunt, a similar set of software for the previous Windows to _nix password-less setup is required: putty.exe, puttygen.exe and pscp.exe, with the addition of pageant.exe. The tools mentioned can be downloaded off the author's website. Download the binaries enumerated.

Start off by launching puttygen.exe. This tool will generate the public and private key pairs needed for the password-less setup. This was done previously but this time, protect the private key with a passphrase. Press "Generate" and put a passphrase in the fields where it is required. Then, save the private and public keys.

Open putty.exe. Scroll down to Connection --> Data.. fill in the field "Auto-login username". For this example, the username used is "user" (fill this field with your username).

Scroll up to Session, fill up the necessary fields and save.

Open a command tool and using pscp.exe, copy the public key over to the home directory of user. The public key has to be translated to OpenSSH format, then has to be added to authorized_keys file.

user@host:~ > ssh-keygen -i -f PUBLIC_KEY >> $HOME/.ssh/authorized_keys

Now launch pageant.exe (or PuTTY authentication agent). No window will be opened but instead another icon will appear in the system tray. Right-click on this icon and select "Add Key" (see below).

Browse over to the path where the private key was saved. Key in the passphrase when prompted to do so.

(Skipping the above step will prompt the user for the password. When this happens, use the the password to the unix account not the passphrase to the private key.)

Right-click, again, on the pageant icon in the system tray and choose the session saved earlier in this guide. An ssh login will be initiated with the host without asking for a password.

There you go, a more secure password-less ssh from Windows to your _nix workstation. Access to the saved private key does not compromise security as it requires the passphrase to unlock the key. However, this introduces the dependence to pageant, where the passphrase is asked only once but password-less still.

Pageant will be void of keys each time it is launched. And, consequently, each time the private key is "added" to pageant, the passphrase will be asked to unlock the key.

Compare the password-less implementations and select which is easier, applicable and better suited for your use. Each has its own set of advantages and disadvantages.

Read More

FAQ: Password-less SSH -- Windows to Linux

"There’s No Such Thing As A Silly Question" -- does the cliche sound familiar? In this part of pimp-my-rig reloaded, technical questions are answered. Mail them to me and I will post the answers here. If you have a better answer, by all means share it with us.

Password-less SSH setups save time and hassle. It plays an important role especially when triggering scripts or copying files to a remote machine. Previous posts have outlined a few of the scenarios where the set-up can apply -- for the same account and for establishing trust between distinct accounts.

FAQ: There is another scenario where password-less ssh can apply. This scenario is very typical in support teams, as I have experienced, whereby a system administrator administers machines remotely from his Windows desktop. Can password-less ssh still apply?

Start off by downloading the necessary tools. For this trick to work, you will be needing the following software binaries: putty.exe, puttygen.exe and pscp.exe. The tools mentioned can be downloaded off the author's website. If you have them readily available, let us proceed.

Start off by launching puttygen.exe. This tool will generate the public and private key pairs needed for the password-less setup, similar to ssh-keygen.

Press "Generate" and help the software do its thing. Afterward, save the private and public keys. And when prompted with a warning for empty passphrase, confirm the save. As always, save the private key to a secure location, where only you can access it (as much as possible).

Open putty.exe. Scroll down to Connection --> Data.. fill in the field "Auto-login username". For this example, the username used is "user".

Scroll further down.. Under Connection --> SSH --> Auth. Point it to the path where the private key file was saved. This will be the private key to be used for password-less authentication.

Scroll up to Session, fill up the necessary fields and save.

Open a command tool and with pscp.exe, copy the public key over to the home directory of user. The public key has to be translated to OpenSSH format, then has to be added to authorized_keys file.

user@host:~ > ssh-keygen -i -f PUBLIC_KEY >> $HOME/.ssh/authorized_keys

Go back to putty.exe, load the saved session (double click the saved session) or click Open and see password-less ssh in action.

There you go. Password-less ssh from Windows to your _nix workstation. Give it a go and save yourself the hassle of presenting credentials again and again.

Read More

FAQ: Password-less SSH -- Two Distinct Accounts

"There’s No Such Thing As A Silly Question" -- does the cliche sound familiar? In this part of pimp-my-rig reloaded, technical questions are answered. Mail them to me and I will post the answers here. If you have a better answer, by all means share it with us.

FAQ: The previous FAQ outlined password-less ssh setup for a single account. Another scenario where password-less ssh can be set-up is on two distinct accounts. In this scenario, trust can be established one-way or two-way. Again, the question is: How can you set-up password-less ssh?

One-Way Trust. A very good application for this kind of set-up is one user account (username: user) and one application account (username: appl), where the user account is "trusted" by the application account.

Begin by generating the public and private key pair. Use ssh-keygen to generate keys. Just the same, do not use a passphrase for completely password-less logins to work.

user@host:~ > ssh-keygen -t rsa
Generating public/private rsa key pair.
Enter file in which to save the key (/home/user/.ssh/id_rsa):
Created directory '/home/user/.ssh'.
Enter passphrase (empty for no passphrase):
Enter same passphrase again:
Your identification has been saved in /home/user/.ssh/id_rsa.
Your public key has been saved in /home/user/.ssh/id_rsa.pub.
The key fingerprint is:
e8:3a:ad:11:d5:c5:89:7c:32:d6:3f:62:61:12:43:df user@host

You should be seeing the following files inside the .ssh directory (take note of the permissions):

user@host:~/.ssh > ls -la
total 17
drwx------ 2 user users  168 2008-09-09 14:42 .
drwxr-xr-x 9 user users  688 2008-09-09 14:34 ..
-rw-r--r-- 1 user users  622 2008-09-09 14:34 authorized_keys
-rw------- 1 user users 1675 2008-09-09 08:40 id_rsa
-rw-r--r-- 1 user users  396 2008-09-09 08:40 id_rsa.pub

With the public and private key pair generated, the contents of the public key (id_rsa.pub) need to be placed inside the authorized_keys file of the application account. First copy the public key over to the home directory of the application account (key in appl's password when asked):

user@host:~/.ssh > scp id_rsa.pub appl@remote:/home/appl
Password:
id_rsa.pub                                 100%  396     0.4KB/s   00:00

As user "appl", create the .ssh directory with permission 700 (drwx------). A safer and easier way to accomplish this is to generate the public and private key pair as well.

Then save the contents of user's public key to the authorized keys file.

appl@remote:~ > cat id_rsa.pub >> $HOME/.ssh/authorized_keys

On an initial set-up of password-less ssh the file id_rsa.pub can be copied to the file authorized_keys.

user@host:~ > cp id_rsa.pub $HOME/.ssh/authorized_keys

After doing the above steps, subsequent logins for user to appl (at host remote) will not ask for credentials. It will be password-less. One-way trust is established.

Two-Way Trust. As seen above one-way trust can be established by adding the contents of the user's public key to appl's authorized keys. To establish the two-way trust, appl's public key needs to be added to user's authorized keys -- the "reverse".

Generate a public and private key pair, if this has not been generated yet. Assuming that this was done in one of the steps above, all that needs to be done is to copy the public key to the home directory of user.

appl@remote:~/.ssh > scp id_rsa.pub user@host:/home/user
Password:
id_rsa.pub                                 100%  396     0.4KB/s   00:00

Likewise, save the contents of appl's public key to the authorized keys file.

user@host:~ > cat id_rsa.pub >> $HOME/.ssh/authorized_keys

Accomplishing the above steps establishes two-way trust between user and appl. If the accounts are in NIS (auto_home) or a centralized network share for home directories, trust between the accounts can be established from any machine.

In the next FAQ, the steps on password-less ssh from your Windows machine to your _nix machine will be outlined.

Read More

FAQ: Setup Password-less SSH

"There’s No Such Thing As A Silly Question" -- does the cliche sound familiar? In this part of pimp-my-rig reloaded, technical questions are answered. Mail them to me and I will post the answers here. If you have a better answer, by all means share it with us.

FAQ: The age of the use of telnet has gone past. Although it is still used in some applications, ssh seems to have become standard in remotely accessing hosts. However, trying to remotely access another machine (as yourself), you will be asked to present your credentials. This happens again and again. How then can you set-up password-less ssh?

There are several scenarios for password-less ssh. But this FAQ will only cover password-less remote access as the same user (or as yourself).

Begin by accessing a host using your own credentials. Check if the directory $HOME/.ssh already exists. If it exists, ensure that the directory permission is 700 (dwrx------). Otherwise, there is no need to worry as the directory will be created later with the proper permission.

Use ssh-keygen to generate your very own public and private key pair. Do not use a passphrase for completely password-less logins to work.

user@host:~ > ssh-keygen -t rsa
Generating public/private rsa key pair.
Enter file in which to save the key (/home/user/.ssh/id_rsa):
Created directory '/home/user/.ssh'.
Enter passphrase (empty for no passphrase):
Enter same passphrase again:
Your identification has been saved in /home/user/.ssh/id_rsa.
Your public key has been saved in /home/user/.ssh/id_rsa.pub.
The key fingerprint is:
e8:3a:ad:11:d5:c5:89:7c:32:d6:3f:62:61:12:43:df user@host

The above command "ssh-keygen -t rsa" initiated the creation of the key pair. The private key was saved in .ssh/id_rsa. This file is read-only and only for you. No one else must see the content of that file, as it is used to decrypt all correspondence encrypted with the public key, which is aptly named .ssh/id_rsa.pub.

NOTE: Generate the key pair file only once. Otherwise, subsequent re-generations of the key pair will invalidate other password-less set-ups already working for you.

With the public and private key pair generated, the contents of the public key (id_rsa.pub) need to be placed inside the authorized_keys file.

user@host:~/.ssh > cat id_rsa.pub >> authorized_keys

On an initial set-up of password-less ssh the file id_rsa.pub can be copied to the file authorized_keys.

user@host:~/.ssh > cp id_rsa.pub authorized_keys

After doing the above steps, subsequent logins will not ask for credentials. It will be password-less.

Aside from sparing yourself from the hassle of being asked a password for each login, password-less ssh can be deployed and has proven to be advantageous in a lot of situations.

In the next FAQ, the steps on password-less ssh for different users and/or different machines will be outlined.

Read More