Monday, December 14, 2009

NFS/Samba alternative: Mount remote directory over SSH.

It is possible to mount a remote directory as a local file system through SSH by using SSHFS, which is based on FUSE: a library to create filesystems in userspace. There are many other files system based on FUSE .

Advantage:

Secure and easy to setup, only client side needs install SSHFS and FUSE, the server side just needs SSH server with sftp support.

Disadvantage:

Performance Penalty, because data transferred has to be encrypted and decrypted, which is time consuming and CPU intensive.

Install:

Search and install sshfs using package management tool of your Linux flavour.

Example:

- Mount any remote dir:

sshfs user@server:dir /mnt/sshfs

Only root and users in fuse group can use this command.

- Unmount sshfs

fuserumount -u /mnt/sshfs

or

umount /mnt/sshfs (root user only)

Wednesday, November 4, 2009

How to exclude directories with find

To exclude directories with find,grep is the obvious choice  but it is not efficient.Following shows three ways to exclude directories with find

#Sample directories and file, the aim is to exclude directory /tmp/test/test1

$find /tmp/test
/tmp/test
/tmp/test/test1
/tmp/test/test1/test1.file
/tmp/test/test2
/tmp/test/test2/test2.file
/tmp/test/test3

(1) Simple most efficient way, but I can only get it working in ksh

$find /tmp/test/!(test1) -type f
/tmp/test/test2/test2.file
/tmp/test/test3

(2)With not expression “! “

$find /tmp/test ! -path "/tmp/test/test1*" -type f
/tmp/test/test2/test2.file
/tmp/test/test3

(3)With prune, which means exclude preceding path

$find /tmp/test  -path  /tmp/test/test1 -prune  -o -type f -print
/tmp/test/test2/test2.file
/tmp/test/test3


Wednesday, October 14, 2009

Zenoss monitor Windows Server 2008 via WMI

Zenoss supports Windows SNMP, it can get partition and interfaces infomation, but it couldn't get CPU/MEMORY info. WMI script can get almost any info in Windows. Zenoss supports Windows WMI by zenpack
The agent account in remote Windows Server doens't need to be admin user as long as following previleges granted.

Enable DCOM
The easy way is to add the user to group "Distributed COM users"

Alternatively, grant specific rights to the user
Start DCOM GUI by DCOMCNFG command-> Component Services -> Computers->Right-click My Computer, and then select Properties->COM Security tab
Give access permission and launch and activation permission.

Enabling Account Privileges in WMI
Computer Management -> Services and Applications-> WMI Control->right click select Properties->Security
Select CIMV2 under root

Select security button add new user with
Enable Account
Remote Enable

Allowing WMI through the Windows Firewall
Allow pre-defined rule: Windows Management Instrumentation (WMI)

Deny ssh interactive login but allow sftp

SSH interactive login need tty to be allocated but sftp/scp doesn't need tty. So you can disable SSH interactive login by no-pty option in OPENSSH. But no-pty option is valid only in public key authentication, so you have to disable password for the user with “passwd –l username” command.

I have attempted to use pam_listfile.so tty option to achieve this, I found it is impossible because pam_tty name ssh will be allocated in either ssh login or sftp.

All you need to is to put no-pty parameter in ~/.ssh/authorized_keys, it must be in the same line with the public key, multiple options are separated by comma e.g


no-pty,no-X11-forwarding ssh-dss AAAAB3Nz ... key-comment

Another useful feature of public key authentication is forced command, which means the command is invoked whenever the key is authenticated, it is great security feature for remote execution e.g backup job. you can also limit client source with "from= " option.


#Force to run command date only
$ cat /home/test/.ssh/authorized_keys
command="date" ssh-dss AAAAB3NzaC1kc3 ..

#date command was executed even given command is ls
$ ssh test@localhost ls
Wed Oct 14 10:29:39 EST 2009

#forced command can literally disable SSH interactive login.
$ ssh test@localhost
Wed Oct 14 10:29:44 EST 2009
Connection to localhost closed.

Tuesday, September 1, 2009

Command to get system hardware serial number.

SMBIOS/DMI standard includes system manufacturer, model name, serial number, BIOS version, asset tag as well as a lot of other details of varying level of interest and reliability depending on the manufacturer. This will often include usage status for the CPU sockets, expansion slots (e.g. AGP, PCI, ISA) and memory module slots, and the list of I/O ports (e.g. serial, parallel, USB).
Solaris X86:

$smbios
..
Manufacturer: HP
Product: ProLiant DL360 G3
Serial Number: B038XXXXX
..

Solaris Sparc:
smbios is not supported in SPARC yet, the traditional command prtdiag works for both X86 and SPARC ,but it reports less detailed hardware information.

Linux/BSD:
$dmidecode

Create restricted login account

Create a login in restricted shell and doesn’t allow user to change password.

rsh is a limiting version of the standard command inter-
preter sh, used to restrict logins to execution environments
whose capabilities are more controlled than those of sh (see
sh(1) for complete description and usage).

The actions of rsh are identical to those of sh, except that
the following are disallowed:

changing directory (see cd(1)),

setting the value of $PATH,

pecifying path or command names containing /,

redirecting output (> and >>).

Set restricted shell as login shell

Solaris:
usermod -s /usr/lib/rsh userid
Linux:
usermod -s /usr/bin/rbash userid

Set minimum number of days between password changes to large number, so user can’t change password until min days

Solaris:
passwd -n 9999 –x 9998 userid
(Solaris needs to set both Min Max days and Min is greater than Max)
Linux:
passwd –n 9999 userid

Thursday, August 20, 2009

Align partitions on the stripe Boundary for Linux and Windows to boost performance

Aligning partitions on the stripe Boundary can boost IO performance up to 20% depending on file system block size , stripe size and intensity of IO workload etc. Disk alignment issue exists for environment, in which all following factors are met

Disk: Hardware Raid(Including SAN)
Server: X86 32bit or 64bit PC server .
OS: Linux or Windows ( BSD, Solaris not investigated)

Terms:
- Sector Size: Normally 512 byte as industry standard to lower-format a single harddisk.
- Stripe Size: The smallest unit used by SAN, Hardware Raid and software Raid starting from 2 KB, in power of 2. but 32,64,128 is common stripe size
- Block Size: The smallest amount of disk space which can be allocated to hold a file for file system, ext3,NTFS is 4k by default

Issue:
Due to x86 architectures BIOS limitation, the first partition starts at 63 sector by default in Windows or Linux.
As a result, The partition doesn't align the Stripe Boundary, so there are chances that one FS block sits above 2 stripes, so one request involves 2 physical IOs. The chances can be calculated as (FS block size/ stripe size).so It is 100% for 4k FS block on 4K stripe size.

The offset should be multiple of stripe size, if you are not sure the stripe size, start at 1M should be safe.
Take 64K stripe size for example:
((Partition offset) * (Disk sector size)) / (Stripe unit size)

(63 * 512) / 65536=0.4921875
(128* 512) / 65536=1
So the partition should start 128 sector (65536 bytes) at least

Verify:
- Linux: fdisk -lu
$ fdisk -lu
Device Boot Start End Blocks Id System
/dev/sda1 63 37736684 18868311 83 Linux
- Windows:
Any version up to Windows 2003 are affected by default, Windows 2008 has fixed the issue

Fix:
- Linux:
Fdisk go to expert mode by type x then select b to adjust starting block
- Windows:
diskpart detailed in the Windows KB

Reference:
http://support.microsoft.com/kb/929491
http://www.vmware.com/pdf/esx3_partition_align.pdf
http://now.netapp.com/Knowledgebase/solutionarea.asp?id=kb8190