Less known Linux commands.

Completing a task with the simplest command.

#ac prints out a report of connect time (in hours) based on the logins/logouts in the current wtmp file. A total is also printed out

$ac -p
root                                532.96
admin                               63.53
total     536.96

# mcookie generates a 128-bit random hexadecimal number
#The "random" number generated is actually the output of the MD5 message digest fed with various piece of random information: the current time, the process id, the parent process id ..

$mcookie
9612f5db846dd4d0655572b33ea1cb6f

#pidof -- find the process ID of a running program.
##match program name only, it can't match argument of the program

$ pidof /sbin/mingetty
3773 3770 3769 3768 3767 3766

# procinfo :display memory, disk, IRQ & DMA module and device info #$procinfo -D refresh values every few seconds just like top.

$procinfo
Memory:      Total        Used        Free      Shared     Buffers
Mem:       3897500      684240     3213260           0      269308
Swap:       779112           0      779112

Bootup: Wed Feb 24 00:20:02 2010    Load average: 0.11 0.07 0.02 3/223 5641

user  :       0:54:16.96   0.2%  page in :     103469  disk 1:    10272r 1857163w
nice  :       0:00:28.10   0.0%  page out:   12920948
system:       3:13:02.34   0.6%  page act:     126534
IOwait:       0:11:44.80   0.0%  page dea:          0
hw irq:       0:00:08.20   0.0%  page flt:   84232828
sw irq:       1:30:52.17   0.3%  swap in :          0
idle  :  22d  9:49:49.93  98.9%  swap out:          0
uptime:  22d 15:38:40.56         context : 1476505392

irq  0: 489275054 timer                 irq  9:         0 acpi         
irq  1:         8 i8042                 irq 12:       105 i8042        
irq  3:         1                       irq 14:  17590508 ide0         
irq  4:         1                       irq169:   7179622 ioc0, vmxnet ether
irq  6:         5                       irq177:   3543610 vmxnet ether 
irq  7:         0 parport0              irq185:     42843 vmxnet ether 
irq  8:         2 rtc                   irq193:  15376791 vmxnet ether 

#readlink - display destination of a symbolic link

$ls -l test2
lrwxrwxrwx 1 root wheel 5 2010-03-19 10:09 test2 -> test1
$readlink test2
test1

#Who links me? find source softlink with "lname" argument

$ touch /var/tmp/test1
$ ln -s /var/tmp/test1 /tmp/test1
$ find /tmp -lname /var/tmp/test1 -ls
11    0 lrwxrwxrwx   1 root wheel          14 Jun 30 11:07 /tmp/test1 -> /var/tmp/test1

#Who links me? find source hardlink with "samefile" argument
#hardlinks have same inode number(inode=13316 in this example), so it can also be achived with inum argument as well.

$touch /tmp/test1/test-hl
$ln  /tmp/test1/test-hl /tmp/test-hl
$find /tmp/ -samefile /tmp/test1/test-hl -exec ls -il {} \;
13316 -rw-r--r-- 2 root  wheel 0 2010-06-30 11:15 /tmp/test1/test-hl
13316 -rw-r--r-- 2 root wheel 0 2010-06-30 11:15 /tmp/test-hl

# seq - print a sequence of numbers
# default increment is 1
# It was used in "for loop", but now brace expansion {1..3} is better choice.

$seq
1
2
3
$seq -s " "  3
1 2 3

#convert vertical lines to horizontal lines

$ seq 4 | paste -s
1       2       3       4

$ seq 4 | xargs -n 2
1 2
3 4

#convert horizontal lines to vertical lines

$echo "123456"| fold -w 2
12
34
56

#stat - display file or file system status
##Links:2 means the file has two copies(hard link files)

$stat test1
File: `test1'
Size: 3               Blocks: 8          IO Block: 4096   regular file
Device: fd04h/64772d    Inode: 767061      Links: 2
Access: (0644/-rw-r--r--)  Uid: (    0/    root)   Gid: (    0/    root)
Access: 2010-03-18 16:21:01.000000000 +1100
Modify: 2010-03-18 16:21:00.000000000 +1100
Change: 2010-03-18 16:21:00.000000000 +1100

##display parent file system info.It can be used to find parent partition by matching inodes Total/Free of df -i

$stat -f test1
File: "test1"
ID: 0        Namelen: 255     Type: ext2/ext3
Block size: 4096       Fundamental block size: 4096
Blocks: Total: 2633708    Free: 2371194    Available: 2237409
Inodes: Total: 1338240    Free: 1335916

#tac - concatenate and print files in reverse

$cat t1
12
$ tac t1
21

#usleep - sleep for the specified number of microseconds

usleep 300

#watch - execute a program periodically, showing output fullscreen
#monitor eth0 output, refresh every 2 seconds

watch -n 2 ifconfig eth0

#Zdump prints the current time in each zone name named on the command line.

$date
Thu Mar 18 05:51:55 GMT 2010
$zdump GMT
GMT  Thu Mar 18 05:51:57 2010 GMT
(alternative command: $export TZ=GMT;date)

#zgrep - search possibly compressed files (.Z,gz,bz2)for a regular expression
#other z commands zcat zmore ...

$zgrep .*  test1.gz
test1.gz:test message

#shell tips
# - is equivalent to $OLDPWD;When a is used as the operand,this shall be equivalent to the command: cd "$OLDPWD" && pwd ; (reference man ksh or man bash)

$cd /tmp
$cd /var/tmp
$cd -
/tmp

# "$_" .. expands to to the last argument to the previous command.. (reference Special Parameters in man ksh/bash)
# ( bash only "!$" "!!:$" designates the last argument of the preceding command. This may be shortened to "!$")

$ ping -c 1 172.16.1.1
PING 172.16.1.1 (172.16.1.1) 56(84) bytes of data.
64 bytes from 172.16.1.1: icmp_seq=1 ttl=64 time=0.396 ms
$ telnet $_ 80
Trying 172.16.1.1...
Connected to 172.16.1.1.
Escape character is '^]'.
^]
telnet> quit
Connection closed.

Regular expression in condition statement, the ksh and bash examples

Ksh has better regular expression support than bash
bash has = to support Basic regular expressions and its result is exact match
bash has =~ to support Extended regular expressions ( Metacharacter ? + and character classes like [:alnum:] ) and its result is partial match

The followings use "if statement" as examples

##Example 1 :Match string host1 or host2

#!/bin/ksh
if [[ "$1" = host[12] ]];then echo Match; fi

#!/bin/bash 

if [[ "$1" = host[12] ]]; then echo match ;fi

#!/bin/bash

#Note the ^ $ due to  partial match nature of =~
if [[ "$1" =~ ^host[12]$ ]]; then echo match ;fi

##Example 2 :Match string linux or solaris

#!/bin/ksh
if [[ "$1" = @(linux|solaris) ]];then echo Match; fi

# bash's = doesn’t support |, You have to use the traditional way to join with –o
#!/bin/bash
if [[ "$1" =~ "^linux$|^solaris$" ]]; then echo match ;fi

##Example 3 :Match any three digits

#!/bin/ksh
if [[  "$1" = {3}(\d)  ]];then echo Match; fi

#!/bin/bash 

if [[ "$1" = [0-9][0-9][0-9] ]]; then echo match ;fi

#!/bin/bash
if [[ "$1" =~ "^[0-9]{3}$" ]]; then echo match ;fi

#Example 4 : Match liNux or soLaRis case insensitive match

#!/bin/ksh
if [[  "$1" = @(~(i:linux|solaris))  ]];then echo Match; fi

#!/bin/bash
shopt -s nocasematch
if [[ "$1" =~ "^solaris$|^linux$" ]]; then echo match ;fi

Finally, bash support similar syntax to ksh with extglob option enabled.

#!/bin/bash
shopt -s extglob
shopt -s nocasematch
if [[ "$1" = @(solaris|linux) ]]; then echo match ;fi

#reference
man ksh

A pattern-list is a list of one or more patterns separated from each other with a & or â. A & signifies that all patterns must be matched whereas
â requires that only one pattern be matched. Composite patterns can be formed with one or more of the following sub-patterns:
?(pattern-list)
Optionally matches any one of the given patterns.
*(pattern-list)
Matches zero or more occurrences of the given patterns.
+(pattern-list)
Matches one or more occurrences of the given patterns.
{n}(pattern-list)
Matches n occurrences of the given patterns.
{m,n}(pattern-list)
Matches from m to n occurrences of the given patterns. If m is omitted, 0 will be used. If n is omitted at least m occurrences
will be matched.
@(pattern-list)
Matches exactly one of the given patterns.
!(pattern-list)
Matches anything except one of the given patterns.

By default, each pattern, or sub-pattern will match the longest string possible consistent with generating the longest overall match. If more
than one match is possible, the one starting closest to the beginning of the string will be chosen. However, for each of the above compound patâ
terns a - can be inserted in front of the ( to cause the shortest match to the specified pattern-list to be used.
When pattern-list is contained within parenthesis, the backslash character \ is treated specially even when inside a character class. All ANSI-C
character escapes are recognized and match the specified character. In addition the following escape sequences are recognized:
\d Matches any charcter in the digit class.
\D Matches any charcter not in the digit class.
\s Matches any charcter in the space class.
\S Matches any charcter not in the space class.
\w Matches any charcter in the word class.
\W Matches any charcter not in the word class.
man bash
If the extglob shell option is enabled using the shopt builtin, several extended pattern matching operators are recognized. In the following
description, a pattern-list is a list of one or more patterns separated by a . Composite patterns may be formed using one or more of the followâ
ing sub-patterns:
?(pattern-list)
Matches zero or one occurrence of the given patterns
*(pattern-list)
Matches zero or more occurrences of the given patterns
+(pattern-list)
Matches one or more occurrences of the given patterns
@(pattern-list)
Matches one of the given patterns
!(pattern-list)
Matches anything except one of the given patterns

Change Solaris user password with Expect script

Changing Linux user password  without user keyboard input  is very easy by  builtin command "chpasswd "or "passwd --stdin".
Solaris doesn't have such commands , the result can be achieved  by Expect script.

#Expect script
$cat chpwd.sh

#!/usr/local/bin/expect --
# Input: username password
set USER [lindex $argv 0]
set PASS [lindex $argv 1] 

if { $USER == "" || $PASS == "" }  {
   puts "Usage:  ./scriptname username password\n"
   exit 1
 }

spawn sudo passwd $USER 
expect "assword:"
send "$PASS\r"
expect "assword:"
send "$PASS\r"
expect eof

#Shell script: Generate a random password and call the Expect script to set it


$cat createpwd.sh

#!/bin/ksh
# Generate a random password  for a user and set the new password 
USER=$1
PASS=`tr -dc [:alnum:] </dev/urandom | fold -8 | head -1`
if [ -z "$USER" ]; then
echo "Usage $0 username"
exit
fi
./chpwd.sh $USER $PASS 
echo "username=$USER"
  
echo "password=$PASS"

Linux memory management study notes

32-bit architectures can reference 4 GB (2^32) of physical memory.
 #Virtual Memory: User-space virtual space resides lower 3G, Kernel virtual space resides upper 1G
 #Physical Memory: Three zones:
ZONE_DMA=0-16M, ZONE_NORMAL=16-896M  (896M-1024M kernel reserved ) and   ZONE_HIGHMEM=>1024M
32-bit architectures
Kernel has page tables  to map "virtual addresses" to "physical addresses"
The kernel virtual area  is mapped from  HIGH  1GB ( 3GB-4GB)  virtual space to LOW ( 1 GB)  physical RAM.
- RAM size is less than 896 MB
Liner mapping is possible from 1GB kernel address to 1GB  of physical RAM,which are ZONE_DMA and ZONE_NORMAL (not including 128M reserved space).
Kernel page tables must transform linear addresses starting from 0xc0000000 (3GB) into physical addresses starting from 0.
- RAM size is between 896 MB and 4096 MB
Dynamic remapping is done in the 128M reserved space, because ZONE_HIGHMEM zone includes page frames that cannot be directly accessed by the kernel through the linear mapping.
- RAM size is more than 4096 MB
dynamic rempapping with three-level paging model.
(With PAE capable hardware and hugemem Linux kernel , 32bit Linux can support up to 64G memory)
64-bit architectures
ZONE_HIGHMEM is empty, all are ZONE_NORMAL, no remapping needed.
#/proc/meminfo displays physical memory info
#32bit Kernel has both low and high memory

[ 32bit Kernel]$ cat /proc/meminfo 
MemTotal:      3897500 kB
MemFree:       3280456 kB
..
HighTotal:     3014592 kB
HighFree:      2685548 kB
LowTotal:       882908 kB
LowFree:        594908 kB

#64 bit kernel ZONE_NORMAL is huge, all fits in Lowtotal, so HighTotal is zero

[64bit Kernel]$ cat /proc/meminfo  
MemTotal:     37025752 kB
MemFree:      17509720 kB
..
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:     37025752 kB
LowFree:      17423152 kB
..

#From kernel document
HighTotal:

HighFree: Highmem is all memory above ~860MB of physical memory, Highmem areas are for use by userspace programs, or for the pagecache.  The kernel must use tricks to access this memory, making it slower to access than lowmem.

LowTotal:
LowFree: Lowmem is memory which can be used for everything that highmem can be used for, but it is also available for the kernel's use for its own data structures.  Among many other things, it is where everything from the Slab is allocated.  Bad things happen when you're out of lowmem.
#Links
BOOK:Understanding the Linux Kernel By Daniel Pierre Bovet, Marco Cesatí
http://books.google.com.au/books?id=h0lltXyJ8aIC&dq=Understanding+the+Linux+Kernel+By+Daniel+Pierre+Bovet,+Marco+Cesat%C3%AD&source=gbs_navlinks_s
High Memory In The Linux Kernel
http://kerneltrap.org/node/2450
Kernel document
http://www.mjmwired.net/kernel/Documentation/filesystems/proc.txt