Aligning partitions on the stripe Boundary can boost IO performance up to 20% depending on file system block size , stripe size and intensity of IO workload etc. Disk alignment issue exists for environment, in which all following factors are met
Disk: Hardware Raid(Including SAN)
Server: X86 32bit or 64bit PC server .
OS: Linux or Windows ( BSD, Solaris not investigated)
Terms:
- Sector Size: Normally 512 byte as industry standard to lower-format a single harddisk.
- Stripe Size: The smallest unit used by SAN, Hardware Raid and software Raid starting from 2 KB, in power of 2. but 32,64,128 is common stripe size
- Block Size: The smallest amount of disk space which can be allocated to hold a file for file system, ext3,NTFS is 4k by default
Issue:
Due to x86 architectures BIOS limitation, the first partition starts at 63 sector by default in Windows or Linux.
As a result, The partition doesn't align the Stripe Boundary, so there are chances that one FS block sits above 2 stripes, so one request involves 2 physical IOs. The chances can be calculated as (FS block size/ stripe size).so It is 100% for 4k FS block on 4K stripe size.
The offset should be multiple of stripe size, if you are not sure the stripe size, start at 1M should be safe.
Take 64K stripe size for example:
((Partition offset) * (Disk sector size)) / (Stripe unit size)
(63 * 512) / 65536=0.4921875
(128* 512) / 65536=1
So the partition should start 128 sector (65536 bytes) at least
Verify:
- Linux: fdisk -lu
$ fdisk -lu
Device Boot Start End Blocks Id System
/dev/sda1 63 37736684 18868311 83 Linux
- Windows:
Any version up to Windows 2003 are affected by default, Windows 2008 has fixed the issue
Fix:
- Linux:
Fdisk go to expert mode by type x then select b to adjust starting block
- Windows:
diskpart detailed in the Windows KB
Reference:
http://support.microsoft.com/kb/929491
http://www.vmware.com/pdf/esx3_partition_align.pdf
http://now.netapp.com/Knowledgebase/solutionarea.asp?id=kb8190
Thursday, August 20, 2009
Friday, July 24, 2009
Generate random password with shell script.
Generate random passwords
Generate random passwords consist of letters, numbers and any special characters.
Pick the appropriate password
The above one liner is fine for general purpose, but with password policy, you have to choose one adheres to the password policy
The following script will pick the right password in form of at least 1 upper case, 1 lower case and 1 digit
$ ./genpwd.sh
Password Generated : 8sZrR1az
Generate random passwords consist of letters, numbers and any special characters.
$ tr -cd \#_[:alnum:] < /dev/urandom | fold -w 8 | head -5 shu6_BEN BIL2b#0w xgHWgOJ2 E1NmtxX1 WqcTAsRN or $openssl passwd "$RANDOM" | cut -c1-8 cH75S94U
Pick the appropriate password
The above one liner is fine for general purpose, but with password policy, you have to choose one adheres to the password policy
The following script will pick the right password in form of at least 1 upper case, 1 lower case and 1 digit
#!/bin/bash
# Generate random password adhere to password policy
# caveat: if you need more strict policy e.g. 2 upper cases,2 lower cases, 2 digit, adjust the number retuned by head
LENGTH=8
MIN_U=1
MIN_L=1
MIN_D=1
for i in $(tr -dc [:alnum:] </dev/urandom | fold -w $LENGTH | head -20)
do
UPPERS=$(echo $i | $AWK '{print gsub(/[A-Z]/,"")}')
LOWERS=$(echo $i | $AWK '{print gsub(/[a-z]/,"")}')
DIGITS=$(echo $i | $AWK '{print gsub(/[0-9]/,"")}')
if [ $UPPERS -ge $MIN_U -a $LOWERS -ge $MIN_L -a $DIGITS -ge $MIN_D ];then
FOUND=1; break
fi
done
if [ -z $FOUND ];then
echo "ERROR: could not generate appropriate password"
else
echo "Password Generated :" $i
fi
$ ./genpwd.sh
Password Generated : 8sZrR1az
Thursday, June 25, 2009
few scripting tips.
How to get the last digits of a string e.g print 201 for string ua07app201?
#sed back reference: print the first match pattern enclosed by ( )
echo ua07app201 | sed 's/.*[^0-9]\([0-9]*\)$/\1/'
#Sed delete: Delete the longest match of non-digits char
echo 'ua07app201' | sed 's/.*[^0-9]//'
#Expr matching operator : Similar to sed back reference, without (), it returns the the number of matched chars.
expr ua07app201 : '.*[^0-9]\([0-9]*\)$'
#Awk: set non-digit as seperator, print the last filed $NF
echo 'ua07app201' | nawk -F'[^0-9]' '{print $NF}'
#Perl in command line mode
echo ua07app201 | perl -nle ' $_ =~m /(\d+$)/; print $1'
or the simplizied version
echo ua07app201|perl -nle'print/(\d+)$/'
#Parameter Substitution, delete the longest match of non-digits chars from beginning.
a='ua07app201';echo "${a##*[!0-9]}"
How to get path only from full path of a file?
#Parameter Substitution, delete the shortest match from end
$ var=/var/tmp/test.txt;echo ${var%/*}
/var/tmp
How to sort a string?
#one-liner to sort a string
$echo "s03 s08 s01" | tr '[:space:]' '\n' | sort -n | paste -s
s01 s03 s08
Monday, June 1, 2009
ctrl+A and ctrl+E key shortcuts can’t move cursor in bash.
ctrl+A and ctrl+E are mostly used key shortcuts to move cursor the start/end of the line, if they stop working, you need to check the command line editor option in bash. There are two command line editors for bash: emacs or vi. The default editor is emacs, that is why ctrl+A and ctrl+E works by default. Don't regard ctrl+A ctrl+E as built-in feature of bash, it can be disabled by changing editor to vi.
set -o #list current value of the options
emacs off
vi on
set -o vi #enable vi as editor, it will disable emacs automatically like set +o emacs does
set -o emacs #enable emacs as editor, it will disable vi automatically like set +o vi does
Command Editing -- Cursor Movement in command mode for vi
l (el) Move the cursor forward one character
h Move the cursor back one character
w Move the cursor forward one word
b Move the cursor back one word
fc Find character c in the line
0 (zero) Cursor to the start of the line
$ Cursor to the end of the line
l, h, w, f and b can be preceded by a number, thus 2b moves the cursor back two words, and 3fx finds the third occurrence of x in the line. In emacs and gmacs mode, cursor movement is different.
Command Editing -- Cursor Movement in emacs/gmacs
CONTROL-F Move the cursor forward one character
CONTROL-B Move the cursor back one character
ESCAPE then f Move the cursor forward one word
ESCAPE then b Move the cursor back one word
CONTROL-A Cursor to the start of the line
CONTROL-E Cursor to the end of the line
http://www.gnu.org/software/bash/manual/bashref.html#Command-Line-Editing
Lastly, set -o options work for ksh as well, but csh/sh doesn't support it.
Thursday, May 28, 2009
Dtrace Basics
- D Program Structure
probe descriptions
/ predicate /
{
action statements
}
-- Probe Descriptions
provider : subsystem : function : probeName
syscall::*lwp*:entry, syscall::*sock*:entry #support wildcards, one or more fields of the probe description are omitted(means any value)
--- list available provider moduel function name
$dtrace -l more
ID PROVIDER MODULE FUNCTION NAME
1 dtrace BEGIN
2 dtrace END
3 dtrace ERROR
4 nfsmapid209 nfsmapid check_domain daemon-domain
5 nfsmapid209 nfsmapid resolv_query_thread thread-domain
6 syscall nosys entry
-- Predicates
Predicates are expressions enclosed in slashes / / that are evaluated at probe firing time to determine whether the associated actions should be executed.
D language doesn't has control-flow constructs such as if-statements and loops. it use Predicates
-- Actions
Probe actions are described by a list of statements separated by semicolons (;) and enclosed in braces { }.If no additional action need, an empty set of braces with no statements inside
-- example
$dtrace -n syscall::read:entry #
-n means to match probename from command line, -m=match module name
$dtrace -s counter.d #- s read input from script
$vi counter.d
dtrace:::BEGIN
{
i = 10;
}
profile:::tick-1sec
/i > 0/
{
trace(i--);
}
profile:::tick-1sec
/i == 0/
{
trace("blastoff!");
exit(0);
}
dtrace/profile are providers
tick-5sec #tick-xsec is the function name of provider profile, like sleep in shell
trace (100) #print out a value or string(needs enclosed by " ",trace ("hello")), like echo in shell
printf ("%s","hello") # print out in particular format
/* ... */ #comment lines
if no
dtrace:::END statement, you need to press ctrl+c to see to the result.
--Use built-in variables for predicates
syscall::read:entry,
syscall::write:entry
/pid == 12345/
{
}
execname: Name of the current process's executable file
pid:Process ID of the current process
tid: Thread ID of the current thread
...
- Aggregations
DTrace stores the results of aggregating functions in objects called aggregations. The aggregation results are indexed using a tuple of expressions similar to those used for associative arrays. In D, the syntax for an aggregation is
@name[ keys ] = aggfunc ( args );
Aggregations is used for result data,the entire data set need not be stored
Arrgegations are printed out by default no need to print statement
-- Example
# Syscall count by process,
dtrace -n 'syscall:::entry { @num[pid,execname] = count(); }'
-- DTrace Aggregating Functions
count: The number of times called.
sum: The total value of the specified expressions.
.....
- Structs
If you have programmed in the Java programming language, think of aDstruct as a class, but one with data members only and no methods.
struct callinfo {
uint64_t ts; /* timestamp of last syscall entry */
uint64_t elapsed; /* total elapsed time in nanoseconds */
uint64_t calls; /* number of calls made */
size_t maxbytes; /* maximum byte count argument */
};
You can use the operator -> to access struct members through a pointer. callinfo->ts
- Further reading..
Solaris Dynamic Tracing Guide - Official Dtrace guide
DTrace Tools - collection of useful scripts
probe descriptions
/ predicate /
{
action statements
}
-- Probe Descriptions
provider : subsystem : function : probeName
syscall::*lwp*:entry, syscall::*sock*:entry #support wildcards, one or more fields of the probe description are omitted(means any value)
--- list available provider moduel function name
$dtrace -l more
ID PROVIDER MODULE FUNCTION NAME
1 dtrace BEGIN
2 dtrace END
3 dtrace ERROR
4 nfsmapid209 nfsmapid check_domain daemon-domain
5 nfsmapid209 nfsmapid resolv_query_thread thread-domain
6 syscall nosys entry
-- Predicates
Predicates are expressions enclosed in slashes / / that are evaluated at probe firing time to determine whether the associated actions should be executed.
D language doesn't has control-flow constructs such as if-statements and loops. it use Predicates
-- Actions
Probe actions are described by a list of statements separated by semicolons (;) and enclosed in braces { }.If no additional action need, an empty set of braces with no statements inside
-- example
$dtrace -n syscall::read:entry #
-n means to match probename from command line, -m=match module name
$dtrace -s counter.d #- s read input from script
$vi counter.d
dtrace:::BEGIN
{
i = 10;
}
profile:::tick-1sec
/i > 0/
{
trace(i--);
}
profile:::tick-1sec
/i == 0/
{
trace("blastoff!");
exit(0);
}
dtrace/profile are providers
tick-5sec #tick-xsec is the function name of provider profile, like sleep in shell
trace (100) #print out a value or string(needs enclosed by " ",trace ("hello")), like echo in shell
printf ("%s","hello") # print out in particular format
/* ... */ #comment lines
if no
dtrace:::END statement, you need to press ctrl+c to see to the result.
--Use built-in variables for predicates
syscall::read:entry,
syscall::write:entry
/pid == 12345/
{
}
execname: Name of the current process's executable file
pid:Process ID of the current process
tid: Thread ID of the current thread
...
- Aggregations
DTrace stores the results of aggregating functions in objects called aggregations. The aggregation results are indexed using a tuple of expressions similar to those used for associative arrays. In D, the syntax for an aggregation is
@name[ keys ] = aggfunc ( args );
Aggregations is used for result data,the entire data set need not be stored
Arrgegations are printed out by default no need to print statement
-- Example
# Syscall count by process,
dtrace -n 'syscall:::entry { @num[pid,execname] = count(); }'
-- DTrace Aggregating Functions
count: The number of times called.
sum: The total value of the specified expressions.
.....
- Structs
If you have programmed in the Java programming language, think of aDstruct as a class, but one with data members only and no methods.
struct callinfo {
uint64_t ts; /* timestamp of last syscall entry */
uint64_t elapsed; /* total elapsed time in nanoseconds */
uint64_t calls; /* number of calls made */
size_t maxbytes; /* maximum byte count argument */
};
You can use the operator -> to access struct members through a pointer. callinfo->ts
- Further reading..
Solaris Dynamic Tracing Guide - Official Dtrace guide
DTrace Tools - collection of useful scripts
Difference between Red Hat Linux and SUSE Linux
Red hat Linux and SUSE linux don't have too much difference, the universal management tool: YaST makes it easier for Red hat Linux users to grasp SUSE Linux: The following is important difference between Red Hat Linux and SUSE Linux in sysadmin perspective (based on SUSE Linux Enterprise Server 10)
- Management tool
YaST is the central system management tool for SUSE Linux.
YaST is collection of modules,e.g "yast users " will go to user management module directly, yast -l to list all modules.
-- launch YaST:
yast: Start in text mode
yast2:Start in text mode/gui mode depending on the terminal capability
-Installation automation
Red Hat Linux: kickstart SUSE linux: AutoYast
- Boot
-- boot files:
/etc/init.d/boot #master boot control file called by init process
/etc/init.d/boot calls various scripts in /etc/init.d/boot.d/* to load modules/mount FS/activate network etc.
Files in /etc/init.d/boot.d/Sxxboot.* are sym link to /etc/init.d/boot.*, e.g. (/etc/init.d/boot.d/S08boot.localfs -> ../boot.localfs).
They are executed in ascending order.
Finally, /etc/init.d/boot call /etc/init.d/boot.local (like /etc/rc.local in RedHat Linux for user defined startup command )
-- Service control
Both can use chkconfig tool to control service, in addition. SuSE linux has rcSVCNAME command linked to the start script to control service directly . e.g
#file /usr/sbin/rcsshd
/usr/sbin/rcsshd: symbolic link to `/etc/init.d/sshd'
- User
New user added to users group by default, RedHat Linux added to new private group
For bash, local profile use ~/.profile, RedHat Linux use ~/.bash_profile
- Network
/etc/sysconfig/network/* #all network related config files
/etc/sysconfig/network-scripts/ifcfg-eth-id-MA-CA-DD-RE-SS #interface config files
/etc/HOSTNAME #set hostname,the file name is upper case.
/etc/sysconfig/network/routes #static routes
- Security
Red Hat Linux: SElinux
SuSE Linux: AppArmor
Both are based on Linux Security Modules (LSM) to overcome the limitation of traditional Discretionary Access Control (DAC), but the configuration details are totally different. The default configuration may cause weird issue, AppArmor can be turned with script /etc/init.d/boot.apparmor
Detailed document for Migrating from RedHat to SUSE Linux Enterprise Server 10
- Management tool
YaST is the central system management tool for SUSE Linux.
YaST is collection of modules,e.g "yast users " will go to user management module directly, yast -l to list all modules.
-- launch YaST:
yast: Start in text mode
yast2:Start in text mode/gui mode depending on the terminal capability
-Installation automation
Red Hat Linux: kickstart SUSE linux: AutoYast
- Boot
-- boot files:
/etc/init.d/boot #master boot control file called by init process
/etc/init.d/boot calls various scripts in /etc/init.d/boot.d/* to load modules/mount FS/activate network etc.
Files in /etc/init.d/boot.d/Sxxboot.* are sym link to /etc/init.d/boot.*, e.g. (/etc/init.d/boot.d/S08boot.localfs -> ../boot.localfs).
They are executed in ascending order.
Finally, /etc/init.d/boot call /etc/init.d/boot.local (like /etc/rc.local in RedHat Linux for user defined startup command )
-- Service control
Both can use chkconfig tool to control service, in addition. SuSE linux has rcSVCNAME command linked to the start script to control service directly . e.g
#file /usr/sbin/rcsshd
/usr/sbin/rcsshd: symbolic link to `/etc/init.d/sshd'
- User
New user added to users group by default, RedHat Linux added to new private group
For bash, local profile use ~/.profile, RedHat Linux use ~/.bash_profile
- Network
/etc/sysconfig/network/* #all network related config files
/etc/sysconfig/network-scripts/ifcfg-eth-id-MA-CA-DD-RE-SS #interface config files
/etc/HOSTNAME #set hostname,the file name is upper case.
/etc/sysconfig/network/routes #static routes
- Security
Red Hat Linux: SElinux
SuSE Linux: AppArmor
Both are based on Linux Security Modules (LSM) to overcome the limitation of traditional Discretionary Access Control (DAC), but the configuration details are totally different. The default configuration may cause weird issue, AppArmor can be turned with script /etc/init.d/boot.apparmor
Detailed document for Migrating from RedHat to SUSE Linux Enterprise Server 10
Tuesday, May 19, 2009
Netapp Notes
#== Basics
- Introduction
Data ONTAP is the name of Netapp's Platform OS, it is based on BSD.
Netapp appliance is based on i86 PC hardware(mostly AMD Opteron nowadays).
NetApp appliance is unified storage supports file-based protocols such as NFS, CIFS, FTP, TFTP, and HTTP, block-based protocols such as FC and iSCSI.
NetApp V-Series can attach and manage third party storage systems.
NVRAM: log transitions, it can replay the log in the event of unplanned shutdown.
RAM: system memory, data read/write cache
Flash card(not all models): system initial boot media
FC Controller port 0a/0b/0c/0d ....
Jumper to setup Disk shelf ID 1 - 7
Disk numbering from right to left 16 ......
FC:
WWNN - world wide name
is a World Wide Name assigned to a port in a Fibre Channel fabric, it performs a function equivalent to the MAC address in Ethernet protocol
WWPN - world wilde port name
is valid for the same WWNN to be seen on many different ports (different addresses)
ISCSI:
Initiator= client, target = server
Software initiator with standard NIC
TCP offload Engine TOE with soft initiator – offload computing from CPU
ISCSI HBA -Hardware initiator and provide diskless boot
No need to use ISCSI HBA, if diskless boot feature is not needed and CPU resource is plenty
Netapp use software ISCSI if no ISCSI HBA present.
# setup iscsi on linux
[root@linux /]# iscsi-iname
iqn.1987-05.com.cisco:01.8494d06dc35d
The ID generated is random, write it to:/etc/initiatorname.iscsi to be persistent
- help
help # list all cmds
cifs help shares #display help for sub command
man cmd
priv set advanced # to use advanced command e.g ls
- account managment
useradmin group list
useradmin user add admin -g Administrators
- Access netapp
http://IP/na_admin # web gui URL.
ssh IP # ssh (or rsh, telnet in) for CLI access, enable ssh with secureadmin cmd
rsh IP #excute cmd remotely from admin server, the admin server should be added to /etc/hosts.equiv
get root mount of /vol/vol0/ in a unix machine to edit config files
- Read/write file
rdfile /etc/exports
wrfile -a /etc/hosts 1.1.1.1 filer1 #append to file
wrfile /etc/hosts.equiv #Rewrite file, type in lines, then ctr+C
- Error messages
rdfile /etc/messages
- backup config file
config dump config.bak
it is saved to /etc/configs/config.bak
- server setting
options command control server setting
options ftpd.enable on #enable ftp server for example
options nfs.export.auto-update off #turn off auto export, otherwise the new volume will be exported automatically
all options are saved to /etc/registry
- system stats
stats show
sysstat -su 1
- copy entire volume
vol copy
ndmpd on
ndmpcopy -f /vol/vol0 /vol/vol0_trad
- Cluster
Netapp cluster doesn't do I/O load balancing, it is just for fail-over purpose.You need to allocate disk in each node for different services
#= =Boot
Ctrl+C to go to boot menu.
there is option to reset password
type in "22/7" to show secret boot menu
#= =Storage
disk zero spare # zero all spare disk so they can be added quickly to a volume
Data ONTAP supports 100 aggregates (including traditional volumes) on a single storage system.
Data ONTAP supports 500 volumes per head(FAS2020 and FAS200 series, the limit is 200 FlexVol volumes.), So in cluster enviroment, the combined volumes number in both nodes should not exceed the limit for the sake of failover.
Netapp wafl(Write Anywhere File layout) block size = 4 KB.
Netapp support Raid0, Raid4, RaidDP (double-parity), Raid 1 (via snapmirror)
- Traditional Volume
It is tightly coupled swith its containing aggregate. No other volumes can get their storage from this containing aggregate. It can'b be shrinked. can be expanded by adding more disks
- FlexVol volumes
A FlexVol volume is a volume that is loosely coupled to its containing aggregate. A FlexVol volume
can share its containing aggregate with other FlexVol volumes,Thus, a single aggregate can be the
shared source of all the storage used by all the FlexVol volumes contained by that aggregate.
- flex clone
FlexClone volumes always exist in the same aggregate as their parent volumes
You cannot delete the base Snapshot copy in a parent volume while a FlexClone volume using that
Snapshot copy exists. The base Snapshot copy is the Snapshot copy that was used to create the
FlexClone volume, and is marked busy, vclone in the parent volume.
- snapshot
255 snapshot per volume
A Snapshot copy is a frozen, read-only image of a traditional volume, a FlexVol volume, or an aggregate
that captures the state of the file system at a point in time. It doesn't consume space initially, snapshot grows only as data changes.
- Aggregate
Max space: 16TB Max Number: 100
1 big aggregate runs faster than mulitiple aggregates created on same number of physcial disks
16 disks setup is the sweetspot for space usage utilization and performance
- Lun
Lun is created on top of volume, it can't exceeds the volume size. it is used to for FC/ISCSI mount
#= = command
sysconfig -r # show raid group and spare disks
aggr status -s #only show spare disks
sysconfig -V #show aggregate name and the numer of disk owned
sysconfig -d # show physical disks
storage show disk # show physical disks
- Traditional volume
vol create travol1 3 #create a traditional volume with 3 disks, disk are selected automattically
vol create trad02 -d 0a.24 0a.25 0a.27 #create a traditional volume by specifying disk names
- Flex volume
aggr create aggr2 4 #create aggregate first
vol create flexvol1 aggr2 20M #create volume on the aggregate
vol offline trad02
vol destroy trad02 #delete volume, need to be brought offline first
vol options vol1 nosnap on #turrn off automatic shceduled snapshot, not snapshot ablity
aggr show_space #show aggregate and volume space
aggr options aggr1 raidtype raid_dp #change raidtype between raid4 and raid_dp from raid0 to raid4/raid_dp
df -h #show volume space
- Snapshot
snap create vol0 mysnap0
snap delete vol0 mysnap0
snap list vol0 #list snapshot
snap delta vol0 #show size of changed data
/vol/vol0/.snapshot/mysnap0 #access snapshot data
#= = Network
- show ip/change ip
ifconfig
- add route
route add net 100.100.100.0 192.168.211.2 1
- permanet add
wrfile -a /etc/rc route add net 100.100.100.0 192.168.211.2 1
- vi /etc/rc
routed on # turn on RIP routing
- Link aggregation
vif
- Package tracing
pktt start ns0
pktt dump ns0 #a xx.trc file will be saved to /
pktt stop ns0
#= =NFS
- turn off nfs auto export for new volume
options nfs.export.auto-update off
- Show exports
exportfs -v
- show detailed export options
exportfs -q /vol/vol0
/vol/vol0 -sec=sys,(ruleid=0),rw,anon=0,nosuid
- Permanent export and add entry to /etc/exports
exportfs -p sec=sys,rw,nosuid /vol/vol1
- Temp export and don't add entry to /etc/exports
exportfs -io sec=sys,rw,nosuid /vol/vol1
- permanent unexport, remove from /etc/exports
exportfs -z path
- Temp export
exportfs -u path
- Re-read /etc/exports and re-export
export -r
- Control access
exportfs -io sec=sys,rw=10.10.10.1 /vol/vol1
or
exportfs enable nosave 10.10.10.1 /vol/vol1
#==CIFS
- Stop/start service
cifs terminate / cifs restart
- Initial setup
cifs setup /* select (3) Windows Workgroup authentication using the filer's local user accounts */
- Determine if both nfs client/cifs client access system
options wafl.default_security_style unix ntfs mixed
- display shares
cifs shares
- add share
cifs shares -add HOME /vol/vol0/home
- add permission
cifs access -delete home everyone
cifs access HOME Administrators "Full Control"
- Introduction
Data ONTAP is the name of Netapp's Platform OS, it is based on BSD.
Netapp appliance is based on i86 PC hardware(mostly AMD Opteron nowadays).
NetApp appliance is unified storage supports file-based protocols such as NFS, CIFS, FTP, TFTP, and HTTP, block-based protocols such as FC and iSCSI.
NetApp V-Series can attach and manage third party storage systems.
NVRAM: log transitions, it can replay the log in the event of unplanned shutdown.
RAM: system memory, data read/write cache
Flash card(not all models): system initial boot media
FC Controller port 0a/0b/0c/0d ....
Jumper to setup Disk shelf ID 1 - 7
Disk numbering from right to left 16 ......
FC:
WWNN - world wide name
is a World Wide Name assigned to a port in a Fibre Channel fabric, it performs a function equivalent to the MAC address in Ethernet protocol
WWPN - world wilde port name
is valid for the same WWNN to be seen on many different ports (different addresses)
ISCSI:
Initiator= client, target = server
Software initiator with standard NIC
TCP offload Engine TOE with soft initiator – offload computing from CPU
ISCSI HBA -Hardware initiator and provide diskless boot
No need to use ISCSI HBA, if diskless boot feature is not needed and CPU resource is plenty
Netapp use software ISCSI if no ISCSI HBA present.
# setup iscsi on linux
[root@linux /]# iscsi-iname
iqn.1987-05.com.cisco:01.8494d06dc35d
The ID generated is random, write it to:/etc/initiatorname.iscsi to be persistent
- help
help # list all cmds
cifs help shares #display help for sub command
man cmd
priv set advanced # to use advanced command e.g ls
- account managment
useradmin group list
useradmin user add admin -g Administrators
- Access netapp
http://IP/na_admin # web gui URL.
ssh IP # ssh (or rsh, telnet in) for CLI access, enable ssh with secureadmin cmd
rsh IP #excute cmd remotely from admin server, the admin server should be added to /etc/hosts.equiv
get root mount of /vol/vol0/ in a unix machine to edit config files
- Read/write file
rdfile /etc/exports
wrfile -a /etc/hosts 1.1.1.1 filer1 #append to file
wrfile /etc/hosts.equiv #Rewrite file, type in lines, then ctr+C
- Error messages
rdfile /etc/messages
- backup config file
config dump config.bak
it is saved to /etc/configs/config.bak
- server setting
options command control server setting
options ftpd.enable on #enable ftp server for example
options nfs.export.auto-update off #turn off auto export, otherwise the new volume will be exported automatically
all options are saved to /etc/registry
- system stats
stats show
sysstat -su 1
- copy entire volume
vol copy
ndmpd on
ndmpcopy -f /vol/vol0 /vol/vol0_trad
- Cluster
Netapp cluster doesn't do I/O load balancing, it is just for fail-over purpose.You need to allocate disk in each node for different services
#= =Boot
Ctrl+C to go to boot menu.
there is option to reset password
type in "22/7" to show secret boot menu
#= =Storage
Qtree, and/or subdirectories, export-able
|
Volume (TradVol, FlexVol), export-able, snapshot configured at this level.
|
agregate (OnTap 7.0 and up)
|
plex (relevant mostly in mirror conf)
|
raid group
|
disk
disk zero spare # zero all spare disk so they can be added quickly to a volume
Data ONTAP supports 100 aggregates (including traditional volumes) on a single storage system.
Data ONTAP supports 500 volumes per head(FAS2020 and FAS200 series, the limit is 200 FlexVol volumes.), So in cluster enviroment, the combined volumes number in both nodes should not exceed the limit for the sake of failover.
Netapp wafl(Write Anywhere File layout) block size = 4 KB.
Netapp support Raid0, Raid4, RaidDP (double-parity), Raid 1 (via snapmirror)
- Traditional Volume
It is tightly coupled swith its containing aggregate. No other volumes can get their storage from this containing aggregate. It can'b be shrinked. can be expanded by adding more disks
- FlexVol volumes
A FlexVol volume is a volume that is loosely coupled to its containing aggregate. A FlexVol volume
can share its containing aggregate with other FlexVol volumes,Thus, a single aggregate can be the
shared source of all the storage used by all the FlexVol volumes contained by that aggregate.
- flex clone
FlexClone volumes always exist in the same aggregate as their parent volumes
You cannot delete the base Snapshot copy in a parent volume while a FlexClone volume using that
Snapshot copy exists. The base Snapshot copy is the Snapshot copy that was used to create the
FlexClone volume, and is marked busy, vclone in the parent volume.
- snapshot
255 snapshot per volume
A Snapshot copy is a frozen, read-only image of a traditional volume, a FlexVol volume, or an aggregate
that captures the state of the file system at a point in time. It doesn't consume space initially, snapshot grows only as data changes.
- Aggregate
Max space: 16TB Max Number: 100
1 big aggregate runs faster than mulitiple aggregates created on same number of physcial disks
16 disks setup is the sweetspot for space usage utilization and performance
- Lun
Lun is created on top of volume, it can't exceeds the volume size. it is used to for FC/ISCSI mount
#= = command
sysconfig -r # show raid group and spare disks
aggr status -s #only show spare disks
sysconfig -V #show aggregate name and the numer of disk owned
sysconfig -d # show physical disks
storage show disk # show physical disks
- Traditional volume
vol create travol1 3 #create a traditional volume with 3 disks, disk are selected automattically
vol create trad02 -d 0a.24 0a.25 0a.27 #create a traditional volume by specifying disk names
- Flex volume
aggr create aggr2 4 #create aggregate first
vol create flexvol1 aggr2 20M #create volume on the aggregate
vol offline trad02
vol destroy trad02 #delete volume, need to be brought offline first
vol options vol1 nosnap on #turrn off automatic shceduled snapshot, not snapshot ablity
aggr show_space #show aggregate and volume space
aggr options aggr1 raidtype raid_dp #change raidtype between raid4 and raid_dp from raid0 to raid4/raid_dp
df -h #show volume space
- Snapshot
snap create vol0 mysnap0
snap delete vol0 mysnap0
snap list vol0 #list snapshot
snap delta vol0 #show size of changed data
/vol/vol0/.snapshot/mysnap0 #access snapshot data
#= = Network
- show ip/change ip
ifconfig
- add route
route add net 100.100.100.0 192.168.211.2 1
- permanet add
wrfile -a /etc/rc route add net 100.100.100.0 192.168.211.2 1
- vi /etc/rc
routed on # turn on RIP routing
- Link aggregation
vif
- Package tracing
pktt start ns0
pktt dump ns0 #a xx.trc file will be saved to /
pktt stop ns0
#= =NFS
- turn off nfs auto export for new volume
options nfs.export.auto-update off
- Show exports
exportfs -v
- show detailed export options
exportfs -q /vol/vol0
/vol/vol0 -sec=sys,(ruleid=0),rw,anon=0,nosuid
- Permanent export and add entry to /etc/exports
exportfs -p sec=sys,rw,nosuid /vol/vol1
- Temp export and don't add entry to /etc/exports
exportfs -io sec=sys,rw,nosuid /vol/vol1
- permanent unexport, remove from /etc/exports
exportfs -z path
- Temp export
exportfs -u path
- Re-read /etc/exports and re-export
export -r
- Control access
exportfs -io sec=sys,rw=10.10.10.1 /vol/vol1
or
exportfs enable nosave 10.10.10.1 /vol/vol1
#==CIFS
- Stop/start service
cifs terminate / cifs restart
- Initial setup
cifs setup /* select (3) Windows Workgroup authentication using the filer's local user accounts */
- Determine if both nfs client/cifs client access system
options wafl.default_security_style unix ntfs mixed
- display shares
cifs shares
- add share
cifs shares -add HOME /vol/vol0/home
- add permission
cifs access -delete home everyone
cifs access HOME Administrators "Full Control"
Subscribe to:
Posts (Atom)
Posts
