PowerCLI>$esxcli=get-esxcli -vmhost esx01 PowerCLI>$esxcli.storage.vmfs.extent.list() | ft devicename,volumename -autosize DeviceName VolumeName ---------- ---------- naa.600601605bc02e00007fb97cacbee211 datastore01 naa.600601605bc02e00fac8cd88acbee211 datastore02
vSphere Single Sign On is a new feature in vSphere 5.1, vSphere SSO controls authentication service, so you can no longer add new authentication provider in vCenter by standard vSphere client. It has to be done in vSphere webclient, which can talk to vSphere SSO service.
Steps to add Windows Active Directory provider.
1.Create a generic user in AD for LDAP search, define user and group base DN.
2.Install Webclient from vCenter installation media, just like vSphere Client, it doen’t need to be installed on vCenter server.
3.Launch Webclient https://client-ip:9443/vsphere-client and login
The account used for login is important, if you installed SSO service when login with local account, local account can login Webclient, But it doesn’t have permission to configure SSO, you have to login with the default SSO account “admin@System-Domain” created during installation.
4.Navigate to Administration/Sign-on and Discovery/configuraiton( the configurion node won’t be shown, if login with local Windows account), and add “+” sign to add identity sources.
The login credentials will be sent in clear text with ldap, if it is a concern, enable ldaps by creating certificate
The username should be in LDAP syntax, find the exact string in ADSI edit tool in AD.
![clip_image002[4]](http://lh3.ggpht.com/-6y7SkOK8IZg/UKzBnkAUypI/AAAAAAAAALg/j38j3Ql5bek/s1600-h/clip_image002%25255B4%25255D%25255B3%25255D.jpg "clip_image002[4]")
VMware vCenter supports DB2,Oracle or MS SQL server as backend database, the built-in database for vCenter in Windows is SQL Server 2008 express,which has limits on disk space and memory, it is not suitable for enterprise. For enterprise deployment, it is recommended to use proper database engine, such as Oracle 11g R2
Install Oracle Database
Select a compatible version of Oracle listed in VMware website.
http://www.vmware.com/resources/compatibility/sim/interop_matrix.php
The Versions used in this test:
- vCentre 5.0.0 build 623373
- Oracle 11g R2 11.2.0.3.0
Setup Oracle database for vCenter
- Create an Oracle SQL login account for vCenter
Estimate vCentre database tablespace size requirement.
vCenter has a tool to estimate the size.
vCenter->administration->Server setting->statistics
for example, to keep data of 500 VMs for 1 year needs ~5GB storage
old data can be automatically purged by setting up retention policy
vCenter->administration->Server setting->database retention policy
Extract from “vSphere Installation and Setup” document
#1 Log in to a SQL*Plus session with the system account.#2 Run the following SQL command to create a vCenter Server database user with the correct permissions.#The script is located in the vCenter Server {installation media}/vcenter/dbschema/DB_and_schema_creation_scripts_oracle.txt file.#In this example, the user name is VPXADMIN.CREATE USER "VPXADMIN" PROFILE "DEFAULT" IDENTIFIED BY "oracle" DEFAULT TABLESPACE"VPX" ACCOUNT UNLOCK;grant connect to VPXADMIN;grant resource to VPXADMIN;grant create view to VPXADMIN;grant create sequence to VPXADMIN;grant create table to VPXADMIN;grant create materialized view to VPXADMIN;grant execute on dbms_lock to VPXADMIN;grant execute on dbms_job to VPXADMIN;grant select on dba_tablespaces to VPXADMIN;grant select on dba_temp_files to VPXADMIN;grant select on dba_data_files to VPXADMIN;grant unlimited tablespace to VPXADMIN;#By default, the RESOURCE role has the CREATE PROCEDURE, CREATE TABLE, and CREATE#SEQUENCE privileges assigned. If the RESOURCE role lacks these privileges, grant them to the vCenter#Server database user.#NOTE Instead of granting unlimited tablespace, you can set a specific tablespace quota. The#recommended quota is unlimited with a minimum of at least 500MB. To set an unlimited quota, use the#following command.#alter user "VPXADMIN" quota unlimited on "VPX";#If you set a limited quota, monitor the remaining available tablespace to avoid the following error.#ORA-01536: space quota exceeded for tablespace '<tablespace>'#3 (Optional) After you have successfully installed vCenter Server with the Oracle database, you can revoke#the following privileges.revoke select on dba_tablespaces from VPXADMIN;revoke select on dba_temp_files from VPXADMIN;revoke select on dba_data_files from VPXADMIN;
Prepare Windows server for vCenter
- Install Oracle ODBC client
Download both basic and ODBC client from Oracle website.
instantclient-basic-windows.x64-11.2.0.3.0.zip
instantclient-odbc-windows.x64-11.2.0.3.0.zip
#unzip instantclient-basic-windows.x64-11.2.0.3.0.zip to:C:\Program Files\Oracle\instantclient_11_2unzip instantclient-odbc-windows.x64-11.2.0.3.0.zip to the same directory as basic installant clientrun odbc_install.exe in command lineC:\Program Files\Oracle\instantclient_11_2>odbc_install.exeOracle ODBC Driver is installed successfully.mkdir C:\Program Files\Oracle\instantclient_11_2\network\admin#copy tnsnames.ora on Oracle server to the directory#Add new Windows system variable "ORACLE_HOME=C:\Program Files\Oracle\instantclient_11_2"#system variable take effects immediately, open a new command line to check this:C:\ >echo %ORACLE_HOME%C:\Program Files\Oracle\instantclient_11_2
open "odbc source" in administrative tools, create "system DSN", select oracle driver, type in service name defined in tnsnames.ora,username etc
Make sure "test connection" result is ok
Install vCenter
Follow the installation Wizard to install vCenter, you may receive warning about the Oracle client need to be updated, select ok to continue
Update vCenter ojdbc client
cd /d "C:\Program Files\VMware\Infrastructure\tomcat\lib\"#backup original filecopy ojdbc5.jar ojdbc5.jar.orig#overwrite with new ojdbc5.jar from instant clientcopy C:\Program Files\Oracle\instantclient_11_2\ojdbc5.jar ojdbc5.jar
Configure Advanced IP options (yes/no)? [n]: noEnable the ssh service? (yes/no) [y]: yesConfigure svs domain parameters? (yes/no) [y]: yesEnter SVS Control mode (L2 / L3) : L3#(VEM-VSM communication can operate in Layer 2 mode or Layer 3 mode, L3 is recommended.But N1000V is layer 2 device, ip routing is not supported)Configure Advanced IP options (yes/no)? [n]: noEnable the ssh service? (yes/no) [y]: yesConfigure svs domain parameters? (yes/no) [y]: yesEnter SVS Control mode (L2 / L3) : L3#(VEM-VSM communication can operate in Layer 2 mode or Layer 3 mode, L3 is recommended.But N1000V is layer 2 device, ip routing is not supported)
#Launch the JAVA installerC:\Nexus1000v.4.2.1.SV1.5.1\VSM\Installer_App>java -jar Nexus1000V-install.jar VC#Following the wizard to establish connection between VSM and vCenter, the result is that a new distributed switch will be created in “networking “ view in vCenter.Verify the connection in N1000V CLI:nv1> show svs connections….config status: Enabledoperational status: Connectedsync status: Completeversion: VMware vCenter Server 5.0.0 build-623373vc-uuid: F1D0CEBA-C365-4F55-830D-A0B9BB6F8520#Don’t continue to the next step without a successful connection is seen.
C:\Nexus1000v.4.2.1.SV1.5.1\VSM\Installer_App>java -jar Nexus1000V-install.jar VEM#Launch the Java installer, which will connect to vCenter and VSM to push VEM module to ESX host.#You can also install the VEM module manually.C:\Nexus1000v.4.2.1.SV1.5.1\VEM\cross_cisco-vem-v140-4.2.1.1.5.1.0-2.0.1.vib#Transfer the file to ESXi 5 host and runESXi>esxcli software vib install -v /tmp/cross_cisco-vem-v140-4.2.1.1.5.1.0-2.0.1.vib
#Because VEM and VSM are on the host, you need to put VEM VSM into separate physical NICs in order to migrating from standard switch to N1000V switch.#Create port profile on N1000V#create an uplink profile to be linked to ESXi host pNIC# note the type is Ethernet and switchport is trunkport-profile type ethernet vm-uplinkvmware port-groupswitchport mode trunkswitchport trunk allowed vlan 3-10no shutdownsystem vlan 3,5-6state enabled#Create port-group for vmkernel management interface.#note the type is venthernet and “capability l3control” and system vlan are mandatoryport-profile type vethernet L3vmkernelcapability l3controlvmware port-groupswitchport mode accessswitchport access vlan 3no shutdownsystem vlan 3state enabled# the following 3 profiles for N1000V port-groups, system vlan is mandatoryport-profile type vethernet ds_ctlvmware port-groupswitchport mode accessswitchport access vlan 3no shutdownsystem vlan 3state enabledport-profile type vethernet ds_mgtvmware port-groupswitchport mode accessswitchport access vlan 3no shutdownsystem vlan 3state enabledport-profile type vethernet ds_pktvmware port-groupswitchport mode accessswitchport access vlan 3no shutdownsystem vlan 3state enabled#the following is for generic VM traffic, vCentre VM will be migrated to this in the first state.#system vlan is optionalport-profile type vethernet vmdatavmware port-groupswitchport mode accessswitchport access vlan 3no shutdownsystem vlan 3state enabled
nv1> show moduleMod Ports Module-Type Model Status--- ----- -------------------------------- ------------------ ------------1 0 Virtual Supervisor Module Nexus1000V active *3 248 Virtual Ethernet Module NA ok
#Sample QOS configuration bound for a port-grouppolicy-map type qos po-vmdataclass class-defaultpolice cir 1500 mbps bc 200 ms conform transmit violate set dscp dscp table pir-markdown-mapport-profile type vethernet vmdataservice-policy input po-vmdataservice-policy output po-vmdata#Control VEM in ESXi host directly when VSM is not available.#unblock a port by defining correct system vlanESXi > vemcmd show portLTL VSM Port Admin Link State PC-LTL SGID Vem Port Type49 DOWN UP BLK 0 testlinux.eth0ESXi > vemset system-vlan 3 ltl 49ESXi > vemcmd show portLTL VSM Port Admin Link State PC-LTL SGID Vem Port Type49 UP UP FWD 0 testlinux.eth0
What is changed in ESXi 5 kickstart:
1. ESXi 5 supports EFI PXE boot(the file is under install media\EFI\boot)
2. ESXi 5 loads all needed packages into memory, there is no need to specify installation media in http/nfs etc
3. ESXi 5 supports vlan tagging by –vlanid parameter, previously PXE boot NIC has to be native vlan (VLAN 1)
4. ESXi 5 no longer supports set license data in kickstart file
ESXi 4 kickstart file example
vmaccepteula#create encrypted password with command “openssl passwd -1”rootpw --iscrypted $1$XXXXXXXXXXXXXXXXautopart --firstdisk=mpx.vmhba32:C0:T0:L0 --overwritevmfsinstall url http://172.16.1.18/boot/XXXXnetwork --device=vmnic0 --hostname=esxi1 --bootproto=static --ip=172.16.1.4 --netmask=255.255.255.224 --gateway=172.16.1.30 --nameserver="172.16.1.18,172.16.1.19"serialnum --esx=4H42M-XXXX-XXXXX-XXXXX-XXXXX%post --interpreter=busybox --ignorefailure=truereboot#enable local console and remote ssh login%firstboot --interpreter=busybox/etc/init.d/TSM start/etc/init.d/TSM-SSH start/sbin/chkconfig TSM on/sbin/chkconfig TSM-SSH on
vmaccepteula#create encrypted password with command “openssl passwd -1”rootpw --iscrypted $1$XXXXXXXXXXXXXXXXclearpart --firstdisk=local --overwritevmfsinstall --firstdisk --overwritevmfs#ESXi 5 retrieve software from the image loaded into memory by PXE, so there is no need to specify software repository locationnetwork --device=vmnic0 --vlanid=3 --hostname=esxi2 --bootproto=static --ip=172.16.1.5 --netmask=255.255.255.224 --gateway=172.16.1.30 --nameserver="172.16.1.18,172.16.1.19"#loading license data in kickstart file not longer supported in ESXi 5#serialnum --esx=4H42M-XXXX-XXXXX-XXXXX-XXXXX%post --interpreter=busybox --ignorefailure=truereboot#enable local console and remote ssh login%firstboot --interpreter=busybox/etc/init.d/SSH start/etc/init.d/ESXShell start/sbin/chkconfig SSH on/sbin/chkconfig ESXShell on
PXE Kickstart VMware ESX host was usually performed on the native VLAN, It was not possible in a trunked interface (Tagged VLAN) due to two obstacles:
The above have been overcome by the following technologies:
1. Mutiple boot agent(MBA): NIC supports VLAN tag in BIOS, such as MBA of Broadcom NICs (otherwise PXE client couldn’t get IP address from DHCP in the first place)
2. VMware ESXi 5: A new parameter, vlanid, is introduced as a boot option.(not the same as vlanid in kickstart configuration file)
Configure MBA for Broadcom NetXtreme II BCM5709 in IBM X3850 server
F1 go to BIOS->System Setting->Network->select the network adapter in device list -> configure mutiple boot agent(MBA) and iSCSI parameter -> MBA configuration menu-> vlan mode/vlan id
Add VLANID in ESXi 5 boot option
A sample boot.cfg for gPXE
bootstate=0title=Loading ESXi installerkernel=http://pxe-server.example.com/boot/os/esxi-5.0-x64/pxeboot/tboot.b00#BOOTIF is mandatory for gPXE, it seems ESXi 5 couldn’t get name server and domain name and from DHCP, So the name server is assigned manually.kernelopt=BOOTIF=01-5c-f3-fc-94-e4-18 vlanid=3 nameserver=172.16.1.1 ks=http://pxe-server.example.com/boot/hosts/linux-ks/5c-f3-fc-94-e4-18.txtmodules=http://pxe-server.example.com//boot/os/esxi-5.0-x64/pxeboot/b.b00 ... < ..omitted ..>
VMware can also customize Windows guest OS by Windows sysprep tool, though the process is more complex than Linux guest OS
There are two options to clone and “sysprep” VMware Windows guest OS:
1.Install sysprep tools in Windows guest OS and run sysprep.exe in guest OS command line, then clone it by VMware
2.Install sysprep tools in Virutal Center and let VMware tools in Windows guest in to control sysprep process either by GUI or script. (It seems sysprep rely on VMware tools, so the VMware tools must be installed in guest OS)
Option #2 is the preferred method, because you can use script to easily customize unique information e.g computer name, ip addresses etc.
Install sysprep tools in Virtual Center
VMware KB: Sysprep file locations and versions
Windows Vista onwards(vista/2008/7/2008R2/) don’t need this step, because its sysprep is built-in.
Create guest customization by GUI
Virtual Center -> view-> management -> “customization specification manager”. Create a new customization and select it when asked for guest customization information in GUI clone action.
Create guest customization by script.
Customization by GUI is not flexible, customization by script can be created on the fly. The following is enhancement to the clonevm.ps1 in Vsphere PowerCLI script to clone and customize Linux guest OS, just replace the “Identity for Linux” part with following code block $s_ostype=Retrieve-values $lines "ostype"if ( $s_ostype -eq "linux" ) {## Identity for Linux$vmclonespec_os.identity= New-Object VMware.Vim.CustomizationLinuxPrep$vmclonespec_os.identity.hostname= New-Object VMware.Vim.CustomizationFixedName$vmclonespec_os.identity.hostname.name= $s_dstname$vmclonespec_os.identity.domain=$s_domain}elseif ( $s_ostype -eq "windows" ) {# WinOptions$vmclonespec_os.Options = New-Object VMware.Vim.CustomizationWinOptions$vmclonespec_os.Options.ChangeSID = 1#sysprep$vmclonespec_os.identity = New-Object VMware.Vim.CustomizationSysprep# GUIUnattended$vmclonespec_os.Identity.GuiUnattended = New-Object VMware.Vim.CustomizationGuiUnattended$vmclonespec_os.Identity.GuiUnattended.AutoLogon = 0#timezone codes: http://msdn.microsoft.com/en-us/library/ms145276(v=sql.90).aspx$vmclonespec_os.Identity.GuiUnattended.TimeZone = 255$vmclonespec_os.Identity.GuiUnattended.Password = New-Object VMware.Vim.CustomizationPassword$vmclonespec_os.Identity.GuiUnattended.Password.PlainText = 1$vmclonespec_os.Identity.GuiUnattended.Password.Value = "Secret01"# Identification$vmclonespec_os.Identity.Identification = New-Object VMware.Vim.CustomizationIdentification$vmclonespec_os.Identity.Identification.joinWorkgroup = "workgroup2"## Userdata$vmclonespec_os.identity.userData = New-Object VMware.Vim.CustomizationUserData$vmclonespec_os.identity.userData.computerName = New-Object VMware.Vim.CustomizationFixedName$vmclonespec_os.identity.userData.computerName.name = $s_dstname$vmclonespec_os.Identity.UserData.FullName = "Administrator"$vmclonespec_os.Identity.UserData.OrgName="myOrg"$vmclonespec_os.Identity.UserData.Productid=""}else {write-host "Unknown ostype: $s_ostype. Please set it to linux or windows"exit}}
Troubleshoot after RHEV-H has been installed.#RHEV-H boot prompt options: #Just press enter to start installation.:linux rescue #same as RHEL rescue mode:linux firstboot #invoke interactive installation menu:linux upgrade #upgrade hypervisor:linux nocheck #disable installation media check#Hypervisor Configuration MenuRed Hat Enterprise Virtualization Hypervisor release 5.5-2.2Hypervisor Configuration Menu1) Configure storage partitions 6) Configure the host for Red Hat EnterpriseVirtualization2) Configure authentication 7) View logs3) Set the hostname 8) Install locally and reboot4) Networking setup 9) Support Menu5) Register Host to RHN#options notes“5) Register Host to RHN” is optional, just configure 1,2,3,4,6 then choose 8“9) Support Menu” has an option to uninstall existing RHEV-H
NFS store#re-invoke the Hypervisor Configuration Menu$setup #select option 6 to re-configure hostname for RHEV-M#restart registration process/etc/init.d/vdsm-reg restart#check registration log/var/log/vdsm-reg/vdsm-reg.log#Configure files in RHEV-H#vdsm registration script#register itself to RHEV-M, it seems it doesn't need to be running once registration is successful/etc/init.d/vdsm-reg #start-up script,/etc/vdsm-reg/vdsm-reg.conf #configuration file/var/log/vdsm-reg/vdsm-reg.log #log file#Management agent#by default, listening on port 54321 to communicate with RHEV-M/etc/init.d/vdsmd/etc/vdsm/vdsm.conf/var/log/vdsm/vdsm.logYou are not supposed to create new configuration files in RHEV-H, any new files in /etc/ will be lost after reboot. To survive reboot, you need copy your customization files, e.g /etc/hosts, /etc/resolv.conf, to “/config/etc/” once. Next time RHEV-H boots up, it will synchronize all files in /config/etc/* to /etc
| RHEV 2.2 | ESX 4 | |
| Manager | ||
| Name | RHEV-M | vCenter |
| Compatible OS | Windows 2003 Windows 20008 R2 | Windows XP Windows 2003 Windows 2008 Windows 2008 R2 |
| Backend DB | Microsoft SQL Server | Microsoft SQL server Oracle |
| Application Type | Web application (WPF .xbap application) | Windows native application |
| User Interface | Web UI | Web UI Windows native application |
| CLI [1] | Powershell | Powershell(PowerCLI) vCLI |
| SDK&API | Powershell | Powershell, Perl,C#, Java |
| Hypervisor | ||
| Type | Linux kernel (KVM) | Proprietary |
| Manager Agent | Python script | Binary daemon |
| HA/Migration [2] | YES | YES |
| Manager independent [3] | NO | YES |
| CLI [4] | NO | esxcfg-*/vimsh commands |
| SDK&API | NO | Powershell, Perl,C#, Java |
| Storage Type [5] | NFS/iSCSI/FC | local disk/NFS/iSCSI/FC |
| Guest OS | ||
| supported OS [6] | Red Hat Enterprise Linux Windows | All major Linux distributions Windows Solaris Mac OS/BSD |
| Clone [7] | Supported | supported |
| Snapshot [8] | limited support | supported |
| Supported Hard disk [9] | IDE, VirtIO | IDE,SCSI |
| Cost | ~2/3 of VMware cost | expensive |
$brctl addbr br-name; ifconfig br-name IP upWhere does libvirt save the VM configuration file?
Install libvirt on Debianfunction dumpkey() {local param=${1}local keylocal resultresult=$(xenstore-list ${param})if [ "${result}" != "" ] ; thenfor key in ${result} ; do dumpkey ${param}/${key} ; doneelseecho -n ${param}'='xenstore-read ${param}fi}for key in /vm /local/domain /tool ; do dumpkey ${key} ; done
(xend-unix-server yes) $/etc/init.d/xend restartDefine new network bridge
Type “virsh” to enter an virsh interactive prompt<network><name>private</name><bridge name="virbr2" /><ip address="192.168.152.1" netmask="255.255.255.0"></ip></network>
Install Centos para-virtualization guest.Network private defined from /tmp/net.xmlvirsh # net-autostart privateNetwork private marked as autostartedvirsh # net-start privateNetwork private startedvirsh # net-listName State Autostart-----------------------------------------private active yesroot@xen4:/# ifconfig virbr2virbr2 Link encap:Ethernet HWaddr de:49:4e:43:c5:5dinet addr:192.168.152.1 Bcast:192.168.152.255 Mask:255.255.255.0UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1RX packets:0 errors:0 dropped:0 overruns:0 frame:0TX packets:0 errors:0 dropped:0 overruns:0 carrier:0collisions:0 txqueuelen:0RX bytes:0 (0.0 B) TX bytes:0 (0.0 B)
After the VM has been created, you can use xen native tool /usr/sbin/xm or libvirt virsh command to start/stop VM. But any configuration change require the virsh edit commands (edit, net-edit, pool-edit vol-edit, iface-edit)--paravirt \--name pv2 \--ram 256 \--disk path=/data/pv2.raw,size=2,format=raw \--os-type=linux --os-variant=rhel5.4 \--nographics \--network network=private \--location http://192.168.152.1/pkgs/
Posts
All Comments
