- Provisioning Services 6.1
- Provisioning Services 6.0
The VHDs must be fixed because internal structure of dynamic VHDs is different and can cause alignment problems concerning disk subsystem (that is NetApp- Filer). There is a whitepaper created by NetApp covering alignment considerations (same applies to Hyper-V VHDs). 2 vDisk Storage Considerations The following chapter outlines different options for configuring the Provisioning Server vDisk store. Local vDisks Using the local hard disk subsystem of the Provisioning Servers to store the vDisks provides the easiest way of implementing vDisk high availability without additional cost. Note: When configuring a vDisk store pointing to a local directory of.
This article contains information about the Best Practices for Provisioning Services 6.x.
Personal vDisks running Windows 7 cannot use the Backup and Restore feature when the Windows system protection feature is enabled. If system protection is disabled, the user profile is backed up, but the userdata.v2.vhd file is not. Citrix recommends disabling system protection and using Backup and Restore to back up the user profile. VDisks are sized dynamic, not fixed – Saves disk space. Standard Mode vDisks don’t grow so no performance impact. VDisk files are defragmented. VDisk files are backed up. VDisk updates are automated. Target Devices: Target Device Boot Method is highly available – Target Devices on same subnet Provisioning Servers. In the Welcome to the Installation Wizard for Citrix Provisioning Server x64 page, click Next. In the License Agreement page, select I accept the terms, and click Next. In Citrix Provisioning 1811 and newer, you’ll see a Default Firewall Ports page. You can optionally select Automatically open all Citrix Provisioning ports in Windows Firewall.
Provisioning Services Caching
Caution! Refer to the Disclaimer at the end of this article before using Registry Editor.
- Install Provisioning Services as x64 to leverage Large System Cache if Server has enough memory.
- Activate Large System Cache on Provisioning Services:
Guided reading assessment pdf. Note: This is for Windows 2003 and specifies whether the system maintains a standard size or a large size file system cache, and influences how often the system writes changed pages to disk - http://technet.microsoft.com/de-de/library/cc784562%28WS.10%29.aspx
It is no longer used since Server 2008: http://blogs.technet.com/b/askperf/archive/2008/02/01/ws2008-upgrade-paths-resource-limits-registry-values.aspx
Refer to article CTX119469 - Understanding Write-Cache in Provisioning Services Server for additional information.
High Availability Timeouts
Configure High Availability Server Fail Over Timeouts – Registry Settings to Improve Failover Times for Citrix Provisioning Services on target machine - CTX119223 - Registry Settings to Improve HA Failover Times for Provisioning Service
- HKEY_LOCAL_MACHINESYSTEMCurrentControlSetServicesBNIStackParameters IosPacketTimeoutms <DWORD>
- HKEY_LOCAL_MACHINESYSTEMCurrentControlSetServicesBNIStack > Parameters > IosRetryLimit <DWORD>
Note: Tuning this key can reduce failover from maximum 95 sec. down to 2 sec. Remember that Target Device does not start to toggle between Provisioning Services servers.
- Provisioning Services Daylight Saving Time problem, 1 hour time difference and impact on Kerberos authentication (impacts all streamed targets).
If changes are made to default console port of Provisioning Services, do this for MCLI as well:
“mcli.exe run SetupConnection –p Port=PortNumber”
- Same applies to “SetupToolApplication.exe.config” for XenDesktop 4 Setup Wizard
- If your environment is based on MS SMS 2003, you should consider http://support.microsoft.com/kb/828367 to avoid duplicate GUIDS.
SQL Permissions for Provisioning Services Database
- To install Provisioning Services, the user must have local administrator privileges.
Note: This user does not require any permission to access the db.
- The user running Configuration wizard must have SQL administrator permission (sysadmin) to be able to create and configure the db.
- The user which Stream/SOAP services run as, must have db_datareader and db_datawriter roles:
CTX120080 - Service Account Configuration for Accessing SQL
- Use DBSCRIPT.exe to pre-create DB on SQL server.
- If you cannot connect to SQL2008, then leave the instance empty as well as port while running configuration wizard. You must point to the IP address of the SQL server.
- Provisioning Services uses an optimized UDP-based protocol to communicate with the target devices
- Data is streamed to each target device only as requested by the OS and applications running on the target device
- In most cases, less than 20% of any application is ever transferred
- Network utilization is most significant when target devices are starting as the OS loads, after target devices start, there is minimal network utilization
Configure active network components accordingly:
Fast Link option name
PortFast or STP Fast Link
Spanning Tree FastLink
Note: For additional information, check CTX117374 – Best Practices for Configuring Provisioning Server on a Network
PXE or Network Boot
- Create a dummy collection in Provisioning Services console and enable the Auto-Add feature in the farm.
- Create a target device inside the dummy collection and define as template.
- Assign a small vDisk (that is 100 MB) and select Boot from Hard Disk.
- In site properties, point the Auto-Add feature to dummy collection.
This ensures that no machines are unresponsive and showing No vDisk found if configured to PXE boot and no target device entry is created in the Provisioning Services database.
- Set following registry keys in vDisk - Golden Master (Target Device):
DWORD = EnableOffload
Disable TCP Large Send Offload for Provisioning Services driver:
Adds latency as packets re-segmented
Must be set on Provisioning Server and Target Device:
Key: 'DisableTaskOffload' (dword)
- Try to synchronize all MTUs across your network if possible.
Do this on Virtual Desktop Agents, Provisioning Services, XenServer to avoid network problems when tunneling protocols, adjust MaxICAPacketSize, and all MSS accordingly.
CTX117374 - Best Practices for Configuring Provisioning Server on a Network
CTX117491 - Excessive Amounts of Retries Occur when a Provisioning Server Target Device is Deployed on a XenServer Platform
ARP Cache Changes – Windows 2008 / Vista / Windows 7
The default lifespan of ARP cache entries was lowered from 10 minutes in Windows Server 2003 to a random value between 15 and 45 seconds in Vista/W2K8. As a result, the Provisioning Services bootstraps are 20 times more likely to experience a timeout during a Vista/W2K8 boot. The workaround is to increase the ARP cache entry lifespan for Provisioning Services -bound NIC’s: Provisioning Services Server and VDA:
- Open a command shell window. At the command prompt, enter the following command:
netsh interface ipv4 show interfaces
- To set the ARP cache entry lifespan to 600 seconds, enter the following command:
netsh interface ipv4 set interface <PVS interface number> basereachable=600000
To verify the new setting, enter the following command:
netsh interface ipv4 show interface <PVS interface number>
Note: The Base Reachable Time should be set to 600,000 ms, and the Reachable Time to a value between 300,000 and 900,000 ms.
For more information, refer to http://support.microsoft.com/kb/949589.
Boot Sector / PAE / TFTP
If Target Device would not start with bootloader or NTLDR not found:
- Verify this partition has 0x20 reserved sectors in MBR: run dskprobe.exe, read from PhysicalDrive1 (local disk) sector 0. Verify data offset 0x1c6 is 0x20.
- To verify, this partition has 0x20 reserved sectors in PBR: run dskprobe.exe, read from PhysicalDrive1 (local disk) sector 32. Verify data offset 3 is ‘NTFS’ and offset 0x18 is 0x20 (when formatting with some SCSI/Raid controllers windows format places a 0x3F in this location which causes the machine not to start).
When Windows 7 does not start, it is likely caused by PAE (Advanced Memory Support). Starting in 5.1.2, this option is enabled by default in the Bootstrap. However, in the BDM and Bios Bootstrap (OROM in DELL FX Series), this option is still disabled by default. Windows 7 and newer always require PAE to start in RAM Cache mode. This is the reason why PXE works and BDM and OROM does not, if not rewritten or configured properlyNote: For additional information, refer to CTX126107 - Error: 'vDisk Not Available' When Creating a New vDisk After Reboot using BDM ISO
- Use BOOTPTAB Editor to allow or prevent only special clients to obtain ARDBP32.BIN through PXE/TFTP.
- Use TFTPD32.exe (Freeware) or DHCPExplorer (free from SoftPedia.com) to discover which TFTP/PXE services are already running in the environment.
- In order to bind TFTP Daemon to a specific NIC/Port, configure logging for TFTP Daemon, set the “GET” Directory ( server side) and use:
“%Program Files%CitrixProvisioning Servicestftpcpl.cpl”
- In order to bind TFTP Daemon to a specific NIC/Port, configure logging for TFTP Daemon, set the “GET” Directory for the TwoStageBootloader (server side) and use:
- In order to bind the Stream service to specific NIC (target device side) use:
- If in your environment PXE/TFTP has problems with finding ardbp32.bin or tsbbdm.bin you should check the following registry keys:
Default configuration sets the TFTP directory to
If tsbbdm.bin, for example, is not found during boot:
Try to copy the file from “C:Program FilesCitrixProvisioning ServicesTSBboot”
To directory specified in HKLMSYSTEMCurrentControlSetServicesBNTFTPParametersGetDirectory
Common vDisk Tuning
Delete Bitlocker 100 MB Partition during setup of Golden Master using Shift+F10 during first setup screen of the installation > Start “Diskpart” - and inside “Diskpart” tool complete the following steps:
- select disk 0
- create partition primary
- select partition 1
- format fs=ntfs quick
Disable Last Access Timestamp of Files in vDisk (can be done through XenConvert optimizer)
“FSUTIL behavior set disablelastaccess 1
Note: You can add many customizations through editing optimizations.xml in XenConvert directory (target device side)
- Prior to building any vDisk Flush the DNS Resolver Cache:
- Run chkdsk before starting XenConvert or Imaging Wizard.
- Use Sysinternals sDelete –c driveletter to zero out empty vDisk areas and reduce storage when creating golden master.
- Disable Windows Indexing Service and System Restore.
- Configure redirection of spool directory, virus patterns, RADECache, EdgeSight DB, AppSense Profile, Databases, Event logs, Log files to a persistent CacheDisk or CacheVolume if possible
- If provisioning hardware, you must use newest BIOS or Firmware to avoid hardware conflicts.
- If XenConvert throws an error while creating the vDisk, try to exclude directories through XenConvert.ini, which could not be copied.
- If virtualizing XenApp Server Configuration Tool 1.1 for XenApp 6 Sealing before Provisioning Services image creation (includes preparation for MSMQ):
CTX124981 - XenApp Server Configuration Tool - Update 1.1.0 for XenApp 6 for Windows Server 2008 R2
- Before switching vDisk to Standard Image mode: “ipconfig/release” (release DHCP address).
- Do updates by creating a new version of the vDisk and modifying the maintenance version. When the update is ready, promote the version to test. After testing, promote the version to production using immediate or scheduled availability.
- Schedule merge of the vDisk versions after a number of versions have been created to save space and increase performance.
- As Virtual Hard Disk (VHD) expands:
- Disk can become fragmented on physical media
- Expansion algorithm occurs in 1 MB increments
- Rapid expansion wreaks havoc on SAN such as first boot or page file creation
- Alignment issues
- Constructed with extra byte at end of file
- Dynamic VHD always misaligns disk with storage
- Use only fixed-size VHDs for write-cache drives and Provisioning services vDisks.
High Availability per Component
- Use High Availability (HA)-Setup for DHCP and TFTP Services
- Load Balance TFTP (NetScaler VPX is a good option)
- Use multi-server PVS Farms – Target devices can switch to other PVS when streaming PVS gets unavailable
- Use streamed apps - multiple profile shares + NetScaler for Load Balancing
- Use load-balanced XenApp-Farms for best load distribution and user density
- Use HA for License server because in file mps-wsxica_mps-wsxica.ini information about licenses + timestamp are stored during contact of license server. In standard image mode this file cannot be updated and 30 days after vDisk creation, there will be no grace period if license server goes down because timestamp is too old
- Use STAs of XenApp Farm, because health checks are available and can be used
- Use multiple PVS Servers for high availability and redundancy
- Place PVS Servers as near as possible to target devices for high performance and bandwidth
Note: For additional information, refer to:
CTX116337 - How to Load Balance Trivial File Transfer Protocol Servers
CTX119286 - Provisioning Server High Availability Considerations
Traffic Bottleneck if only 100Mbit Available
- Resource bottlenecks are mostly I/O related and hardly ever RAM or CPU dependent, test scalability if write cache on server should be used
- Amount of write cache is related to user activity and applications used (check in PoC or real world scenario)
- How to Grant Rights to add Workstations to a Domain (Delegated Administration) - CTX121201 - How to Grant Rights to Manage Computer Accounts using Provisioning Services Console
- Never use power settings like hard disk power savings on Provisioning Services ( server disks )
- The following will effectively disable TSO and increase the performance substantially with XenServer 5.5 (it is enabled by default in XenServer 5.6):
- Create registry key HKLMSystemCurrentControlSetServicesxenevtchnParameters
- Create a DWORD value called SetFlags in that key and set it to 30000 hexadecimal
- Restart the Virtual Machine and test it. Hotfix CPVS51SP2E003 – described in the write cache section
NFS Storage Usage
Maximum benefit from the Read-Only vDisk Storage feature can be obtained in environments that use SAN for vDisk storage and are using client-side write-back cache (disk or RAM). In these environments, use of this feature eliminates the requirement to deploy shared or clustered file system software, reducing deployment costs and complexity, and maximizing scalability and performance.
- When running the mount IP-Address:/vol/vf000/foobar x: command to mount a NFS share, Windows translates the path into a normal UNC
- Use IPvolvf000foobar when configuring the store (as UNC Path)
VHD and Storage Alignments
The VHDs must be fixed because internal structure of dynamic VHDs is different and can cause alignment problems concerning disk subsystem (that is NetApp- Filer). There is a whitepaper created by NetApp covering alignment considerations (same applies to Hyper-V VHDs).
- Best Practices for File System Alignment in Virtual Environments: NetApp (March 2009 TR-3747)
- For vDisks use RAID 5 ( read-intensive) , for Write Cache use RAID 1 / RAID10 (write-intensive) enable Write Back Cache /and ensure there is a Battery Backup Unit for RAID Controller/ SAN Systems in place
When using write caching on local device HDs, CPVS51SP2E026 introduces a fix to allow for alignment by default on a 4K boundary. This is particularly applicable in a virtual environment where the local disks attached to the Virtual Machines are actually Virtual Desktop Infrastructures (VDIs) stored on Storage Area Networks (SANs). The hotfix allows for full alignment when reading and writing the cache data thereby improving the performance of the SAN.
1 or 10
- Quick and Dirty estimates:
- 5 simultaneous boot ups per spindle
- 12 simultaneous logons per spindle
- 14 simultaneous logoffs per spindle
- 18 simultaneous users per spindle
IOPS calculations impacted by:
- Disk speed
- RAID level
- Read/Write % (20/80)
- User Activity
Virtual desktops are WRITE intensive (not READ)
- 20% Read
- 80% Write
- Requires RAID that supports heavy writes
- RAID 1 for 2 disks
- RAID 10 (1+0) for 4+ disks
- Provisioning services vDisk storage
Write Cache Storage Location
Virtual Desktops per Spindle
Spindles required for 60 desktop loads
- Start with write cache on Provisioning Services Server
- Start with initial size of 2 GB + swap file
- If target device write cache is chosen, available space must be sufficient or OS might display blue screen
- Minimize network impact (limit number of hops)
- Do NOT use Provisioning Services server
- Use RAM with extreme caution (if you run out of RAM you do not have time to restart the machine before it becomes unresponsive and you would not get a warning, just a BSOD).
- Use local disk (shared/local)
Note: If you do not have enough space when using RAM cache, the target device stops. If there is not enough local storage for client side cache (local disk/SAN) then if it fills up, it will degrade performance similar to your local drive filling up on a laptop, but it will not stop.
Calculating the proper size for the write cache
- Start with write cache on Provisioning Services Server to get some information on write cache – start with initial size of 2 GB + pagefile.
Note: Hyper-V requires additional space for the memory save file
- Pagefile is written to target device partition where write cache is located, if target device write cache is chosen, available space must be sufficient.
Note: If there is not enough space on the target device’s local drive while starting, the cache will be sent to the server. This only occurs when it determines where to place the local cache and does not find a suitable sized partition locally.
- To enable a Target Device leveraging locally attached disk for write caching, it is necessary to enable the Windows Auto Mount functionality. This can be done by completing either of the following options:
- Open up command line, start “Diskpart”
- type “automount enable”
- Open command line
- Execute Mountvol /E
Write Cache Considerations
R/W ration dependent on the environment/load, usually more writes
# of reads likely to go up the longer uptime of the target
- RAID 1 or 10 is ok, RAID 5 or 6 *not* recommended (unless a huge amount of spindles)
- Usually local disk system, and in virtualized environments NFS, iSCSI, or FC
- If using server side caching, use multiple write cache paths to increase performance
- RAID controller with battery backed write cache can help a lot
Remember to set check registry setting (with streamed server OS only):
CTX126042 – When to Enable Intermediate Buffering for Local Hard Drive Cache
Losing the write cache will cause a BSOD in most cases (might fail over to server side)
- Things that causes write cache activity to be high
- Boot / Shutdown / User logging in or off
- User starting application (streamed or local, hosted should have minimal effect)
- Application behavior
- Windows Perfmon <Physical Disk Disk Writes/sec> ( Disk Transfers / sec gives you the whole picture)
- Hotfix CPVS51SP2E003 describes how to enable file buffering for Write Cache if Write Cache size < vDisk size when using Target Device Cache (it has been noticed performance gain up to 350% for Write Cache throughout)
Note: For additional information, refer to:
CTX125126 - Advanced Memory and Storage Considerations for Provisioning Services
CTX119286 - Provisioning Server High Availability Considerations
CTX128645 - Design Considerations for Virtualizing Provisioning Services
- Read-only, unless updating:
- Provisioning Services 6.x: create a maintenance version
- Create a separate set of disks for write cache to better optimize the I/O load
- NetApp PAM (Performance Acceleration Module)
- Use a disk subsystem that causes the Windows Server to cache the vDisk
- Not NFS or Windows 2008 R2
- Provisioning Services does not take lightly loosing the vDisk connection, use multipathing
- RAID controller read cache can help, especially in larger Provisioning Services farms:
- All servers must read at least one time
- How many vDisks will be heavily utilized at one time (how many targets with different vDisks are started at the same time)
- Windows Perfmon <Physical Disk Disk Reads/sec> (This gives an idea on the throughput needed)