I may have found out why it was happening in our environment. We are using Provisioning Services and with it we are using two NIC's, one for the Provisioning Services and one for Standard networking.
It appears the XTE service became configured to use the Provisioning Services NIC. This was verified in the httpd.conf in the C:\Program Files (x86)\Citrix\XTE\conf folder.
 |
| Provisioning NIC and Production (network) NIC |
 |
| httpd.conf as was when the system booted (and non-functional) |
When I traced the XTE service using procmon.exe and wireshark with this non-functional conf this is what I saw when I attempted to launch the application:
 |
| You can see it attempt to connect to itself via 1494 but then nothing else happens |
 |
| Wireshark shows virtually nothing on the network and nothing related to IMA |
When I edited the file to have the Production NIC...
then restarted the XTE service and retraced via Procmon and Wireshark...
We now see tons of activity and the application now launches without issues.
================EDIT===============
We have now found why we are getting this error, and why we are getting it intermittently. The issue is we are using PVS with multi-homed NIC's. One NIC (LanAdapter 1) is the "Provisioning" network, and the second NIC (LanAdapter 2) is the "Production" network. The Provisioning network is on a completely seperate vLan and sees no traffic outside of it's little network. The ICA Listener was attaching itself to the Provisioning network instead of the production network, so when we tried to connect to the server it would fail with the CGP tunnel error because the outside network cannot talk to the Provisioning network. To attempt to resolve this issue one of our techs (Saman) created a group policy preference registry key that set the following value (HKEY_LOCAL_MACHINE\SYSTEM\ControlSet001\Control\Terminal Server\WinStations\ICA-TCP - LanAdapter):
By setting it to "2" we could ensure the ICA listener is always listening on LanAdapter 2, our production network. Unfortunately, a Windows Update appears to have caused either Group Policy Registry Preferences to execute (sometimes) *after* the IMAService service started, or allowed the IMAService service to start *before* Group Policy Registry Preferences. IMAService will recreate that file every second restart. To resolve this issue I created a startup script that executes after 65 seconds, deleting the httpd.conf file and restarting the appropriate services until the httpd.conf file is recreated.
In my testing it appears you need to restart the "IMAService" service twice to get it to recreate the httpd.conf file. Because of this, I created the script to retry up to 3 times to try and regenerate the file.
:: ===========================================================================================================
::
:: Created by: Trentent Tye
::
::
::
:: Creation Date: May 28, 2013
:: Modified Date: May 28, 2013
::
:: File Name: Citrix_Restart_IMASrv_Delayed.cmd
::
:: Description: This script fixes the CGP Tunnel issue where the IMASrv.exe starts before
:: group policy prefencess start. This causes the httpd.conf file in the
:: XTE folder to have the wrong IP and the ICA client listener to actually
:: be listening on the wrong NIC. To resolve this issue we will ensure
:: group policy is applied then restart the IMASrv service *twice*.
:: We have to do it twice because the IMASrv won't recreate the httpd.conf
:: file on the first restart.
::
:: ===========================================================================================================
@ECHO OFF
CLS
SET COUNT=0
eventcreate /ID 1 /L APPLICATION /T INFORMATION /SO "Local GP Startup Script" /D "Starting Citrix_Restart_IMASrv_Delayed.cmd script"
del /q "C:\Program Files (x86)\Citrix\XTE\conf\httpd.conf"
ECHO N | GPUPDATE /FORCE >NUL
ping 127.0.0.1 -n 65 >NUL
:RetryCreate
net stop CitrixWMIService
net stop IMAService
net stop CitrixXTEServer
ping 127.0.0.1 -n 5 >NUL
net start IMAService
net start CitrixWMIService
ping 127.0.0.1 -n 5 >NUL
net stop CitrixWMIService
net stop IMAService
ping 127.0.0.1 -n 5 >NUL
net start IMAService
net start CitrixWMIService
net start CitrixXTEServer
IF %COUNT% GEQ 3 EXIT
SET /A COUNT=%COUNT%+1
IF NOT EXIST "C:\Program Files (x86)\Citrix\XTE\conf\httpd.conf" GOTO RetryCreate
eventcreate /ID 1 /L APPLICATION /T INFORMATION /SO "Local GP Startup Script" /D "Completed Citrix_Restart_IMASrv_Delayed.cmd script"
================EDIT===============
We have now found why we are getting this error, and why we are getting it intermittently. The issue is we are using PVS with multi-homed NIC's. One NIC (LanAdapter 1) is the "Provisioning" network, and the second NIC (LanAdapter 2) is the "Production" network. The Provisioning network is on a completely seperate vLan and sees no traffic outside of it's little network. The ICA Listener was attaching itself to the Provisioning network instead of the production network, so when we tried to connect to the server it would fail with the CGP tunnel error because the outside network cannot talk to the Provisioning network. To attempt to resolve this issue one of our techs (Saman) created a group policy preference registry key that set the following value (HKEY_LOCAL_MACHINE\SYSTEM\ControlSet001\Control\Terminal Server\WinStations\ICA-TCP - LanAdapter):
By setting it to "2" we could ensure the ICA listener is always listening on LanAdapter 2, our production network. Unfortunately, a Windows Update appears to have caused either Group Policy Registry Preferences to execute (sometimes) *after* the IMAService service started, or allowed the IMAService service to start *before* Group Policy Registry Preferences. IMAService will recreate that file every second restart. To resolve this issue I created a startup script that executes after 65 seconds, deleting the httpd.conf file and restarting the appropriate services until the httpd.conf file is recreated.
In my testing it appears you need to restart the "IMAService" service twice to get it to recreate the httpd.conf file. Because of this, I created the script to retry up to 3 times to try and regenerate the file.
:: ===========================================================================================================
::
:: Created by: Trentent Tye
::
::
::
:: Creation Date: May 28, 2013
:: Modified Date: May 28, 2013
::
:: File Name: Citrix_Restart_IMASrv_Delayed.cmd
::
:: Description: This script fixes the CGP Tunnel issue where the IMASrv.exe starts before
:: group policy prefencess start. This causes the httpd.conf file in the
:: XTE folder to have the wrong IP and the ICA client listener to actually
:: be listening on the wrong NIC. To resolve this issue we will ensure
:: group policy is applied then restart the IMASrv service *twice*.
:: We have to do it twice because the IMASrv won't recreate the httpd.conf
:: file on the first restart.
::
:: ===========================================================================================================
@ECHO OFF
CLS
SET COUNT=0
eventcreate /ID 1 /L APPLICATION /T INFORMATION /SO "Local GP Startup Script" /D "Starting Citrix_Restart_IMASrv_Delayed.cmd script"
del /q "C:\Program Files (x86)\Citrix\XTE\conf\httpd.conf"
ECHO N | GPUPDATE /FORCE >NUL
ping 127.0.0.1 -n 65 >NUL
:RetryCreate
net stop CitrixWMIService
net stop IMAService
net stop CitrixXTEServer
ping 127.0.0.1 -n 5 >NUL
net start IMAService
net start CitrixWMIService
ping 127.0.0.1 -n 5 >NUL
net stop CitrixWMIService
net stop IMAService
ping 127.0.0.1 -n 5 >NUL
net start IMAService
net start CitrixWMIService
net start CitrixXTEServer
IF %COUNT% GEQ 3 EXIT
SET /A COUNT=%COUNT%+1
IF NOT EXIST "C:\Program Files (x86)\Citrix\XTE\conf\httpd.conf" GOTO RetryCreate
eventcreate /ID 1 /L APPLICATION /T INFORMATION /SO "Local GP Startup Script" /D "Completed Citrix_Restart_IMASrv_Delayed.cmd script"
No comments:
Post a Comment