AppV 5 - Unable to launch applications but other users can launch without issue

We have AppV 5 SP2 HF4 installed on a Citrix PVS server with our AppV PackageInstallationRoot redirected to a static drive.  I recently updated the PVS server than found my account could not launch AppV applications but other accounts could launch them without issue.  I deleted my user account from the server, tried launching applications with /appvve (which appeared to work but didn't actually initialize the AppV environment) and procmon'ed the launch of the application.  Procmon showed me the application actually launching, but immediately terminating.

Bad Launch

Good Launch

We can see that the application stops launching at the moment it initializes the virtual environment.  Next we'll dive into the event logs and see if it gives us any more relevant information.  I chose to enable a few event logs that deal with the early process of AppV5.  I can't really tell you why I chose those ones, they are just a few that I've learned to monitor when I have problems launching apps.

I enabled:
Client-Integration
Client-Orchestration
Client-Vemgr
Client-VirtualizationManager
Subsystem-Venv

Launching the application gave me some error messages.  From Client-VirtualizationManager:

64 notification failed with error 5746736078216233004. Package {49bda935-0a17-43e5-aada-44858d95affa}, version {d506ff21-4e9a-40e5-b60f-94481ccbe76a}, pid 5176, ve id 0.
VirtualizationManager component failed to handle 46 activity. Error 4746865492684177448.

If we do the backwards HRESULT dance to try and get a useful error code we get this for the two errors:

PS C:\Users\trententtye> (5746736078216233004).ToString("x")
4fc082040000002c

PS C:\Users\trententtye> (4746865492684177448).ToString("x")
41e0410400000028

According to the link, the object is 04 (

04

Virtualization Manager

Client-VirtualizationManager Log or Client-Vemgr Log

)

with error code 0x28 and 0x2c.  Googling those HRESULT codes reveals nothing.  I then looked into Client-Vemgr and found some error messages there:

Request to generate mappings for globally-published package failed. package moniker 49BDA935-0A17-43E5-AADA-44858D95AFFA_D506FF21-4E9A-40E5-B60F-94481CCBE76A. group moniker . user sid S-1-5-21-38857442-2693285798-3636612711-15138285. result {Operation Failed}
The requested operation was unsuccessful..

Well, this is interesting.  So it appears that AppV can't do mappings; which I assume are the CoW (Copy on Write) for the filesystem...?  Procmon does not reveal anything with a access denied or failed to create directories, as a matter of fact it actually creates the %userprofile%\local\Microsoft\AppV\VFS directories.  But it is specifically targeting my user account, as referenced by the SID.

At this point I scanned Procmon for my SID and found it existed in the AppV registry path here:

HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\AppV\Client\Virtualization\LocalVFSSecuredUsers

Procmon did not notice anything wrong (no access denied or such error messages).  I then launched another application with a runas and saw it that a new registry key SID combination was dynamically generated.  I then suspected that maybe the existence of this key maybe causing my issue...  To verify I rebooted the PVS server and without using my account, checked and saw the registry key was there through reboots.  My next test was to delete the key and try launching the application again.

Lo and behold it worked!  AppV 5 regenerated the key and launched the application.  It appears if the key exists then it will fail to create the CoW mappings in the registry (which you can see in the screenshot when it's creating the MAV keys) and my issue is resolved.