Quantcast
Channel: Jan Hendriks Blog
Viewing all 19 articles
Browse latest View live

(Initial) StoreFront Configuration Replication Fails with error 31

$
0
0

In one of my last projects I had an interesting problem with Storefront. Neither the initial configuration replication nor following ones worked. 2014-01-10 09_51_04-storefront01 (24.4.21) on W2012-LENOVO - Virtual Machine Connection

On the first StoreFront Server the following error was logged:

Log Name:      Citrix Delivery Services
Source:        Citrix Configuration Replication Service
Event ID:      31

An error has occured during the all server configuration update process.
Citrix.DeliveryServices.ConfigurationReplication.Exceptions.ServerUpdateConfigurationException, Citrix.DeliveryServices.ConfigurationReplication, Version=2.4.0.0, Culture=neutral, PublicKeyToken=e8b77d454fa2a856
An error occured running the command: ‘Add-DSFeatureInstances’
RemoteEndpoint: net.tcp://storefront02/Citrix/ConfigurationReplication

2014-01-10 09_51_27-storefront01 (24.4.21) on W2012-LENOVO - Virtual Machine Connection

On the second Server I found the following three errors:

Log Name:      Citrix Delivery Services
Source:        Citrix Configuration Replication Service
Event ID:      1

An error occured running the command: ‘Add-DSFeatureInstances’
Exception of type ‘Citrix.DeliveryServices.Framework.Web.Deployment.Exceptions.WebApplicationAlreadyExists’ was thrown.
Citrix.DeliveryServices.PowerShell.Command.Runner.Exceptions.PowerShellExecutionException, Citrix.DeliveryServices.PowerShell.Command.Runner, Version=2.4.0.0, Culture=neutral, PublicKeyToken=e8b77d454fa2a856
An error occured running the command: ‘Add-DSFeatureInstances’

2014-01-10 09_52_37-storefront02 (24.4.22) on W2012-LENOVO - Virtual Machine Connection

Log Name:      Citrix Delivery Services
Source:        Citrix Configuration Replication Service
Event ID:      12

Failure to notify of configuration update.
Citrix.DeliveryServices.PowerShell.Command.Runner.Exceptions.PowerShellExecutionException, Citrix.DeliveryServices.PowerShell.Command.Runner, Version=2.4.0.0, Culture=neutral, PublicKeyToken=e8b77d454fa2a856
An error occured running the command: ‘Add-DSFeatureInstances’
   at Citrix.DeliveryServices.PowerShell.Command.Runner.PowerShellCommandRunner.RunCommand(IPowerShellCommand command)
   at Citrix.DeliveryServices.ConfigurationReplication.PowerShell.ConfigurationWriterBL.AddFeatureInstances(List`1 instancesToCreate, List`1 featureClasses)
   at Citrix.DeliveryServices.ConfigurationReplication.BL.Synchronisation.ConfigurationSync.SynchroniseInstances(FeatureInstancesInfo instances)
   at Citrix.DeliveryServices.ConfigurationReplication.BL.Synchronisation.ConfigurationSyncController.Update(Guid primarySynchronisationLevel)
   at Citrix.DeliveryServices.ConfigurationReplication.BL.Synchronisation.ConfigurationSyncController.UpdateHostConfiguration(Guid synchronisationLevel, IConfigurationSync sync, IConfigurationReplication proxy, IPersistServiceState store)
   at Citrix.DeliveryServices.ConfigurationReplication.BL.Synchronisation.ConfigurationSyncController.UpdateHostConfiguration(Guid synchronisationLevel, String proxyEndpoint)
   at Citrix.DeliveryServices.ConfigurationReplication.WCF.ConfigurationReplication.<>c__DisplayClass3.<BeginUpdateConfiguration>b__1(Object asr)

2014-01-10 09_53_19-storefront02 (24.4.22) on W2012-LENOVO - Virtual Machine Connection

Log Name:      Citrix Delivery Services
Source:        Citrix Configuration Replication Service
Date:          10.01.2014 09:50:53
Event ID:      19

Failed to get the end status of the sever configuration update.
System.ServiceModel.FaultException, System.ServiceModel, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089
An error occured running the command: ‘Add-DSFeatureInstances’
   at Citrix.DeliveryServices.ConfigurationReplication.WCF.ConfigurationReplication.EndUpdateConfiguration(IAsyncResult asyncResult)

2014-01-10 09_53_57-storefront02 (24.4.22) on W2012-LENOVO - Virtual Machine Connection

If you google for problems like these you find hints like:

  1. Delete some specific files on the secondary StoreFront Server and synchronize again
  2. Remove StoreFront from the secondary StoreFront Server – don’t forget do manually delete C:\inetpub\wwwroot\Citrix and C:\ProgramFiles\Citrix\Receiver StoreFront – Remove second StoreFront Server from StoreFront Group – Reinstall second StoreFront Server and join again
  3. Uninstall StoreFront on both Servers – remove folders mentioned in two – reinstall StoreFront and create a new StoreFront Group

None of them worked for us – but if you read the first Event on the secondary StoreFront Server carefully you already get a got hint why the replication fails:

‘Citrix.DeliveryServices.Framework.Web.Deployment.Exceptions.WebApplicationAlreadyExists’

Though we also checked the StoreFront Log-Files in C:\ProgramFiles\Citrix\Receiver StoreFront\Admin\logs and then we found an interesting entry in the file Add-DSWebReceiver :

There is already a web app at 1 /Citrix/StoreWeb
WebReceiverInstance:Deploy failed due to exception: Citrix.DeliveryServices.Framework.Web.Deployment.Exceptions.WebApplicationAlreadyExists: Exception of type ‘Citrix.DeliveryServices.Framework.Web.Deployment.Exceptions.WebApplicationAlreadyExists’ was thrown.

Ok – we removed StoreFront (on the second Server) again completely (with all files) and opened the IIS-Management-Console:
2014-01-10 10_00_41-storefront02 (24.4.22) on W2012-LENOVO - Virtual Machine Connection
What’s that? StoreFront is removed but there’s still a “StoreWeb” available? After checking the file-system – there was no StoreWeb availble – we removed the folder in the IIS console.

After reinstalling StoreFront and joining to the StoreFront Group the configuration synchronization worked without any problems.
2014-01-10 10_00_59-storefront01 (24.4.21) on W2012-LENOVO - Virtual Machine Connection

Hope this helps some of you if you have problems with your StoreFront synchronization.

Just one more hint at the end – StoreFront also supports a more intensive verbose logging. The usage is described in CTX139592.



Citrix Receiver: The configuration manager cannot be initialized

$
0
0

Today I would like to describe you an interesting problem I had during my last receiver deployments. The start of the installation always failed with the message:

The configuration manager cannot be initialized.

2014-02-12 22_02_41-win7-en on W2012-LENOVO - Virtual Machine Connection

After removing all old Receiver Installations the error still occurred. I cleaned up the registry and file system and removed everything which contained Citrix – but still the same error.
The next step was to use the Citrix Receiver Clean-Up Utility (CTX137494) – normally this tool removes everything related to old Receiver or Online Plugin Versions. But this time this didn’t fix the problem – after using the tool the same error was displayed like before. Time for a more intensive debug and check with procmon which files and registry keys are accessed during the setup start. And there it was – the setup tried to access the following registry key:

HKEY_CLASSES_ROOT\Installer\UpgradeCodes\9B123F490B54521479D0EDD389BCACC1

2014-02-12 22_01_03-win7-en on W2012-LENOVO - Virtual Machine Connection

I removed the complete registry key and tried again to install the receiver – this time the setup started without any error messages and I was able to install the receiver.

2014-02-12 22_14_37-win7-en on W2012-LENOVO - Virtual Machine Connection


Upgrading the Nvidia Grid Card Firmware

$
0
0

During the last days I was able to start some testings of the new XenServer vGPU feature – thanks to my colleague Benny Tritsch for allowing me to use his Test-System with a Nvidia Grid K2 Graphics Card. During my tests I had some problems so we decided to upgrade the Firmware – well but the description of the update process looks like:

1. Create Bootable USB-Stick with Linux
2. Copy Firmware to USB stick
3. Boot from USB stick and open the update file

As you can see it’s a realy short description. Furthermore I had some problems during the update process – so I would like to share with you how I was able to upgrade the Firmware:

1. Download the Universal-USB-Installer
2. Download the Ubuntu Desktop Version (I used 12.04.4)
3. Open the Universal-USB-Installer and agree to the license
2014-02-21 08_39_17-Universal USB Installer 1.9.5.2 Setup

4. Choos Ubuntu as the Linux distribution, the downloaded iso and the USB-Stick to be used for creating the bootable USB stick
WARNING: All data on the USB stick will be removed!
2014-02-21 11_06_43-Universal USB Installer 1.9.5.2 Setup

5. After starting the USB installation with “Create” you need to confirm the start.
2014-02-21 11_06_54-Muster-DB01 on W2012-LENOVO - Virtual Machine Connection

6. When everything is done you can see the following message:
2014-02-21 11_46_25-Universal USB Installer 1.9.5.2 Setup

7. Now you need to copy the Firmware Files to a second USB stick. You should have two files – one named gridpro-update_R1.0.run and the other one named gridpro-update-auto_R1.0.run.
[Update]
You can also copy the files to the root directory of the first USB stick. Then they will be available on /cdrom after booting”. Thanks to @shawnbass
[/Update]
8. The next step is to boot server (which contains the Grid Card) from the just created USB stick. You don’t need to install Ubuntu – so choose "Try Ubuntu” to start Ubuntu directly from the stick.
2014-02-20 07_47_26-DVI2USB Solo [D3U51862]  - 1440x900 60.2 Hz 

9. After Ubuntu is started you attach the second USB stick and copy the Firmware files to Documents.
2014-02-20 08_34_10-DVI2USB Solo [D3U51862]  - 1440x900 60.2 Hz

10. Open the properties of both files and change all “Access” types to “Read and write”. Furthermore activate “Allow executing files as a program”.
2014-02-20 08_44_04-DVI2USB Solo [D3U51862]  - 1440x900 60.2 Hz

11. Time to switch to the console – press the upper left icon and enter “Ter” to find the “Terminal”.
2014-02-20 08_16_36-DVI2USB Solo [D3U51862]  - 1440x900 60.2 Hz

12. Inside the Terminal you first switch to the root account (sudo su) and then you open the folder Documents (cd Documents). To start the update you enter “./gridpro-update_R1.0.run” (without quotation marks). Alternatively you can enter “./gridpro-update-auto_R1.0.run”. The second one runs an autoupdate without userinteraction while the first asks you for every update if you would like to install the update.
2014-02-20 08_45_11-DVI2USB Solo [D3U51862]  - 1440x900 60.2 Hz

13. If you didn’t choose the Auto-Update-Version confirm the updates (dependen on the installed version it’s possible that you are asked more than once) with “y”.2014-02-20 08_45_31-DVI2USB Solo [D3U51862]  - 1440x900 60.2 Hz

14. After all updates are installed the following message is displayed:
Update successful.
No more matches found.

2014-02-20 08_47_58-DVI2USB Solo [D3U51862]  - 1440x900 60.2 Hz

15. That’s it – shut down Ubuntu – remove the USB sticks and start your normal OS.

PS:
Until now I only found the Firmware Updates on the IBM-Website….


Citrix XenDesktop / XenApp 7.x PowerShell Documentation Script

$
0
0

Hi everyone,

I think you all know the absolute fantastic PowerShell Documentation Scripts for Citrix XenApp (up to 6.5) and PVS created by Carl Webster. With these scripts it’s absolutely easy to create a Documentation of a XenApp- or PVS-Farm. Unfortunately Carl didn’t create (or release) a documentation script for XenDesktop / XenApp 7.x yet, so I decided to create my own script. It’s not comparable (and actually that’s not my intention) to Carls scripts but documents the following things (in a more basic way):

  • General Site Informations
  • Logging Settings
  • Licensing
  • Controllers
  • Databases
  • Administrators
  • Machine Catalogs
  • Applications
  • Storefront Servers (not the StoreFront Servers)
  • App-V Server

The following main things are missing:

  • Policies
  • Hosting

The script is divided into two files:
jhmeier-xd7-xa-7.x-documentation-v03.ps1
en-US.xml

The first file is the script himself, while the second file is for an easy modification of the script-output for other languages.

Script Usage

To use the scripts a few prerequisites must be fulfilled:

  • XenDesktop / XenApp PowerShell Extensions
  • Microsoft Word
  • XenDesktop / XenApp Admin Account (with at least Read privileges)
  • PowerShell Execution Policy
  • The script and the language file need to be in the same location.

That should it be – just execute the script. The script starts and creates an (invisible) Word-Document in the background. When it’s finished it saves the document with the name XD-XA-SITE-NAME.docx in the script folder.

Parameters

The script supports (but doesn’t need) the following parameters:
-TextLanguage en-US
If you don’t want to have an output in the default language (English) you need to define the wanted language. Attention: Therefore you need a separate language xml file – see Language Files.
-DeliveryControllerAddress DeliveryControllerFQDN
If you run the script without this parameter it tries to connect to localhost – but if that’s not a Delivery Controller you need to specify it

Language Files

If you run the script – without creating a language specific xml file – the output looks like this:
Description: Description in the System language
If that’s ok for you – you have nothing to do – but if you prefer a complete output in your language like (in German):
Beschreibung: Beschreibung in Deutsch
you need to copy the file en-US.xml and rename it to e.g. de-DE.xml. After that open the new file with an editor of your choice. You now find a lot of entries which look like the following:
<Description>Description</Description>
Now you need to translate the middle for every necessary line so that it looks like this:
<Description>Beschreibung</Description>

License

The script is published under the GPL.

Download

jhmeier-xd-xa-7.x-documentation-v03.zip

Feedback

Any feedback would be really welcome – if you have something to make the script better or have any questions – feel free to contact me.

That’s it – hopefully the script is useful for some of you.


IT-Administrator Article – Citrix XenMobile

$
0
0

I have written another article for the print magazine IT-Administrator. This time it’s an article about Mobile Device and Mobile Application Management with Citrix XenMobile. You can find the full article in the May edition.

ita201405-titel

Mobile Geräte drängen in die Unternehmen. Kein Wunder, dass die großen Software-Anbieter sich immer stärker diesem Thema widmen. Seit der Übernahme von Zenprise hat sich auch bei Citrix im Bereich Mobile viel getan. So konnte Citrix das Mobile Device Management ursprünglich nicht bedienen. Im neuen XenMobile ist dies nun möglich. Das ehemalige Cloud Gateway hat Citrix gleich mit hineingepackt. Doch welche Editionen enthalten welche Funktionen und wofür benötigen Sie die einzelnen Komponenten? Diese Fragen klären wir für Sie in diesem Beitrag.


Updated Citrix XenDesktop / XenApp 7.x PowerShell Documentation Script

$
0
0

I have updated the XenDesktop / XenApp 7.x PowerShell Documentation Script. Next to a small bugfix I have added the Documentation of the Hosting-Connections. Thus the following things are now documented:

  • General Site Informations
  • Logging Settings
  • Licensing
  • Controllers
  • Databases
  • Administrators
  • Machine Catalogs
  • Applications
  • Hosting
  • Storefront Servers (not the StoreFront Config)
  • App-V Server

The following main thing is missing:

  • Policies

Files

The script is divided into two files:
jhmeier-xd7-xa-7.x-documentation-v04.ps1
en-US.xml

The first file is the script himself, while the second file is for an easy modification of the script-output for other languages.

Script Usage

To use the scripts a few prerequisites must be fulfilled:

  • XenDesktop / XenApp PowerShell Extensions
  • Microsoft Word
  • XenDesktop / XenApp Admin Account (with at least Read privileges)
  • PowerShell Execution Policy
  • The script and the language file need to be in the same location.

That should it be – just execute the script. The script starts and creates an (invisible) Word-Document in the background. When it’s finished it saves the document with the name XD-XA-SITE-NAME.docx in the script folder.

Parameters

The script supports (but doesn’t need) the following parameters:
-TextLanguage en-US
If you don’t want to have an output in the default language (English) you need to define the wanted language. Attention: Therefore you need a separate language xml file – see Language Files.
-DeliveryControllerAddress DeliveryControllerFQDN
If you run the script without this parameter it tries to connect to localhost – but if that’s not a Delivery Controller you need to specify it

Language Files

If you run the script – without creating a language specific xml file – the output looks like this:
Description: Description in the System language
If that’s ok for you – you have nothing to do – but if you prefer a complete output in your language like (in German):
Beschreibung: Beschreibung in Deutsch
you need to copy the file en-US.xml and rename it to e.g. de-DE.xml. After that open the new file with an editor of your choice. You now find a lot of entries which look like the following:
<Description>Description</Description>
Now you need to translate the middle for every necessary line so that it looks like this:
<Description>Beschreibung</Description>

License

The script is published under the GPL.

Download

jhmeier-xd-xa-7.x-documentation-v04.zip

Feedback

Any feedback would be really welcome – if you have something to make the script better or have any questions – feel free to contact me.


Book: Citrix XenDesktop 7.5 and XenApp

$
0
0

51G-KY1yyuL

In the last week I finally received a copy of the book  about Citrix XenDesktop and XenApp 7.5 which Nico Luedemann and I have written.

It’s an absolutely awesome feeling to see the printed book after a lot of work (and time of waiting). I can only suggest everyone to try and write his own book – it’s a really interesting experience.

A big thank you goes to my family for beeing so patient with me while I was spending such a lot of time for writing the book.

Many thanks also go to Sebastian Kestel from Galileo Computing – thanks for the good collaboration during the whole project.

I hope you all enjoy reading the book – if you have any questions feel free to contact me. The book is published by Galileo Computing. You can buy it as a print copy or ebook.

XenDesktop 7.5 and XenApp


Auto Fill User Start Menu with Citrix Receiver 4.x

$
0
0

One of the biggest challenges in a lot of migrations to Citrix Receiver 4.x (and XenDesktop / XenApp 7.x) is the (missing) function to automatically fill the users start menu. In the past it was quite easy to push all published applications for the user into his start menu using the Citrix Receiver Enterprise (aka. Online Plugin). Citrix decided to stop the development of the Citrix Receiver Enterprise and fully switched to the Citrix Receiver.
But how can we now automatically fill the user start menu without any user interaction?

First of all you need to make sure that the Receiver Single-Sign-On is working. If you don’t know how to configure this read the article “Citrix Receiver Single-Sign-On (Pass-through Authentication) does not work with StoreFront”.

Also it is necessary to configure the Receiver automatically with your StoreFront Store. You can either achieve this by creating a GPO (with the Receiver adm) or using another command line parameter (next to /includeSSON for Single-Sign-On). To use a GPO import the Receiver ADM-File into the GPO. You can find the ADM-File on a computer with an installed Receiver in the following path:
C:\Program Files (x86)\Citrix\ICA Client\Configuration   (or C:\Program Files\….)

After importing the icaclient.adm you find the following settings inside the GPO:
Computer Configuration => Administrative Templates => Classic Administrative Templates (ADM) => Citrix Receiver => Storefront => Storefront Accounts List
2014-09-03 08_39_59-RDS - DEDAM-SV421 - Royal TS

Change the setting to enable and enter the Store (account) details under “Show”.
2014-09-03 08_41_11-RDS - DEDAM-SV421 - Royal TS

Enter your StoreFront Store in the following format:
STORE NAME;STOREURL;ON;STORE DESCRIPTION
2014-09-03 08_40_36-RDS - DEDAM-SV421 - Royal TS

Alternatively you can add the following parameter to your Receiver-Installation:
STORE0="STORE NAME;STORE URL;On;STORE DESCRIPTION"

The next step is to add the following registry keys under the path HKLM\SOFTWARE\Wow6432Node\Citrix\Dazzle (if it’s a 32Bit-System without Wow6432Node):

Name Type Data
InitialRefreshMinMs REG_SZ 1
InitialRefreshMaxMs REG_SZ 1
DontWarnOfRemovedResources REG_SZ True

The last one (DontWarnOfRemovedResources) is optional – if you activate this a user is not informed if one of his applications is removed.
For adding the keys the following commands can be used:

REG ADD HKLM\SOFTWARE\Wow6432Node\Citrix\Dazzle /f /v InitialRefreshMinMs /t REG_SZ /d 1
REG ADD HKLM\SOFTWARE\Wow6432Node\Citrix\Dazzle /f /v InitialRefreshMaxMs /t REG_SZ /d 1
REG ADD HKLM\SOFTWARE\Wow6432Node\Citrix\Dazzle /f /v DontWarnOfRemovedResources /t REG_SZ /d True

Don’t forget to change the path on a 32Bit-System.
2014-09-02 11_26_27-dedam-wv210 CTX Test - Copy

Now you need to create a new Shortcut in the Startup Folder. With this Shortcut the Receiver automatically connects to Storefront and receives the subscribed applications of the user after he logged in. For every application a Shortcut is created in the Start Menu. The Shortcut gets the following settings:

Target:
"C:\Program Files (x86)\Citrix\SelfServicePlugin\SelfService.exe" -logon -poll
Start in:
"C:\Program Files (x86)\Citrix\SelfServicePlugin\"

2014-09-08 08_12_21-dedam-wv210 - Remote Desktop Connection

If your users now log in not all of their available applications won’t be published to the Start Menu. The reason for this is that the user first needs to subscribe all of his available applications. The good thing is: You can also automate this Smiley

Switch to your Delivery Controller and start the Desktop Studio. Now open  “Delivery Groups => Applications” and change the Settings of one Application. You need to change the “Description and keywords”. Either enter
KEYWORDS:mandatory
or enter
KEYWORDS:auto
In this situation both keywords have the same effect – after the user logs on and the SelfService is triggered with –logon and –poll every application is subscribed for the user. Of course only applications with a configured Keyword are subscribed.2014-09-09 10_16_04-RDS - DEDAM-SV421 - Royal TS

The difference between the keywords is that with “auto” the application is only once subscribed – if the user removes it from his applications inside the Receiver it stays removed. If the keyword is “mandatory” the application is subscribed with every user logon. Don’t forget to enter the Keyword for every application which should be automatically published to the Start Menu.

That’s it – now Applications are automatically pushed to the user start menu after he logged in – if anything is unclear feel free to contact me.



(Undocumented) Issues when Upgrading to XenDesktop / XenApp 7.6

$
0
0

During the last days I tested the Upgrade from XenDesktop 7.5 to XenDesktop 7.6. I found some interesting “Issues” during the upgrade which seem to be not documented.

Upgrading StoreFront on a separate System

If you upgrade StoreFront on a separate Server and use the Upgrade Wizard you receive the following message:
2014-10-07 11_45_22-RDS - DEDAM-SV361 - Royal TS

As you can see the Upgrade process try’s to check something which is not available on a standalone StoreFront system. Luckily you can ignore the warning and continue with your installation. Actually StoreFront even doesn’t check the licenses later – so even if no licenses are available StoreFront works.

Upgrading Citrix License Server on a separate System

The more confusing message is shown if you use the Upgrade Wizard to upgrade the Citrix License Server which is not installed on a Delivery Controller. If you try to upgrade the following message is displayed:

2014-10-07 14_56_37-RDS - DEDAM-SV367 License Server - Royal TS
This product requires a newer license. You must install a license with a Subscription Advantage date of “2014.0815” or later.

Even if you have licenses with a Subscription Advantage date after “2014.0815” the message is displayed. Luckily you can also safely ignore this message (of course only if your Subscription Advantage date is after 2014.0815) – your users will still be able to access their Desktops and Applications.


Windows 2012R2 stops at Boot screen while Network Boot is activated for Citrix Provisioning Server

$
0
0

During my last Citrix Provisioning Server deployments I had an interesting problem. The Windows 2012R2 Master Template always stopped at the Boot screen. This happened even before it was converted into a PVS Device (Disk) nor the Target Device Tools had been installed.

The VM was running on VMWare ESXi 5.5 and we used Citrix Provisioning Server 7.6. After changing the first boot device to Network – though that an initial connection to Citrix PVS happened – the boot stopped while showing the Windows Boot screen. Although we were still booting from the local hard drive – so it didn’t make any sense why this was happening.
2014-10-13 08_25_32-DEDAM-CTXSV0011

After some more testing we discovered that this was not happening with all Windows 2012R2 VMs – only with new deployed ones. Ok let’s check where the difference between the VMs are.

New VMs where running in the ESXi Compatibility mode 5.5 (VM-Version 10).
2014-10-15 12_49_19-vSphere Web Client

While old VMs where still running in the ESXi Compatibility mode 5.0 (VM-Version 8).
2014-10-15 12_52_16-vSphere Web Client

After checking what has changed we saw the new VMs were using a SATA-Controller for the CD-/DVD-Drive instead of an IDE-Controller.

Old VM (Version 8):
2014-10-15 12_56_00-vSphere Web Client

New VM (Version 10):
2014-10-15 13_00_00-vSphere Web Client

Though we removed the CD-/DVD-Drive and SATA Controller from a new VM and added the CD-/DVD-Drive using an “old” IDE-Port.
2014-10-13 08_27_33-vSphere Web Client

After this change the VM was booting without any problems.
2014-10-13 08_30_00-DEDAM-CTXSV0011

I don’t know what causes this problem – but hopefully it will be fixed from either Citrix or VMWare in one of their next versions.


Citrix Receiver 4.2 Startmenu integration

$
0
0

After a long time of waiting Citrix released Receiver 4.2. This release brings back functionality to easily integrate Published Applications into the Startmenu known from the Receiver Enterprise. There are two ways available to configure the new Options.

  1. Parameters during the installation and Registry entries
  2. Group Policy with imported ADM files

With both you can disable the SelfServiceStore, configure where and how Startmenu / Desktop Icons are created and if they are removed at logoff / exit.

Installation Parameters / Registry Setting

To disable the SelfServiceStore you must start the installation with the parameter /SELFSERVICEMODE=False . Furthermore it’s a good idea to also enable SingleSignOn ( /includeSSON) and configure the Store ( STORE0=”JHMeier;https://storefront.jhmeier.local/Citrix/Store/discovery;on;JHM Store”) through Parameters . Otherwise the user needs to manually configure the Store and SingleSignOn wouldn’t work.

2014-12-03 13_28_00-dedam-ctxsv0013 win_2008r2-standalone-test on dedam-esx-1-3.corp.grimme.com

CitrixReceiver.exe /includeSSON /SELFSERVICEMODE=False STORE0=”JHMeier;https://storefront.jhmeier.local/Citrix/Store/discovery;on;JHM Store”

If you need to remove the created shortcuts at logoff or exit of the Receiver you must configure the following Registry-Keys:

Remove Shortcuts at Logoff:

HKML\SOFTWARE\Citrix\Dazzle
RemoveAppsOnLogOff
Value = True

Remove Shortcuts at Receiver Exit

HKML\SOFTWARE\Citrix\Dazzle
RemoveAppsOnExit
Value = True

Group Policy with imported ADM files

After installing the Receiver 4.2 you find updated ADM files in the folder C:\Program Files(x86)\Citrix\ICA Client\Configuration”. Open an existing GPO or create a new one. Open Computer Configuration, right click on Administrative Templates and choose Add/Remove Templates…

2014-12-03 20_42_16-RDS - DEDAM-SV396 - Royal TS

For this configuration you need to add the icaclient.adm file – but you can also add the other available adm files.
2014-12-03 20_46_01-RDS - DEDAM-SV396 - Royal TS

After adding the file(s) open the following path:
Computer Configuration => Administrative Templates => Classic Administrative Tempaltes (ADM) => Citrix Components => Citrix Receiver => SelfService2014-12-03 20_56_50-RDS - DEDAM-SV396 - Royal TS

To disable the SelfServiceStore disable the setting Manage SelfServiceMode
2014-12-03 20_59_17-dedam-sv396 - Remote Desktop Connection

Enable the Manage App shortcut setting to configure the following settings:

  • Startmenu Directory
  • Desktop Directory
  • Disable Startmenu Shortcut
  • Enable Desktop Shortcut
  • Disable Categorypath
  • Remove apps on Logoff
  • Remove apps on Exit

2014-12-03 20_59_47-dedam-sv396 - Remote Desktop Connection

The store can be configured under Citrix Receiver => Storefront.

Results

After installing the Receiver with Parameters or configuring it through GPO Shortcuts are automatically published to the Startmenu. Furthermore the SelfServiceStore is removed – it is not possible to open it. If you right click the Receiver the following options are displayed:

2014-12-03 13_29_32-dedam-ctxsv0013 win_2008r2-standalone-test on dedam-esx-1-3.corp.grimme.com
Instead of Open only Refresh is available to check if new apps are published for the user.

Known limitation

There is one known limitation of this solution. If you enable removing the Shortcuts at logoff or exit the app enumeration after every logon is slow. To fix this two workarounds are available.

The first is to add the following registry setting:

HKLM\Software\Citrix\Dazzle
Name = ReuseStubs
Type = REG_SZ
Value = true

The other one is to create Stub-Files on a Share. These files are necessary to start a published application. To create them start a Receiver and subscribe all published applications. The files are created under %Appdata%\Citrix\SelfService. Copy the *.exe files to your share. Furthermore you need to create the following registry settings (If it’s a 64Bit system the Path is HKLM\SOFTWARE\Wow6432Node\Citrix\Dazzle):

HKLM\Software\Citrix\Dazzle
Name = CommonStubDirectory
Type = REG_SZ
Value = PATHTOSHARE (e.g. \\server\citrixstubs)

HKLM\Software\Citrix\Dazzle
Name = CopyStubsFromCommonStubDirectory
Type = REG_SZ
Value = true

After a restart of the Receiver the settings are active and the enumeration should be faster.


Installing a SSL Certificate on a Citrix NetScaler Insight Center Appliance

$
0
0

If you plan to connect your Citrix Director Installation to a NetScaler Insight Center Appliance and would like to use a HTTPS connection you need to exchange the SSL Certificate on the appliance again a trusted one.

The first step is to create a host entry for your NetScaler Insight Center Appliance (NSICA) on your internal DNS server. Then you need to create a certificate (split into a PEM and a KEY file) with the FQDN of your NSICA. After you have created the certificate open the NSICA configuration and switch to CONFIGURATION => NETSCAKER INSIGHT CENTER => SSL CERTIFICATE FILES2014-09-10 10_15_28-Citrix NetScaler Insight Center - Configuration - Internet Explorer

Click on ACTION and choose UPLOAD.
2014-09-10 10_15_52-Citrix NetScaler Insight Center - Configuration - Internet Explorer

Browse to your PEM File of the Certificate and Upload it to the NSICA.
2014-09-10 10_26_53-Citrix NetScaler Insight Center - Configuration - Internet Explorer

The PEM File should now be listed under “sdx_default_ssl_cert”.
2014-09-10 10_27_28-Citrix NetScaler Insight Center - Configuration - Internet Explorer

Switch to SSL KEYS and choose UPLOAD again.
2014-09-10 10_27_46-Citrix NetScaler Insight Center - Configuration - Internet Explorer

Browse for the KEY File of the Certificate and upload it.
2014-09-10 10_28_09-Citrix NetScaler Insight Center - Configuration - Internet Explorer

Like before the KEY File is now listed under “sdx_default_ssl_key”.
2014-09-10 10_28_30-Citrix NetScaler Insight Center - Configuration - Internet Explorer

After uploading the PEM and KEY file you need to navigate to SYSTEM => INSTALL SSL CERTIFICATE.
2014-09-10 10_29_14-Citrix NetScaler Insight Center - Configuration - Internet Explorer

Choose the just uploaded PEM and KEY File and confirm it with OK.
2014-09-10 10_30_35-Citrix NetScaler Insight Center - Configuration - Internet Explorer

After a necessary reboot the SSL Certificate is active and you can connect your Director to the NetScaler Insight Center Appliance using HTTPS.
2014-09-10 10_36_54-Citrix NetScaler Insight Center - Configuration - Internet Explorer


Citrix Receiver 4.x Store configuration through Group Policy fails

$
0
0

In one of our last deployments we tried to configure the necessary Citrix Receiver Store Configuration on our VDAs through a Group Policy. Unfortunately it was not working. The GPO was applied – but no store was configured.

The interesting part of the problem was that on other clients the same GPO was working – the user logged on and the store was configured – like it should be. We first started to remove the Citrix, run the Receiver Cleanup Utility and reinstall the Receiver. But it was still the same problem – the store was not configured automatically.

After checking the differences between the clients we found a difference in the registry. On the working client the following registry (with the Store information) was available:

HKLM\SOFTWARE\Policies\Citrix\Receiver

2014-11-07 11_52_33-OneDrive

This key was missing on the other client:

2014-11-07 11_54_05-dedam-ctxsv0009 win_2008r2-default-maintenance on dedam-esx-1-7.corp.grimme.com

We exported the key from the working client and imported it on the not working client. If now a user logged on the Store was automatically configured – like it should be.

The question now was: Why is this key missing? Time to delete the key again and run a gpupdate /force to make sure all Group Policies are applied correctly. Check the registry again and – wohoo – the key is available. So everything looked fine – until we rebooted the VDA – the key was lost again. 
Interestingly other Receiver Configurations (like disabling SelfServiceMode) through the same GPO were applied correctly.

Though it was time to check what happens during the boot with procmon. Inside the procmon log we found some interesting results – the registry key was first created from the Microsoft GPO service and then deleted by the Citrix Group Policy engine.

2014-11-07 11_54_54-dedam-ctxsv0009 win_2008r2-default-maintenance on dedam-esx-1-7.corp.grimme.com

Time to open a support ticket at Citrix. After some discussions and more testing we found the reason for the problem:
The Delivery Controllers had been configured through a GPO. This means that the VDA can only connect to the site after this GPO was applied. If the Citrix Group Policy engine starts and the VDA is not already connected to a Site every Citrix Policy is deleted. One of the deleted settings was the Receiver Store configuration (although that should be independent from other Citrix Policies). Interestingly other Receiver specific settings were not deleted.

Citrix developed a fix for this problem though that the Receiver Store configuration is not any longer deleted. Hopefully it will be integrated in the next Group Policy Client Side Extensions hotfix. If you have the same problem open a support case and ask for fix LC11637.


Citrix Receiver 4.2 – a valid replacement for Receiver 3.x Enterprise (aka Online Plugin)?

$
0
0

At the beginning of December 2014 Citrix released the Receiver 4.2. One of the biggest (and most mentioned) improvements is the better start menu integration (If you want to know how to configure it read this blog article).

The question now is: Is Receiver 4.2 a valid replacement for Receiver 3.x Enterprise?

To make it short – from my point of view it can be a replacement if you plan to use it on your fat clients and can live with some limitations – but not on published (RDS) desktops (or you need to live with bigger limitations). Let’s have a more detailed look why I think Citrix really needs to update / optimize the Receiver as soon as possible.

Fat Clients

Most of the users prefer to start their applications from the start menu – independent if it’s installed locally or published. The ideal world would look like this:
You install the Receiver, configure a GPO and the applications are directly published to the start menu – the user doesn’t need to open the Receiver and is happy. On fat clients this is working with some limitations with Receiver 4.2.

Startup Time / App Enumeration

After logging in the Receiver needs some time to start completely. Furthermore the app enumeration can take really long. There are some optimizations to speed up the App Enumeration a little bit – but only if you remove shortcuts at logoff or exit – it won’t help you if the user logs on for the first time. One way is to create the following registry key:

HKLM\Software\Citrix\Dazzle
Name = ReuseStubs
Type = REG_SZ
Value = true

Application Refresh

In the default configuration the automatic refresh (after logging) in is not always working – with Receiver 3.x Enterprise the user logged on and a refresh of the published applications happened.
To activate the same refresh under Receiver 4.2 you need to configure the following registry keys (see CTX140244):

HKLM\SOFTWARE\Wow6432Node\Citrix\Dazzle\
Name: InitialRefreshMinMs
Type: REG_SZ
Value: 1

HKLM\SOFTWARE\Wow6432Node\Citrix\Dazzle\
Name: InitialRefreshMaxMs
Type: REG_SZ
Value: 1

Shorcut Removal

As known from Receiver Enterprise 3.x the Receiver 4.2 can automatically remove the created application shortcuts. The configuration for this has moved from the Webinterface to the Receiver. The easiest way to configure this is to create a GPO with the Receiver ADM files and configure the following settings:

Computer Configuration => Policies => Administrative Templates => Classic Administrative Templates (ADM) => Citrix Components => Citrix Receiver => SelfService => Manage App shortcut

image

Sounds easy or? It is – it’s only sometimes just not working. This means if someone exits or logs off Receiver the created shortcuts are not removed.

Warning of Removed Applications

Another thing is that the user always receives a notification if an application is no longer available for him. So if you remove the assignment of an application the user receives the following message:
2015-02-02 10_47_49-dedam-wv021 M42 Packaging Dev IP on dedam-esx-2-7.corp.grimme.com

After clicking on this message the following is shown (if SelfServiceMode is disabled – otherwise the Receiver is shown and the removed application is greyed out):
2015-02-04 08_15_34-dedam-wv021 M42 Packaging Dev IP on dedam-esx-2-7.corp.grimme.com

In CTX140244 you can find the following registry:

HKLM\SOFTWARE\Wow6432Node\Citrix\Dazzle\DontWarnOfRemovedResources
Type: REG_SZ
Value: True

After reading the description I thought that this would help me to prevent the above mentioned messages:

To prevent dialog boxes when resources are removed from the server, use the following command syntax:

Well – the setting disables a message – but only the second one so that the user doesn’t need to confirm the removal. The first message is still displayed.

 

XenApp (RDS) Desktop

 

All of the above things are also valid if you use the Receiver on a published desktop – but there is one more thing….

If you have a lot of Thin Clients (and I know a lot of environments where that is the case) you usually publish a full Desktop. Some of the assigned applications are installed on the same server while others are installed on different ones. The start menu is cleared. The user only sees the applications assigned to him (published through Receiver) and not all of the installed applications.
In the past the Receiver (3.x) checked if the application is installed locally and then starts it directly while the other are started remotely (also called remoting). This was easy to handle: Install application, clear start menu, publish application.

With Receiver 4.2 this isn’t working any longer – the application is always started remotely – even if it’s installed on the local server. To fix this some configurations need to be done:

  1. Create a folder on the Server with a shortcut for every application that is installed locally (of course only for the ones you plan to publish)
  2. Create the following Registry-Key
    • HKLM\SOFTWARE\Wow6432Node\Citrix\Dazzle
    • Type: REG_SZ
    • Name: PreferTemplateDirectory
    • Value PATH TO CREATED FOLDER
  3. Change the application properties of every application that is installed on the published desktop and add the following description:
    • KEYWORDS:prefer=SHORTCUTNAME
    • Replace SHORTCUTNAME with the name of the local saved shortcut (Step 1).

Though you can see it’s a lot more to do until you have the same behavior like with Receiver 3.x Enterprise.

Furthermore you need to know that if the Receiver detects a local shortcut it copies and renames this one to the Start Menu – it’s not a new Shortcut created with the settings of the published applications. This means if you change an application property (e.g. parameters) you always need to change this in the published application and in the local saved shortcut. Not nice…

During the last tests and implementations I figured out that after configuring the above mentioned settings – to prevent remoting – only some of the applications are published from the receiver to the start menu. After checking the edocs I saw the following statement:

Before installing an application on a user’s computer, Receiver searches for the specified patterns to determine if the application is installed locally. If it is, Receiver subscribes the application and does not create a shortcut. When the user starts the application from the Receiver window, Receiver starts the locally installed (preferred) application.

Interestingly in this blog post Thomas Fuhrmann says shortcuts are automatically created:

When using such a template directory (which can also be a network share) Citrix Receiver first checks in thatdesignated folder for a matching shortcut and then copies this shortcut to the start menu.

Sounds like even Citrix doesn’t know if shortcuts are copied – or not. During my tests shortcuts were only copied to the start menu when no application category was configured. As soon as this was configured – to structure the applications into folders – the shortcut was not longer published.
2015-01-23 13_56_29-RDS - DEDAM-SV421 - Royal TS

Until now I only found one solution for this; Remove the shortcut creation (on the published desktop) from the receiver and replace it with a simple script – until Citrix publishes a Receiver version which is working in all conditions. I am not a PowerShell Guru – so if you know how to make it better – just send me an information and I will update the script. The script first checks for existing shortcut and removes them to make sure that if an application is no longer assigned to a user it’s not visible any more. Then it creates the folders and shortcuts.

#Get Current User Informations and configure path to start menu
$CurrentUser = [System.Security.Principal.WindowsIdentity]::GetCurrent()
$WindowsPrincipal = New-Object System.Security.Principal.WindowsPrincipal($CurrentUser)
$StartMenuFolder = PATH TO USER STARTMENU

#Remove Existing Shortcut Folders
if (Test-Path "$StartMenuFolder\Programs\FOLDERNAME\" -PathType Container)
{
    Remove-Item "$StartMenuFolder\Programs\FOLDERNAME\" -Recurse
}

#Create Application Shortcut Example
if ($WindowsPrincipal.IsInRole("GROUP NAME") -eq "True")
{   
    if (!(Test-Path "$StartMenuFolder\Programs\FOLDERNAME\" -PathType Container))
    {
        New-Item -ItemType directory -Path "$StartMenuFolder\Programs\FOLDERNAME\"
    }
    Copy-Item "C:\SHORTCUTPATH\SHORTCUTNAME.lnk" -Destination "$StartMenuFolder\Programs\FOLDERNAME\SHORTCUTNAME.lnk"
}

Conclusion

As you can see Citrix made some improvements (compared to the first 4.X releases) to get the receiver as good as it was before – but there are still a lot of things left. Hopefully Citrix will release a nice, fast and easy configurable Receiver in the near feature (and you know – the hope dies last). Actually I don’t understand why such an important component in the whole Citrix universe doesn’t get more attention.


XenDesktop 7.6 Black Screen issue with VMWare SVGA 3D Driver

$
0
0

During the last weeks I was facing a strange black screen problem in a XenDesktop 7.6 environment. A rollout for a new division was planned. An image was created and deployed with MCS. One user was chosen to test if everything is working. During the first few days everything was working fine. But after a few days the user always saw a black screen after logging in – nothing more. Interestingly this only happened when he arrived in the morning. Though after successfully logging in it was possible to logout and login again (to another VM) without any problems. When we tried to reproduce it this was nearly impossible – only sometimes the same problem was happening with our test user. But also this only happened at the first login in the morning.

We started to search for solutions and tried the following ones:

First we activated the Legacy graphics mode and assigned it to the affected VMs.
2015-04-02 15_47_18-RDS - DEDAM-SV421 - Royal TS

Then we disabled the Desktop Composition Redirection.
2015-04-02 16_14_41-RDS - DEDAM-SV421 - Royal TS

Still the black screen occurred. So we changed the Receiver Version (a Thin Client with Linux was used) from 12 to 13 – but it was still the same. So we decided to test it with a Windows Client and Receiver 4.2. But as you expect – still the same.

So it was time for a more intensive debugging.

We changed the following registry entries to make sure there is no problem with the MFAPHOOK:

Path: HKLM\SOFTWARE\Microsoft\Windows NT\CurrentVersion\Windows
Name: AppInit_DLLs
New Value: MFAPHOOK64.DLL

Path: HKLM\SOFTWARE\Wow6432Node\Microsoft\Windows NT\CurrentVersion\Windows
Name: AppInit_DLLs
New Value: MFAPHOOK.DLL

As you can see in the following screenshots the value always contained the full before path.

Before:
2015-02-12 14_23_34-dedam-wv300 win_7-x64-Personalabteilung on dedam-esx-2-7.corp.grimme.com

After:
2015-02-12 14_24_26-dedam-wv300 win_7-x64-Personalabteilung on dedam-esx-2-7.corp.grimme.com

Before:
2015-02-12 14_27_10-dedam-wv300 win_7-x64-Personalabteilung on dedam-esx-2-7.corp.grimme.com

After:2015-02-12 14_28_13-dedam-wv300 win_7-x64-Personalabteilung on dedam-esx-2-7.corp.grimme.com

The next step was to modify some Graphics Drivers Registry settings. We renamed the following keys to .old and restarted the VMs.

HKLM\SYSTEM\CurrentControlSet\GraphicsDrivers\Configuration

HKLM\SYSTEM\CurrentControlSet\GraphicsDrivers\Connectivity

2015-05-06 12_40_25-dedam-ctxwv0509 on dedam-esx-2-6.corp.grimme.com

After the reboot the keys had been recreated – with different sub keys.
2015-05-06 12_46_11-dedam-ctxwv0509 on dedam-esx-2-6.corp.grimme.com

After trying all of these settings I remembered a case where I had the same problem with XenDesktop 5.x – the solution was to remove the VMWare SVGA-Driver. So we now also removed the driver.
2015-05-06 12_52_01-dedam-ctxwv0509 on dedam-esx-2-6.corp.grimme.com

After removing the driver the Graphics-Card was shown as “Default-VGA-GraphicsCard” (with a Warning).
2015-05-06 12_54_20-dedam-ctxwv0509 on dedam-esx-2-6.corp.grimme.com

And guess what? The black screen was gone. Furthermore we disabled Legacy Graphics Mode and enabled Desktop Composition Redirection – the black screen issue still didn’t happen any longer.



Office 2010 Mui leads to big Citrix UPM Profiles

$
0
0

Shortly after going live with a Citrix XenApp 7.6 environment I saw a massive growing of a lot of profiles- Their size was often more than 60MB – even if it was a task worker which only used the ERP Application and Microsoft Word. After checking the UPM Profile I found the reason for the big profiles – it was the Document Building Blocks folder from Microsoft Office under Appdata.
2014-11-24 11_28_01-RDS - DEDAM-SV0432 - Royal TS

(I know that there are some reasons for redirecting AppData but that wasn’t possible here and I had some problems with Programs when AppData was redirected in the past).

Inside the folder the following subfolders had been created:
2014-11-24 11_27_31-RDS - DEDAM-SV0432 - Royal TS

In the environment Microsoft Office was installed with different language packs. For each installed language a separate Document Building Blocks folder was created for each user using Office. In the following Technet Article is described that this behavior is by design. The workarounds is to move the files to a shared folder.

Another possibility is to remove it from the UPM Profile – but this has one side effect. Every time Word is started they are recreated. The easiest way to achieve this in Citrix UPM is to create (or change) a UPM Group Policy and add the folder to the Exclusion list:
2015-06-03 12_20_24-RDS - DEDAM-SV421 - Royal TS

AppData\Roaming\Microsoft\Document Building Blocks


Upgrade to Citrix XenDesktop 7.7 – no Zones shown

$
0
0

After one of my last upgrades to XenDesktop 7.7 the Studio didn’t show the new available “Zones”.

Luckily Kyle Wise had the same problem and already documented how to fix it.

The solution is quite easy – you just need to open PowerShell on a Delivery Controller and enter some commands.

  1. Import the Citrix-Snapins
    Add-PSSnapin Citrix*
  2. Switch to this folder:
    C:\Program Files\Citrix\XenDesktopPoshSdk\Module\Citrix.XenDesktop.Admin.V1\Citrix.XenDesktop.Admin
  3. Run the following script:
    Import-AdminRoleConfiguration .\RoleConfigSigned.de.xml

After doing this restart the Studio to check out the new available Zones:


Printing from the Internet Explorer Print Preview fails with Citrix UPS 7.7

$
0
0

It looks like there is a bug if you use the Citrix Universal Print Server 7.7 and try to print out of the Internet Explorer print preview. If you open the print preview and try to print the following error message is shown:

There was an internal error, and Internet Explorer is unable to print this document.

ie error

To solve the error, you have to change the Universal driver preference. Create a new Citrix User Policy (or modify one) and move the EMF down – with XPS as the first preference the error does not occur.

universal driver preference

We have created a ticket at Citrix – it looks like different customers have the same problem. A developer is investigation the error. I will update this post when we received a solution from Citrix.


Creating a Citrix NetScaler Redirect Policy for StoreFront Web

$
0
0

When a user tries to access Citrix StoreFront with a Web browser he needs to know the full path to the (default) WebStore – if no redirection is configured. You can either configure this on each StoreFront Server through the IIS or on a load balancer (eg NetScaler) in front of them. Thus I would like to show you how to configure this redirection on a NetScaler:

1.Open the Load Balancing Virtual Server for StoreFront
2014-10-24 08_45_47-Citrix NetScaler VPX - Configuration - Internet Explorer

2. Under Advanced activate Policies and add one (+).
2014-10-24 08_45_57-Citrix NetScaler VPX - Configuration - Internet Explorer

3.Choose the following configuration:
Policy: Rewrite
Type: Request
2014-10-24 08_46_19-Citrix NetScaler VPX - Configuration - Internet Explorer

4.Select “+” next to “Select Policy” to add a Policy.
2016-02-17 13_15_49-Heizungsverteiler _ Heizung KNX - OneNote

5. Enter a Name for the policy and add a new Action (“+” under Action).
image

6.Enter a name for the policy and configure the following settings:
Type: REPLACE
Expression to choose target location: HTTP.REQ.URL
Expression to Replace with: “/Citrix/StoreWeb” (If you are not using the default StoreWeb Url replace this with your Url – but only the folder part).
Confirm the settings with Create.
image

7. With the Action we configured what happens – now we need to configure when the action is applied. Thus you enter the following for the Expression:
HTTP.REQ.URL.EQ(“/”)
To create the Policy with the configured action press Create.
image

8.The last step is to bind the policy to the Virtual Server (Bind). We do not have to configure anything like Priority because for this only one policy is necessary.
image

9. That’s it – the policy is now bound to the Virtual Server and users are automatically redirected to the StoreFront Website.
image


Viewing all 19 articles
Browse latest View live