In the name of security, Google has unfortunately made Chrome an even larger burden for virtual workspaces. The change to the of the browser now renders each page in its own memory and process space. This is good for security (think Spectre) … But a nightmare for virtual workspaces, especially Server OS VDA (aka Citrix XenApp).
What to do?
What is Going on With Chrome?
Chrome is the most widely used PC browser
Look- there's no doubts here. Chrome is the most widely used Desktop and Laptop browser in 2018. By… a lot. The fact that I don't even have to preface what Chrome is speaks to that well enough; but if you need a fancy picture:
I can tell you that I have seen Chrome pop up on nearly every virtual environment I've assessed or designed in the last year. I can also tell you that it already has some pretty severe design impacts to consider, and we'll get into those in the next section. The point here is that I think it is safe to say that Chrome is not going anywhere, which makes my next statement more harrowing…
Why do Chrome 67 and above consume more memory?
Chrome has always been a memory hog. In some cases, it can use as much as twice as much RAM as Internet Explorer (do you know how hard it was not to type ‘exploder' right then?) especially with multiple tabs open (it's a light day for me, I only have 12 tabs open on 3 separate windows/monitors).
The bad news here is that according to several sources [including Forbes, PCGamesN, and How-to Geek] Chrome 67 started using 10% more RAM and consume additional CPU threads to deal with Meltdown and Spectre threats. The process is called Site Isolation and is actually part of the Chromium core, not just Chrome itself. [Update on this: I've done validation of this with LoginVSI but honestly- it has been pretty obvious]
Unfortunately – didn't have the ability to do an A/B test of old vs new on this. If I do, I'll certainly follow up on just how much more RAM is being consumed. At the time of testing for this original article, I was using Chrome 67.0.3396.99 on my primary desktop. Here's what I'm saw:
(note- I'm including Franz because unless I am mistaken, Franz uses the Chromium engine for each tab (I have 2 FB Messenger, 3 Twitter, 7 Slack and 1 Hangouts tab open right now)
An important thing to note here is the composition of memory. Windows will show you four key metrics if you ask nicely (Resource Monitor): Commit, Working Set, Shareable and Private. I don't want to consume a lot of time on this so we'll just focus on two key metrics- Overall or Commit and Private:
- “Commit Charge (also called commit size) is the total amount of virtual memory that a program has touched (committed) in the current session, including memory that has been paged out of physical memory to the disk-backed page file. The Memory and Physical Memory counters on Task Manager’s Performance tab represent the sum of this value for all processes and the kernel. The Commit Charge Limit is the total amount of physical RAM and page file available—in other words, the maximum virtual memory.”
- “Working Set is the term that defines the amount of memory currently in use for a process. Private Working Set is the amount of memory that is dedicated to that process and will not be given up for other programs to use; Shareable Working Set can be surrendered if physical RAM begins to run scarce. Peak Working Set is the highest value recorded for the current instance of this process.”
Source
I don't yet know ways to predict what will be private and what will be shareable, but I can show you an example from my system for you to analyze and see just how much the Commit Charge is Private.
I'm really glad I have 32 GB of RAM in my primary workstation. But what will this mean for, say a Windows 10 VDI with 8 GB RAM? Well – it isn't pretty. In my test the “In Use” RAM exceeded 4 GB pretty rapidly.
However, it is worth noting that Chrome will surrender to ‘standby' and paging RAM – but that can be really costly from a performance standpoint.
For some added fun, I went to a site that has a series of auto-play videos. I verified the tab that was taking up the RAM by closing it, but before it faded into… memory… this is what it looked like:
Note the amount of Private RAM here: 557 MB. For one tab. I can't stress enough the kind of impact this would have on a Server OS VDA (XenApp)!
What Should I Do About Chrome 67 and above?
Methodology Point: Plan (Design)
The key to this is thinking ahead and good planning. In my methodology I call this the PLAN phase, but you may also see it as ‘Design' in other places. There are some homework and considerations here that are fairly intense, so I'd recommend bookmarking this page! However, for now, let's dive right on in.
- Deploy Chrome the right way. I could write up a big long complicated article on the topic… but I'd rather point you towards The Man Dennis Span's Google Chrome on Citrix Deep Dive for all the details you need. In fact, this article was updated recently and taught me a thing or two! If you're wanting a more ‘official' answer let me point you to The Man Dennis Span's CITRIX BLOG POST on the same topic.
Pay special attention to the guidelines for Profile Management. Doing this the wrong way can make life miserable in a hurry. Fortunately, Citrix UPM now includes a lot of defaults that help… but Google has changed file locations in the past. I'm willing to bet it will happen again. So pay attention!
The bottom line here is that there is no official support for Chrome on RDSH. It is a consumer browser with some enterprise support tacked on. So the operational cost of ownership lies with you, fair Citrix Professional. - Deploy Citrix Workspace Environment Management.
- WEM allows you to use a Memory Management feature that can send idle tabs to paged RAM but also optimize the Working Set overall. Does this make a difference? Well – I tell you what. At the end of today before you close down, see how many browser tabs and windows you have open. My guess is that you have tabs you opened at 10 am that are still there at 5 pm. Memory Management lets those tabs stay open but frees up the RAM until you actually use the tab again.
- WEM also has the option of CPU Management. This is one of my favorite features of WEM, actually. I think the most useful and lowest impact area to configure here is the CPU Priority. This allows a process to be set with a base priority for ALL THREADS in the process. So, adding iexplore.exe and chrome.exe to the list and setting them to the lowest CPU priority will not limit the process' capability, just its place in line for the CPU. In other words- if Chrome is going to consume more CPU, we let it… but only when other processes don't need the CPU first. I've noted an interesting effect with this. What I've seen is CPU utilization INCREASE, but user experience also increase. This is especially useful for Server VDA (XenApp) workloads. However – if you are running a VDI-heavy environment, you may want to consider also enabling CPU Spikes Protection and possibly even limiting the core usage. In my opinion, you should always enable Intelligent CPU Optimization as a default.
- I can't even begin to describe how crucial this is if you are running resources from a public cloud like Azure!
- Use Group Policy to control Chrome. Google makes the Chrome Group Policy template available readily. If you aren't using it already, you should be. But the key is to update it frequently!
- Reduce Plugins and Extensions. I will confess; I run with far too many unused plugins. If I'm doing it, likely others are as well. From individual browsers, this can be accessed by navigating to chrome://extensions. You can control extension settings with the GPO noted above!
- Optimize the OS. I really can't believe how many times I am finding customers are still running Out-of-the-box Operating Systems. You really MUST optimize your OS for running in a virtual environment. Martin Zugec from Citrix maintains a free Optimizer script. Go get it and test it! I'm amazed at how much CPU and RAM is just thrown away daily. If you have several hundred users, that means a lot of waste!
- Don't use Chrome? So what about pulling Chrome from your workspace or restricting it from launching unless fully justified? Given what I said above about the market share, this could be a tough one. And I feel the pain here; I use Chrome daily. So don't get me wrong, I'm not a hater. I have Firefox and Opera on my PC. But they don't get used. That said – if you can't increase RAM or use WEM for example, you may be in a bit of trouble and need to take this kind of action for now. I think the tradeoff of security may be worth it as opposed to other browsers… but consider the next point before you recommend or make a decision.
- Increase RAM on your Hosts. This one will hurt but it must be designed for if you have to keep Chrome around! The key thing here is that the cost of extra CPU and RAM may be hard to justify right now, but you must count the cost! Here's what I mean:
- Let's say you have 2000 users all using Chrome. Let's then say that doing so now consumes 500 GB per user more than previously. That is still 1 TB of RAM.
- Have you seen the price of ECC RAM these days? At a very rough estimate, we are talking an additional $20,000 USD just for what Chrome will need. Ouch. And we haven't even touched on CPU requirements. You will either need to run ‘hotter' CPUs or face the issue of adding complete blades! Threads matter here, and it is almost always best to spend a little more on CPU per blade in the long run. If you can get 25-50 more users per physical host that more than pays for the difference!
- In recent polls, I found that the big driver people noted above all others was TCO (total cost of ownership). So I can't ignore the impact Chrome will have on that. Neither should you.
- So in making a decision- it's about if the cost is worth it for security sake *or* for user experience.
Sorry, folks. But at scale, this stuff matters. A lot.
It is important to keep in mind justifications and detach yourself from the ‘if' statements as much as possible by using simple facts. If security is a priority at your organization (and I'm betting as word about site isolation spreads, it will be) this is a simple conversation: “If you want to enable this: here is the cost.” This keeps you from being viewed as being difficult or not wanting to put in the extra work.
What other options or concerns are there in the Citrix Workspace?
First, run some extensions. I know, I just told you to reduce them above, but hear me out on these:
- If you are using the “Data Saver” extension for Chrome because your bandwidth at the datacenter is limited, this extension may still be worth the added CPU and RAM it takes to run.
- After some advice at E2EVC last year, I started using uBlock to block ads. I have found this a lot easier to do than using the old hosts file trick (nothing wrong with doing both by the way, just do a lot of testing before you roll out the hosts file trick!!!). By blocking ads, this extension pretty much ‘pays' for itself. [Update from the article's original text – consider deploying a Pi Hole as your domain's DNS forwarder, or in front of your VDI (just be careful with what subnets to forward to the service)!]
- [UPDATED 3-22-2021] There is much talk about “The Great Suspender” which was a good concept but ultimately the plugin was removed by Google. There are some alternatives (here's a list with my thanks to Ian at Comparitech) So far, these plugins don't seem to have a huge impact in a single VDI session but again… at scale this stuff can really matter, especially shared sessions. My bet is that most users would express irritation that they have to ‘click to reload' a tab… but my other bet is that they would have just been closing it anyway. I know I'm guilty, so I'm trying it out! Note you can often whitelist pages with a click so they aren't ever suspended. Great for Prime Day, but probably not great the other 364 days of the year. Keep an eye on security if you do use these plugins – but also know that this is likely to be a feature built into browsers going forward as well, like in the new Microsoft Edge. There was an experimental feature within Chrome 79 as well but on my last search (89 as of this writing) I could no longer find it (within chrome://flags/ ).
Next, there are some notes about Site Isolation and how to control it with Enterprise Policies here. I can see scenarios where you may want to disable this on a per-site basis… however I am NOT recommending you do so at this point without serious examination.
Don't have WEM? Guy Leech brought up a way to do this via his PowerShell script that can release RAM upon idle or lock of a session. This is certainly something to evaluate if you like (ALWAYS test third party solutions heavily before deploying them!). [Updated 7/20/2018]
Finally- though it does nothing for memory per se, I would be remiss if I didn't point out the impact at scale of the amount of CPU that Chrome uses in VDI and especially RDSH as opposed to running on the physical desktop. No- I'm not crazy… let me explain. The majority of today's websites are actually rendered on your GPU (or the GPU on your CPU/APU). Without access to this, the system must render in CPU… TWICE. Yup, once for the browser and another to encode. So if you really want to reduce CPU and increase your scalability you really need to look into virtual GPU options. As of this writing, the defacto standard is NVIDIA and it is where I will typically point you. But if you are running the right hosts and use XenServer, you may have a much more affordable option in Intel's GVT-g or passthru graphics solution on the E3-12xx v4 or E3-15xx v5 chips. Though AMD does also have a solution, I don't personally think it really applies much in this case. At scale though- NVIDIA GRID can't be beat and they support multiple hypervisors including Acropolis. The great news for Citrix Workspace is that the GPU can be used to encode the actual HDX protocol coming back to the end user which reduces both Server and Client CPU even further. This is a feature that went live with XenDesktop 7.16 VDAs, though I'd be remiss if I didn't point out just as I did last week, that 7.16 is Current Release and has a shorter path to End Of Life. But if you are using virtual graphics, you will probably be on the bleeding edge enough to keep up to date on your VDA/Controller versions.
Why do I bring this up? Because what I'm seeing out there is that those that actually do this combination of especially WEM and GRID (specifically vGPU) are able to stack WAY more users per blade. So though there is a cost associated with vGPU the reality is that once again at scale as we are talking above… you could prevent the purchase of entire BLADES. That is huge.
What has your impact been of hosting Chrome in the Enterprise? Let me know with a comment, tweet or join me in our private (non-FB) community!
Don't forget – my book on becoming a Citrix Hero is available on Amazon or direct from me!
https://ctxpro.com/Book
