The goal of this series is to outline some of the more common Citrix Mistakes that I have been seeing in my consulting engagements. These top three were chosen not because of frequency per se, but for the sheer impact they have. I chose them so that when you implement the fixes – you can be the hero. The #CitrixHero.
You Are Making Citrix Mistakes and So Am I
If there's one thing I have found to be true being a Citrix Consultant since 2011 it is that there is ALWAYS something wrong. Even in environments that I've worked in directly. They key is avoiding the big pitfalls first, then going to the others.
EVERY Citrix environment has something wrong with it. Avoid the big Citrix Mistakes and be a #CitrixHero Click To TweetAs you can imagine, the first step is always admitting you have a problem so you can resolve it. The reality is that even I make mistakes or have oversights because we all do. Citrix is a rapidly changing system based on an equally rapidly changing ecosystem of products. The implications change all the time!
Best Practices are a Myth
For me, I often find that the Citrix mistakes that plague me are usually the silly little things that in truth, have nothing to do with Citrix itself. A Group Policy here, an optimization there… the reality is that there is NO SUCH THING as a perfect Citrix environment and probably never will be. So many seasoned consultants and architects like myself have stopped saying “Best Practices” and now say “Leading Practices” because we've come to realize that what it really boils down to is:
- Every Environment is different
- You can't be perfect
- There are trends we can measure
- Success makes failure more obvious
So with that I'll give the classic “It Depends” caveat with all of this. As in, is X practice the best for your environment? It depends on your environment!
This means you should always test any of these changes before putting them into production!
I'm serious about that- in fact before I give you the download for this article I'm going to have you give me a name and email with your promise you'll do your best to test the fixes safely before putting them in front of your users!
Of over 300 individual findings and recommendations that I gave out in the past 6 months- these were the most impactful.
UPDATE July 2020: The free ebook once available has been sunsetted in favor of my full book “Be a Citrix Hero” – you can download the entire book for free with Kindle Unlimited. You can also view the VIDEO version of the original Top 3, and get access to a host of additional training at https://learn.ctx.academy
Additionally – Look for a masterclass upgrade – from a paid webinar where I described all 12 current top bad practices – the Dirty Dozen: The Top Citrix Mistakes and how to fix them. When you do- you'll also be able to download a pre-filled version of the spreadsheet I use for assessments. Whew! That's a lot I know but I've been working at this all year and I'm excited to finally release it to you!
One quick caveat: While I do list in this series the same kinds of things you'll hear from Citrix Consulting Services, and even though when representing CCS I may make the same exact recommendations… you should not take these recommendations as a specific recommendation for YOUR environment. For a recommendation from Citrix… hire Citrix.
People who claim that they got my recommendations ‘from Citrix' will go to the special hell.
The one reserved for child molesters and people who talk in the theater.
…
The special hell…
But as a preview- here's what you'll see in the Top 3 Citrix Mistakes series!
Citrix Mistake #1: Windows OS Optimization
Did you know that the ‘out of the box' configuration for every Microsoft OS is NOT optimized for virtual delivery? I'm guessing it seems silly to read that- but Microsoft builds the operating system not with performance in mind out of the box; but addressing a wider audience. Citrix Consulting has nearly from day one talked about these optimizations… and the need is actually increasing. My study of not only the last 6 months but the last 7 years indicates the trend is for people forgetting to optimize the OS when they deploy Citrix! This costs performance- which means dollars.
[email protected] has talked for years about optimizing @microsoft Windows... but these optimizations are rarely done properly and often not at all according to a CTXPro.com study. #citrixhero Click To TweetRecent tests by LoginVSI proved that Server 2016 is especially bad in this regard- unoptimized VMs can quite literally cost you thousands of dollars because less people can be logged into each VM. This means less users per blade which at scale- is a huge problem. You're basically throwing away money and impacting the environment for literally no benefit!
This past year my observations were that 90% of the companies I visited for assessments did not follow the Citrix recommended guidelines for optimization. When these companies implemented the optimization steps properly, some saw increased users per blade of over 30%. For one company, this saved them from having to purchase additional blades that would have cost them well over $80,000.
Fixing Citrix Mistake #1
Needless to say it matters; however fixing it is easy and free! There are several options, primary of which is the Citrix Optimizer – a free tool that automatically detects variations from the recommended tunings and lets you select which ones to apply and which ones not to (for example, on some servers you may want Windows Search to run for Outlook – on others you may not, so optimize and test appropriately). But there's more. A lot more!
Bottom line- saving your company several thousand dollars = #CitrixHero
UPDATE: The First #CitrixHero Scavenger Hunt location is published!
The story is on the Citrix User Group Community (CUGC) Blogs!
Where will the next be? Keep an eye on Social Media for #CitrixHero… or subscribe to my email list to find out!
Citrix Mistake #2: NetScaler Using Default Settings
Believe it or not- in the same way Microsoft OS's are shipped non-optimized… the same goes for Citrix NetScaler! That's right, NetScaler ships in a compatibility state with the assumption that you will take steps to both lock it down and optimize the settings for your particular use case.
While things like performance can be improved a little bit- the biggest impact I'm seeing right now is when security audits a NetScaler Gateway (and often other internet-facing IPs) that do not pass SSL inspections. #CitrixHero to the rescue!
A true #CitrixHero knows that @Citrix #NetScaler must be optimized and locked down. Defaults are unsafe! A tuned NetScaler is a happy NetScaler! Click To TweetFixing Citrix Mistake #2
There are quite a few settings to check here, and I go thru all of them in my blog series on NetScaler Security. But here's a quick list of the most common findings this year:
- NetScaler Firmware Vulnerable to attack
- Plaintext StoreFront website vulnerable to man-in-the-middle snooping even if secured from NetScaler Gateway front end
- SSL Labs scores not passing; should be an A+
- Other Leading Practices not yet configured
- Drop Invalid HTTP requests
- Enable Selective Acknowledgement
- Configure Window Scaling
- Use TCP tuning for XenApp & XenDesktop
- ACLs not configured
- Management interface on port 80 and enabled on all interfaces
- And my personal favorite finding (twice this year): NSROOT password still set to default. Talk about an easy hack!
So how to fix these? The PDF will have the updated information, however- if you need a hint, see my series on NetScaler Leading Practices!
Citrix Mistake #3: Workload Placement, Sizing and NUMA
What I'm seeing emerge once again these days is the old practice which I thought for sure we had done away with: All the VMs in a single cluster and let the Hypervisor sort them out. This presents a few problems:
- You are no longer able to predict how much adding new users will cost in terms of hardware.
- You cannot accurately predict how many Server VDA VMs you will need.
- You cannot predictably assure performance from one user to the next due to other workloads that co-habitat the same host. For example, when a SQL server goes into freakout mode on the same CPU as your Server VDAs- you'll have users complaining even though CPU is not showing any signs of issue. Strange but true.
- Different workloads can tolerate different overcommit ratios. With a mixed workload style you may have your user workloads on hosts that are actually overcommitted.
What year is it? I thought the science on this was settled way back in 2009 or so; but from what I'm finding out there, it hasn't been taken to heart.
'What year is it?' #CitrixHero DJ Eshelman (@TheCitrixCoach) wonders why people don't think about the impact of their #Citrix VMs mixing with every other workload! Click To TweetBottom line- when people have actually listened and followed this leading practice, they've seen users per server increase, happier users, less calls to the service desk and most of all- happy CIOs. So why they'd ignore this advice is just beyond me… but that's what I've been finding.
Fixing Citrix Mistake #3
What we'll discuss in this session:
- Workload Placement
- Workload Sizing and Cluster Sizing
- NUMA
- Hardware Virtualization Settings
This is one of my favorite topics- it is so hard not to give it all away right now!!!
Subscribe or watch me on Twitter soon to find out where this tip will ‘land'!
What Citrix Mistakes are You Making?
Sound off in the comments or share on Twitter- what Citrix mistakes have you been making or have seen others making lately?