fbpx
NetScaler Security for the XenApp Dummy – Part 2: Design

NetScaler Security for the XenApp Dummy – Part 2: Design

We’re back, talking more NetScaler Security! This is part of our four part series using my recommended Assess (or Understand), Design (Plan), Change (Build) and Maintain (Manage) Methodology.

In Part One we went into detail on how to assess and find just how secure your NetScaler configuration really is. We went thru a few of the very many leading practices and how to determine variances from them. If you didn’t read that one, go ahead and read it… I’ll wait. If you have gone thru the article and have your spreadsheet (or notes), we’re ready to continue! Also check back on it from time to time- I’ll be updating the article every now and again when new threats or methods emerge!

NetScaler Security Goals

First, let me say outright that we are not taking actions on this step. That will come in Part 3, Change. For now, we are making a plan of action- what do we intend to do in order to address the concerns we have from our Assessment. Setting some goals is key here.

Setting NetScaler Security Goals

Let’s go thru the list and determine what we want for the best balance between simplicity to deploy and good NetScaler security. We’ll start with the Highest Urgency and Importance items.

  • Score an A+ at SSLLabs.com. Looking from my example, I can see that my NetScaler Gateway is scoring a “C”. I think having an A+ will be a good indicator of acceptable NetScaler security.
  • Ban SSL3 and TLS 1. With TLS 1.2 supported and 1.3 on the horizon, my goal will be to sunset older SSL ciphers completely for better protection.
  • Secure the NetScaler Management Interfaces. I want to make sure that the leading NetScaler security practices are followed to prevent attacks. This will include defining ACLs and restricting access to the NSIP interface. And, of course- change the NetScaler’s NSROOT password.
  • Upgrade to the Best Firmware Choice. As I mentioned in Part One, just because a firmware update has been released doesn’t mean it is automatically best to use it. In my case, I want to be sure the firmware chosen meets minimums for NetScaler security, not features as my primary concerns.
  • Address Leading Security items. Our assessment indicated at least one leading practice that is security related, so we will validate those settings.

Fine Tune for XenApp and XenDesktop

Several lower priority findings and other items marked as Low Urgency and Importance.

  • Validate Leading Practices. As the Health Check Summary from CIS indicated there were several leading practices in need of adjustment from the defaults. I’ll focus on the ones that affect overall NetScaler security.
  • Address Functionality Items. Our assessment had a few items of low priority that may affect the functionality of other systems. In fact, in our case I’m going to even add one that I know needs to be addressed for functionality of the new EDT protocol.

Design

In the Design phase, we will be determining Design Decisions – in this case declaring the configuration changes that will give us the best NetScaler security that makes sense for our use cases.

Score an A+.

I know that to get to A+ I have a lot of things to address, but I am going to first start with a pretty major caveat. Just as I noted in Part 1, we need to declare a minimum support for our SSL settings. If we go too strict, we lose the ability for certain endpoints to connect entirely. On the other hand, we may only want those with secure and updated operating systems to be able to connect at all. For our example, we know that the following is true:

  • We only support connections from up to date browsers and operating systems. All others are best effort and must meet NetScaler security minimums. That means no XP, Vista or out of date OS configurations. Sorry, for those of you stuck in 2008.
  • We do not have Thin clients in scope so the need to continue allowing TLS 1.1 to support older thin clients does not apply. We can also apply STS.

First- let’s deal with the same things I mention in my original NetScaler security article about getting the Best SSLLabs Rating (which will be updated from time to time)

First we will go with the most secure Cipher sets and configuration possible. Now- I know a few things going into this that I want to share with you:

  • First- upgrading firmware. In 2018, SSL Labs will begin downgrading scores for those that are vulnerable. When upgrading firmware, I typically start with the minimum I need and go up to the latest firmware as long as it is at least a month old. This is from experience in feature releases causing disruption for other functions. The most common I’ve seen is problems with AppFlow. If you’re using HDX Insight (and let’s be honest, you should be) this is something you need to be aware of. If you’re not using AppFlow… knock yourself out with the latest; odds are good you’ll be fine. But I can’t stress enough how important validation testing is when upgrading to new major releases (say, 11 to 12 for example). If you’re stuck or less confident- phone a friend. Or contact me using the form over to your right…
  • The way SSL Ciphers are named gets confusing in a hurry- but the names are organized in such a way that indicates a balance between security and performance. NetScaler Gateway has some pretty specific processing needs, especially when using a VPX which doesn’t have an offload chip. Therefore, the selections we’ll make as primary will be driven by performance first. This means in the case of DHE vs ECDHE, we’re going to favor ECHDE because they will perform better in our use case. If you don’t know what I’m talking about… that’s okay. I barely know myself if I’m being honest. But this what I have tested and deployed to over 50 NetScalers last year. Here’s more information if you are hungry for it: https://docs.citrix.com/en-us/netscaler/12/ssl/supported-ciphers-list-release-11.html Note again- that if you are using a FIPS Appliance, you’ll want to review this: https://docs.citrix.com/en-us/netscaler/12/ssl/fips-approved-ciphers.html The list gets complicated, but few care more about NetScaler Security than those using FIPS appliances. Best to understand what Ciphers will be supported for your appliance because the mileage varies!
    • We will make a set of Ciphers specific to NetScaler Gateway, which we will refer to as “NSG”
    • We will separate RSA from non-RSA keys for TLS 1.2 and above, because there is drama emerging there right now. To be flexible in the future, we’ll make it easy to disable the cipher set for testing.
  • Beware ROBOT. I did a re-scan of my SSLLabs assessment and found that at the end of Feb 2018, my NetScaler security score would be an F! This is due to the firmware being vulnerable to ROBOT attack. You can test for that vulnerability here: https://robotattack.org/. We will update the firmware as part of our process to deal with this threat.
  • Speaking of SSLLabs- another up and coming trend you need to be aware of is Perfect Forward Security (PFS). To explain this as briefly as possible, there are two ways to achieve PFS, by using DHE or ECDHE keys. Based on my limited testing and research, I have found that ECDHE performs better than DHE, though DHE does offer better security. If you are running an MPX or SPX appliance you may want to look into DHE (Diffe-Helman Exchange). I do not yet recommend this for VPX that are not hosted on SPX. A lot more information can be found here: https://support.citrix.com/article/CTX205282 But- as long as the ECC curves are in place (they usually are if you have any ECDHE ciphers bound, which we will), this is not a concern for you until the ‘rules’ change to favor DHE more heavily. You may also want to enable STS on your StoreFront servers if you have internal connections- but be aware that this may cause disruption to certain thin clients so TEST TEST and then TEST AGAIN!
  • Note- There are new Ciphers coming soon to support TLS 1.3 – for this article, however I am assuming that we will not have access to that firmware and I’ll update this later with those new sets. We’ll create a Cipher set name that we will keep updated as the latest and greatest and include “CurrentCiphers” in its name.

Other Recommended Settings

  • Remember that we are going to change our NSROOT Password. I thought about suggesting Active Directory integration for NetScaler as well but that is probably best for another blog post or referring you to other sites. If you’d like to learn more about the process and include it in your document, it is something I recommend as you won’t have your NSROOT password floating all over the place. https://support.citrix.com/article/CTX123782 Regardless- change your NSROOT password on a regular basis even if you have AD authentication configured.
  • Unless our testing reveals something different, we plan on implementing all the recommendations found in our Scout report. Rather than type them twice, I’ll list them in the design document below.

NetScaler Security Design Document

Pulling from all the items above, we can generate a document that lays out our plan for better NetScaler security so we can share it with others if needs be- but is also very useful if it is something that will take some time to get approved. It doesn’t have to be anything fancy, but I always advise people to write things down first – here’s something to get you started. PLEASE NOTE- these are just the items I found on my assessment example. You’ll want to include any findings you have as well!

Item Design Decision Notes
NSROOT Password Change to a complex password Store the new password securely! I use Dashlane to store and securely control sharing of passwords with 2FA security. Sign up here to get $20 off for you and give $20 for me too! Way safer than putting them in a file!
NetScaler Firmware Upgrade Firmware to latest stable within our major version release. We’ll be using the 12.0 56.20 build in our example. I chose this simply as it is the latest at the time of writing and met the criteria of not being vulnerable to CVE-2017-14602 and 2017-17382.

Guidance on the steps to expect here:

https://support.citrix.com/article/CTX127455

To prevent ROBOT Attack, make sure to get a firmware above https://support.citrix.com/article/CTX230238

Management Interfaces Only allow the NSIP as a management interface and force Secure (SSL) communication only Disable Enable Management Access control, Telnet, SSH, and GUI on all Non-Management IPs.

More information here about restricting NSIPs to Only allow management applications: https://support.citrix.com/article/CTX126736
See the “Enable Secure Access to NetScaler GUI” at https://support.citrix.com/article/CTX111531

NetScaler Security – Interface ACLs -Allow HA Communication between NetScalers

-Allow communication from Utility servers

-Allow communication from NMAS Server

-Deny all others

This action must be performed on both NetScalers

NetScaler NSIPs: (list yours)
Utility Server IPs: (list servers or subnets that will have access)
NetScaler MAS: (If you have them- list the IP Addresses of NetScaler MAS appliances)
NetScaler Gateway – SSL Parameters TLS 1.2 only
Enable HSTS (Strict Transport Security
Disallow SSL v3, TLS 1 and 1.1

For HSTS, configure a rewrite action.

Read up on it at https://support.citrix.com/article/CTX205221

Note: this procedure is different depending on if you are using a 12.x based firmware or not. In our case we can use the instructions for 12.x at https://support.citrix.com/article/CTX224172

NetScaler Gateway – Basic Settings Enable DTLS Check box for DTLS
Traffic Management – SSL – Cipher Groups Create custom Cipher Groups NSG-TLS1.2-RSA-Ciphers

NSG-LegacyCiphers

TLS-HighSecureCurrentCiphers

Traffic Management – SSL – Ciphers Favor high performance, High security ciphers and work downward within each set.

TLS1.2-RSA group will be isolated for future removal if needed.

NSG-LegacyCiphers will only be used if older clients are in place, but I recommend creating the group just in case.

Each grouping will start with higher AES and SHA values and work downward.

TLS-HighSecureCurrentCiphers group will utilize the new ECHDE-ECDSA ciphers instead of RSA ciphers.

NSG-TLS1.2-RSA-Ciphers:
TLS1.2-ECDHE-RSA-AES256-GCM-SHA384TLS1.2-ECDHE-RSA-AES128-GCM-SHA256TLS1.2-ECDHE-RSA-AES-256-SHA384TLS1.2-ECDHE-RSA-AES-128-SHA256TLS1.2-DHE-RSA-AES256-GCM-SHA384TLS1.2-DHE-RSA-AES128-GCM-SHA256NSG-LegacyCiphers:

TLS1-ECDHE-RSA-AES256-SHA

TLS1-ECDHE-RSA-AES128-SHA

TLS1-DHE-RSA-AES-256-CBC-SHA

TLS1-DHE-RSA-AES-128-CBC-SHA

TLS1-AES-256-CBC-SHA

TLS1-AES-128-CBC-SHA

TLS-HighSecureCurrentCiphers:

TLS1.2-ECDHE-ECDSA-AES256-SHA384

TLS1.2-ECDHE-ECDSA-AES128-SHA256

TLS1.2-ECDHE-ECDSA-AES256-GCM-SHA384

TLS1.2-ECDHE-ECDSA-AES128-GCM-SHA256

SSL Parameters Disable Client and Server Side SSL Renegotiation Deny SSL Regeneration to the Frontend Client

https://support.citrix.com/article/CTX123680

TCP Defaults Enable Nagle’s Algorithm

Enable Selective Acknowledgement

Enable Window Scaling

https://support.citrix.com/article/CTX121149

Note- check with your Network Administrator to be sure the Window Scaling option will be supported (SACK is tied to Scaling.) If not, only list Nagle’s. Another key metric here is the Factor for Scaling. Typically 4 is correct but see the article at https://support.citrix.com/article/CTX113656 to be sure you will not need to adjust this value after conferring with your Network Administrator.

HTTP Parameters Drop invalid HTTP requests

Deployment Plan

Whenever possible, I believe in using a non-production test environment or components to validate changes before impacting users. Part of our Deployment Plan will involve testing these settings on a test NetScaler VPX prior to deployment in production. Once our changes are validated, we’ll set a time for a production outage so that we can roll back changes if required. For an HA deployment, it is very possible to do the updates in a way that minimizes production impacts. However, with NetScaler Gateway connections will be disrupted when we reboot. So whenever possible with this kind of update, I recommend declaring an outage or at the least setting the expectation that sessions may disrupted during a defined time window when the changes will be made.

So write out a deployment plan with a rollback plan. This is very useful especially when you must generate something for a Change Control board. Here’s the high-level plan:

  1. Save the Running Configuration of the test NetScaler VPX
  2. Export the full NetScaler Backup I recommend reviewing my fellow CTA George Spiers’ guide at http://www.jgspiers.com/netscaler-backup-restore/
  3. Run a snapshot of the NetScaler VPX
  4. Upgrade NetScaler Firmware (if HA, use the advice in the article above to list out your steps)
  5. Secure the NSIP Interface by introducing new ACLs (VERIFY before you continue)
  6. Make the other changes noted above

Next time- I’ll guide you thru the process of actually making the changes! When we’re done, you’ll have NetScaler Security at a level of confidence that will exceed most enterprise customers I’ve been visiting lately!

Citrix’s New Best Practice items – DJ’s Highlights

Citrix’s New Best Practice items – DJ’s Highlights

Gotta boost the signal on this. My friend Nick Rintalan from Citrix Consulting has put together a new ‘best practice’ (or leading practices for the lawyers) update that I feel it’s important for people to see!

Nick Rintalan, Lead Architect and Best Practice Guru at Citrix Consulting

Nick Rintalan, Lead Architect at Citrix Consulting

New Best Practice(s)?

Here are some of the highlights of the article, sorted here by what I feel is most important for you to read:

  • PVS and Memory Buffers. Yes, yes, for the love of all that is holy, yes. I haven’t yet deployed for validated the Write Cache features now in MCS, but I can tell you from experience that XenApp with 2-4 GB of RAM cache with failover to disk has been giving roughly 20-30% faster logons and overall better experience for most of my customers.
  • Protocols (as in HDX). One of my primary frustrations for quite some time now is that Citrix XenDesktop ships by default with a protocol that has a good experience on LAN but tends to be problematic at distance. H.264 is great for video, but frankly I hate it everywhere else. I think it almost singlehandedly ruined things for Citrix since PCoIP can perform better than this hog (my opinion). Thinwire and even the legacy encoder, however- actually deliver on the promises and need to be investigated in nearly every single use case I see. So I agree with Nick- use the policy templates included with 7.6 u3 and above (including LTSR) as a starting point. Odds are good you won’t be disappointed. When I say ‘use’ here what I mean is remember that you can apply these codecs on a per user basis, connection basis or even per delivery group- meaning filters are your friend! It is perfectly acceptable to have multiple codecs going for various use cases. One size nearly NEVER fits all, so test these out!
  • vSphere Cluster Sizing. Number 3 on my list right now. You need a dedicated resource cluster for Enterprise workloads- but honestly- for XenApp workloads, consider more hosts per cluster. You should be using bigger VMs anyway, so the number of managed VMs is about the same- just more computing power. CCS is seeing 24+ hosts per cluster be just fine in XenApp. For XenDesktop with more than 5000 VMs- I will add here that a dedicated vCenter may save you a lot of pain… my opinion, and of course… you guessed it. TEST!
  • XenApp CPU Over-Subscription. Seriously. The “1.5x” thing needed an update so I’m glad to see some clarification here. In all things- I still encourage practical testing instead of just implementing something because “Citrix said to.”
  • PVS Ports and Threads. Those of you who know me know I bang this drum a lot- so here’s some backup for what I’m saying. The defaults are not good enough. Good design is still required!
  • Farm Design. You’re probably like me and are coming along kicking and screaming from XenApp 6.5, which most would agree has been the “Windows XP” of the Citrix world. It just hasn’t been this good yet, and I still feel 7.9 doesn’t have true feature parity… but as Nick describes, they are getting there. As always… TEST, TEST and then TEST some more before you implement zones with FMA!!!!
  • XenMobile Optimizations. I guess we have to talk about it. XenMobile is here to stay, so best to not take the ‘out of the box’ experience there either.

Read More

READ: “New” Citrix Best Practices 2.0 | Citrix Blogs

Give it a read and let me know your thoughts… but most importantly- don’t forget to share this!