Configure high availability deployment
High Availability (HA) refers to a system that is always available to a user without any interruption to the services. High availability setup is crucial during system downtime, network or application failures, and is a key requirement to any enterprise. A high availability deployment of two NetScaler Console nodes in active-passive mode with same configurations provides uninterrupted operations.
Related information
Deployment scenario
Note
The validated maximum storage limit for a single NetScaler Console HA deployment is 3 TB. For more information, see the deployment guide.
Important
To access NetScaler Console 12.1 build 48.18 or later versions using HTTPS:
If you have configured a NetScaler instance to load balance NetScaler Console in a high availability mode, first remove the NetScaler instance. Then, configure a floating IP address to access NetScaler Console in high availability mode.
The following are the benefits of high availability deployment in NetScaler Console:
-
An improved mechanism to monitor heartbeats between the primary and secondary node.
-
Provides physical streaming replication of database instead of a logical bi-directional replication.
-
Ability to configure the floating IP address on the primary node to eliminate the need of separate NetScaler load balancer.
-
Provides easy access to the NetScaler Console user interface using the floating IP address.
-
NetScaler Console user interface is provided only on the primary node. By using the primary node, you can eliminate the risk of accessing and making changes to the secondary node.
-
Configuring the floating IP address handles the failover situation and reconfiguring the instances is not required.
-
Provides built in ability to detect and handle split-brain situation.
The following table describes the terms used in high availability deployment.
| Terms | Description |
|---|---|
| Primary node | First node registered in the high availability deployment. |
| Secondary node | Second node registered in the high availability deployment. |
| Heartbeat | A mechanism used to exchange messages between primary and secondary node in the high availability setup. The messages determine status and health of the application on each individual node. |
| Floating IP address | A floating IP is an IP address that can be instantly moved from one node to another in the same subnet. Internally it is set up as an alias on the network interface of the primary node. If there is a failover, the floating IP address is seamlessly moved from the old primary to the new one. It is useful in high availability setup because it allows clients to communicate with the high availability nodes using a single IP address. |
Note:
For more information on port and protocol details, see Ports.
Components of high availability architecture
The following figure displays the architecture of two NetScaler Console nodes deployed in high availability mode.
In high availability deployment, one NetScaler Console node is configured as the primary node (MAS 1) and the other as the secondary node (MAS 2). If the primary node goes down due to any reason, the secondary node takes over as the new primary node.
Monitoring tool
Monitoring tool is an internal process used to monitor, alert, and handle failover situations. The tool is active and running on each node in high availability. It is responsible for starting subsystems, initiating database on both the nodes, deciding on the primary, or secondary node if there is a failover, and so on.
Primary node
The primary node accepts connections and manages the instances. All processes such as AppFlow, SNMP, LogStream, syslog, and so on is managed by the primary node. The NetScaler Console user interface access is available on primary node. The floating IP address is configured on the primary node.
Secondary node
The secondary node listens to the heartbeat messages sent from the primary node. Database on the secondary node is in read-replica mode only. None of the processes are active in the secondary node and the NetScaler Console user interface is not accessible on the secondary node.
Physical streaming replication
The primary and secondary nodes synchronize through heartbeat mechanism. With the physical streaming replication of database, the secondary node starts in read-replica mode. The secondary node listens to the heartbeat messages received from the primary node. If the secondary node does not receive any heartbeats for a time period of 180 seconds, the primary node is considered to be down. Then, the secondary node takes over as the primary node.
Heartbeat messages
Heartbeat messages are User Datagram Packets (UDP) that are sent and received between primary and secondary node. It monitors all subsystems of NetScaler Console and database to exchange information about the node state, health, processes, and so on. The information is shared between the high availability nodes every second. Notifications are sent as alerts to the administrator if there is a failover or break up of high availability states.
Floating IP address
The floating IP address is associated with the primary node in the high availability setup. It is an alias given to the primary node IP address, that the client can use to connect to NetScaler Console in the primary node. Since the floating IP address is configured on the primary node, the instance reconfiguration is not required in case of failover. The instances reconnect to the same IP address to reach the new primary.
Key points to note
-
In a high availability setup, both the NetScaler Console nodes are deployed in active-passive mode. They must be on the same subnets using the same software version and build, and have same configurations.
-
Floating IP address:
-
Floating IP address is configured on the primary node.
-
Instances need not be reconfigured if there is a failover.
-
You can access a high availability node from the user interface, either by using the primary node IP or floating IP address.Note:We recommend you to use the floating IP address to access the user interface.
-
-
Database:
-
In a high availability setup, all configuration files are synchronized automatically from the primary node to the secondary node at an interval of one minute.
-
Database synchronization happens instantly by physical replication of database.
-
Database on secondary node is in read-replica mode.
-
-
NetScaler Console upgrade:
-
Internal processes implicitly upgrade NetScaler Console from the earlier versions.Note:After the upgrade is successful, you must configure the floating IP address.
-
-
UDP default port 5005 is available on both the nodes for heartbeats to be sent and for messages to be received.
-
MAC address The setting for the "MAC Address Changes" option in a hypervisor affects the traffic that a virtual machine receives. Allow MAC address changes to be enabled on the virtual switch so that the floating IP address moves seamlessly to the new primary node after failover. For example, when deploying NetScaler Console on a high availability on VMware ESXi, ensure you accept changes to MAC address. ESXi now allows requests to change the active MAC address to other than the initial MAC address.Note:For NetScaler Console deployed on ESXI version 6.7, you can set the MAC Address Changes option to Reject also. After failover, the traffic flows to new primary node seamlessly irrespective of the MAC Address Changes setting. Therefore, accept changes to MAC address is not mandatory.If the NetScaler Console is deployed on the ESXI version lower than 6.7, ensure the MAC Address Changes option is set to Accept only.
Prerequisites
Before you set up high availability for NetScaler Console nodes, note the following prerequisites:
-
The NetScaler Console high availability deployment is supported from NetScaler Console version 12.0 build 51.24.
-
Download the NetScaler Console image file (.xva) from the NetScaler site:
<https://www.citrix.com/downloads/>
We recommend you to set CPU priority (in virtual machine properties) at the highest level to improve scheduling behavior and network latency.
The following table lists the minimum requirements for the virtual computing resources:
| Component | Requirement |
|---|---|
| RAM | 32 GB |
| Virtual CPU | 8 CPUs |
| Storage Space | We recommend you to use solid-state drive (SSD) technology for NetScaler Console deployments. The default value is 120 GB. Actual storage requirement depends on NetScaler Console sizing estimation. If your NetScaler Console storage requirement exceeds 120 GB, you have to attach an additional disk. Note: You can add only one additional disk. We recommend you to estimate storage and attach additional disk at the time of initial deployment. For more information, see How to Attach an Additional Disk to NetScaler Console. |
| Virtual network interfaces | 1 |
| Throughput | 1 Gbps or 100 Mbps |
| Hypervisor | Versions |
| Citrix Hypervisor | 6.2 and 6.5 |
| VMware ESXi | 5.5 and 6.0 |
| Microsoft Hyper-V | 2012 R2 |
| Linux KVM | Ubuntu and Fedora |
To set up NetScaler Console in high availability mode
Starting from version 14.1 build 17.x, you can deploy a high availability setup directly from NetScaler Console GUI of the primary node.
-
Register the first server (primary node).
-
Register the second server (secondary node).
-
Deploy high availability setup in the primary node GUI.
Register the first server (primary node)
To register the primary node:
-
Use the .xva image file downloaded from the NetScaler site and import it in to your hypervisor.Note:It might take a few minutes for the .xva image file to import and get started. You can see the status on the bottom of the screen.
-
After the import is successful, right-click and click Start.
-
From the Console tab, configure NetScaler Console with the initial network configurations.
-
After the initial network configuration is complete, the system prompts for login. Log on using following credentials – nsrecover/nsroot.Note:After you log on, if you want to update the initial network configuration, type
networkconfig, update the configuration, and save the configuration.
Register the second server (secondary node)
-
Use the .xva image file downloaded from the NetScaler site and import it in to your hypervisor.
-
From the Console tab, configure NetScaler Console with the initial network configurations as displayed in the following image.
-
After the initial network configuration is completed, the system prompts for login. Log on using following credentials – nsrecover/nsroot.Notes:
-
After you log on, if you want to update the initial network configuration, type
networkconfig, update the configuration, and save the configuration. -
The system will show error messages if there are any issues in the configuration.
-
The system reboots and takes a few minutes for the configurations to take effect.
-
Deploy high availability setup from the primary node GUI
After registering both primary and secondary nodes, log in to the primary node GUI to set up the high availability pair.
Note:
-
Before deploying the nodes into a high availability pair, ensure that the secondary node is completed with a reboot, after the initial network configuration.
To deploy a high availability pair from the primary node, follow these steps:
-
Log in to the primary node GUI.
-
Navigate to Settings > Administration > High Availability Settings > Configure NetScaler Console High Availability (HA).
-
On the Configure NetScaler Console High Availability (HA) page, enter the following details for the secondary node:
-
Peer Node IP Address
-
Peer Node Password
-
Floating IP address
-
-
Click Configure.
-
On the Confirm page, click Yes.
Both primary and secondary nodes are rebooted to form a high availability pair, typically taking approximately 10 minutes.Notes:-
You can now start using the Floating IP address.
-
A floating IP address is mandatory for high availability deployment of nodes.
-
After the high availability deployment is complete, use the floating IP address to access the NetScaler Console user interface.
-
-
Navigate to Settings > Deployment to validate the deployment.Note:The secondary node might take around 10 minutes to come up. Until then, the secondary node status is shown as Down.
For more information, see the Frequently Asked Questions.
Disable high availability
You can disable high availability on a NetScaler Console high availability pair and convert the nodes to standalone NetScaler Console servers.
Note:
Disable high availability from the primary node.
To disable the high availability:
-
In a web browser, enter the IP address of the NetScaler Console server primary node.
-
In the User Name and Password fields, enter the administrator credentials.
-
On the System tab, navigate to Deployment and click Break HA.A dialogue box is displayed. Click Yes to break the high availability deployment.
Redeploy high availability
After you disable the high availability to a standalone deployment, you can redeploy it to high availability mode again. Redeploying high availability is similar to the first time deployment of high availability. For more details see Deploy high availability setup from the primary node GUI.
Note:
After disabling NetScaler® Console high availability, use only one console node as the standalone license server. The second node must be reprovisioned.
High availability failover scenarios
A failover occurs if one of the following conditions is encountered:
-
Node failure: Primary node goes down, no heartbeat is detected from primary node for 180 seconds.
-
Application health failure: Primary node is up and running but one of the NetScaler Console processes is down.
View Database Synchronization Log messages
In the NetScaler Console HA pair, the configuration files are synchronized automatically from the primary node to the secondary node and the physical streaming replication of database happens.
However, if there is a streaming replication error, the Sync Database button appears. You can click the Sync Database button to start the database synchronization process.
To view the progress of the database synchronization, click View Logs. The Database Sync Logs message appears and you can view the details of the synchronization progress real-time.
Split-brain scenario
When there is no communication between both the nodes due to downtime in network link, then:
-
Primary node continues to operate as primary
-
Secondary node takes over as primary because of the failure to receive heartbeats
-
Both the nodes would run their individual database instances
For example, in an enterprise two NetScaler Console nodes have been deployed as primary and secondary. Due to a possible network link downtime, the communication between the two NetScaler Console nodes breaks completely. Since there is no heartbeat exchange for over 180 seconds, both the nodes consider themselves to be the primary node. Both nodes act as active nodes and run their own instances of database.
From NetScaler Console 12.1 or later release, this split-brain situation is handled gracefully after the network link and heartbeat is restored. High availability synchronization is restored automatically. The recovery time depends on the data and speed of the link between the nodes.
Note:
During the split-brain condition, changes that occurred on the old primary node is reset with the new primary when it is rejoined in high availability. The changes that happened on new primary node during split-brain remains intact.
Licensing
The NetScaler Console server can contain VIP, CICO, and pooled capacity licenses. When the licenses are issued to a NetScaler Console server, the licenses are bound to the server host ID. Assigning licenses to a different NetScaler Console server is restricted.
If you configure a NetScaler Console high-availability pair as a license server, license files applied on primary get synchronized to secondary.
Note:
-
In release 12.1-50.x and later, the NetScaler Console licenses are automatically synchronized from primary to the secondary node.