Applies To:
  • CitectSCADA 5.xx
Summary:
Normally Citect redundancy worked perfectly. However, on occasion either trend server missed samples.

It was seen that the missing samples occurred when using ODBC after one of the LANs was disconnected. Windows itself suspends all other operations while trying to contact the other PC over the now disconnected LAN. Citect got ZERO CPU time for between 10 and 30 seconds while Windows tried to contact the other PC. This means that Citect Trends end up with gaps (missed samples) because Citect was effectively locked up by Windows.

The same problem was also observed using the FileCopy Cicode function with either a mapped network drive or a UNC file path.

As this occurs with a FileCopy, it is logical to assume that the [CTEdit]Copy parameter and Redundant Disk PLCs would also demonstrate the same behaviour.

There may be other activities that also show the behaviour.

Reason

Windows is trying to resolve which LAN connection to use to contact 'PC2' for the ODBC connection and as it had been using 'Lan1' with no problem - when that is disconnected, Windows waits to time-out the connection before it will switch to using 'Lan2'. While Windows is performing this activity, all applications are suspended.

 

Solution:
Windows is trying to resolve which LAN connection to use to contact 'PC2' for the ODBC connection and as it had been using 'Lan1' with no problem - when that is disconnected, Windows waits to time-out the connection before it will switch to using 'Lan2'. While Windows is performing this activity, all applications are suspended.

Solution(s)

(1) A helpful setting was to setup the LMHOSTS file with the #PRE and #MH options so both IP addresses were pre-known for BOTH connections to the PC Name, so that 'Name' search delays were minimised.

10.10.0.1 PC1 #PRE #MH

10.10.1.1 PC1 #PRE #MH

10.10.0.2 PC2 #PRE #MH

10.10.1.2 PC2 #PRE #MH

Note: Also add NetBIOS names like IOServer, XXXAlarm, etc

(2) See KB Articles Q1949 & Q2533 for information on TCPIP timeout settings that can be adjusted.

(3) Use network load balancing software (eg : http://www.falconstor.com/productsnicexpress.asp) which sets ONE virtual IP address that the software routes between the two physical LAN adapters. On failure or disconnection of one LAN, the software automatically (and most importantly, quickly) routes all traffic through the other physical LAN.

(4) Similar to option 3 but using special redundant network cards.

Conclusion

Option 3 gave the preferred results for the following reasons.

In our testing, no trend samples were missed on a 1 second sample rate. Options 1 & 2 (combined) still had very short drop-outs.

When both LANs are working, network bandwidth is virtually doubled. Without load balancing, Citect and Windows only switch LANs when needed.

There is now only ONE IP address for each machine, but all LAN hardware is redundant, making all the software configuration much simpler.

Good diagnostic ability of the load balancing software

Further Note:

In Version 5.50, Redundant trend servers back-fill any gaps that occur even when both servers are running, so the gap will most likely be correctly filled over time.

 
Keywords:
 

Attachments