Recommended Reads

Secure Communications

Introduction

This user guide provides procedural instructions to enable secure communications across the various Geneos functional areas and components (Gateway, Netprobe, Active Console etc.).

Secure communications technical reference material for these components can be found here:

Netprobe and Gateway: Command Line Options, Netprobe Security and Netprobe Setup

Licence daemon: Running the Licence Daemon

Webslinger: Webslinger Technical Reference

Active Console: Active Console 2 Gateway Connections

Open Access: Security Overview

Web dashboard: Enable SSL

Auditor: Secure Gateway

Overview

The Geneos Enterprise Management Framework (EMF) protocol that is used to transmit data between Geneos components supports a secure protocol that encrypts data before transmitting it. The protocol used is Transport Layer Security (TLS). TLS versions 1.0, 1.1, and 1.2 are supported.

A Geneos server is any Geneos component that listens on a prescribed port and accepts connections from other Geneos components on that port.

In order for secure communications to be established, each Geneos server must be provided with a key pair composed of a signed x509 certificate and private key.

For example, given the following setup:

  • 1 probe listening on port 7036
  • 1 gateway listening securely on port 7038, which connects to the probe
  • 1 Active Console connecting to the gateway.

In this situation, the netprobe is a server, providing a service to the gateway and the gateway is a server providing a service to the Active Console. To set this up using secure ports, a key pair must be provided for both the netprobe and the gateway.

If self announcing probes are used, the setup of servers is simpler.

  • 0 self announcing probe which connects to the gateway
  • 1 gateway listening securely on port 7038
  • 1 Active Console connecting to the gateway.

Here, the Gateway is the only server. Both the Active Console and the Netprobe connect to the gateway using the TCP port that it makes available. To set this up securely a key pair need only be provided for the gateway.

Minimum TLS versions

TLS connections are supported up to TLS 1.2.

A minimum supported TLS version can be specified on the Gateway, Licence Daemon, Netprobe and Webslinger using the command line argument -minTLSversion and the TLS version value. The accepted values are:

  • 1
  • 1.0
  • 1.1
  • 1.2

A minimum supported TLS version can be specified for Gateway Recorder using the MIN_TLS_VERSION environment variable.

Active Console 2 negotiates the best possible version on connection. Attempting connections with an Active Console 2 earlier than v4.9 with a minimum TLS version higher than 1.0 fail to connect.

Failed attempts to connect securely result in the error message:

-ERROR: SSLConnector:accept SSL Error:[140760fc] SSL23_GET_CLIENT_HELLO:unknown protocolWarning

Caution: When a connection fails, Active Console 2 continually tries to reconnect. This results in a large number of error messages in the Gateway log if the connection keeps failing.

Creating Secure Geneos Servers

Geneos server components read the key pairs at start up. Their locations are provided using the following command line options;

-ssl-certificate <file>
-ssl-certificate-key <file>

The ssl-certificate command line option points to a file that can either contain just the TLS certificate, or both the TLS certificate and the server’s private key. If the private key is not provided in the certificate file it can be provided in a separate file using the ssl-certificate-key option. Both of these files are expected to have their contents stored using PEM (Privacy-enhanced Electronic Mail) format.

The key pair provides the Geneos servers with the ability to provide a TLS-encrypted connection to their clients. In order to enable this the secure service must be turned on. The way that the secure service is turned on differs for different Geneos components.

Netprobe

For normal Netprobes (not self-announcing or floating), TLS is enabled using the -secure command line option.

If the -secure option is not provided then the Netprobe provides a normal TCP socket for communication, and data sent between the Gateway and the Netprobe is unencrypted.

If the -secure option is provided at Netprobe start-up then the Netprobe will provide TLS-encrypted communications so long as a key pair is also provided. If a readable key pair is not provided the Netprobe does not start.

Gateway

For gateways, TLS is enabled or disabled using the gateway setup. The security of the gateway ports is controlled via the operating environment ListenPorts setting. This allow the gateway to be configured to listen on a secure port, an insecure port or both. In the example below, the Gateway has been configured to listen securely on the default port (which is 7038).

Licence Daemon

For licence daemons(licd), TLS is enabled using the -secure command line flag. If the -secure option is provided then licd will serve pages across a TLS encrypted socket so long as a key pair is also provided. If the -secure option is provided at licd start-up without a readable key pair then the licd will not start.

Webslinger

For Webslingers, TLS is enabled using the -secure command line flag. If the -secure option is provided then webslinger will serve pages across a TLS encrypted socket so long as a key pair is also provided. If the -secure option is provided at Webslinger start-up without a readable key pair then the webslinger will not start. Web browsers connecting to it will need to use HTTPS rather than HTTP in order to view the web pages.

Geneos Clients

When a Geneos client connects to a Geneos server it may connect securely or insecurely. The connection type of the client must match the connection type of the server. Different clients configure this in different ways.

Active Console

The Active Console has two different ways to configure a connection to a gateway. The most common way is to add the connection to the connections list in the Active Console settings. This connection has a secure connections check box that is used to indicate if the Active Console should connect securely or insecurely to the Gateway in question.

The second method of specifying gateway connections is to use a Connections file. This file is specified in the Active Console Reference guide, and each connection entry specifies the gateway host(s), port, login details and the connection security setting.ccc

Gateway

When specifying a Netprobe connection in the gateway, along with the host and port for the connection, the administrator must indicate if this connection is secure by using the secure checkbox. This checkbox is available on both the probe and probe groups, allowing groups of probes two be switched from insecure to secure with ease.

Netprobe

When specifying a gateway connection in the self announcing (or floating) netprobe setup file the security of the connection can be set using the <secure> tag as shown below. This tag is optional. If it is not supplied the probe will attempt to connect to the netprobe insecurely.

<?xml version="1.0" encoding="ISO-8859-1"?>
<netprobe
   compatibility="1"
   xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
   xsi:noNamespaceSchemaLocation="http://schema.itrsgroup.com/GA2011.2-110303/netprobe.xsd">
   <selfAnnounce>
      <enabled>true</enabled>
      <probeName>Probe</probeName>
      <managedEntity>
         <name>ME</name>
         <types>
            <type>HW</type>
         </types>
      </managedEntity>
      <gateways>
         <gateway>
            <hostname>localhost</hostname>
            <port>7038</port>
            <secure>true</secure>
         </gateway>
      </gateways>
   </selfAnnounce>
</netprobe>

Open Access

Open Access is able to connect to both secure and insecure gateways. By default, Open Access connects to an insecure gateway. To connect to a secure gateway, simply add the secure=true flag and the correct secure port to the config/settings.conf file.

gateways {
    connection name {
        host = "hostname"        port = 30000
        user = "username"        password = "password"        secure = true
    }
    another connection name {
        secure = true
        primary {
            host = "primary_host"            port = 30002
        }
        secondary {
            host = "secondary_host"            port = 30003
        }
    }
}

Web Dashboard

Web Dashboard is able to connect to both secure and insecure gateways. To connect to a secure gateway, use the Configuration > Connections menu in the application and tick the “Secure” checkbox when adding a connection.

Alternatively, you can edit config.xml and add a secure tag as shown below.

...
<gatewayConnections>
     <gatewayConnection name="Gateway 1">
         <enabled>true</enabled>
         <secure>true</secure>
         <primary>
             <host>gateway-host</host>
             <port>7039</port>
         </primary>
         <secondary>
             <host>gateway-host-2</host>
             <port>7040</port>
         </secondary>
         <description/>
         <user/>
         <password/>
     </gatewayConnection>
             ...
</gatewayConnections>
...

Webslinger

Webslinger is able to connect to both secure and insecure gateways. It has two different lines in its configuration file for connections. Those that start GATEWAY=are used to specify connections to insecure gateways, while those that startSECURE_GATEWAY= are used to connect to secure gateways.

Auditor

Auditor is able to connect to both secure and insecure gateways. It has two different lines in its configuration file for connections. Those that start GATEWAY=are used to specify connections to insecure gateways, while those that startSECURE_GATEWAY= are used to connect to secure gateways.

Validating Geneos Components

As well as specifying the TLS certificates to use when creating a secure connection, some Geneos components support the verification of certificates used by the other party. This is done by providing a key chain of signing certificates via a PEM file to the Geneos component. This file is specified using the -ssl-certificate-chain command line option.

-ssl-certificate-chain <file>

The following Geneos components support this feature;

  • Gateway
  • Netprobe
  • WebSlinger
  • Licence Daemon

The Geneos component will check that all certificates presented to it during the establishment of a secure connection were signed by one of the signing certificates in the chain.

When a client connects to a server securely, the server must provide its certificate in order for a secure connection to be established. The client will always verify this certificate against it’s certificate chain (if it was provided one). If the certificate cannot be verified then the connection will be rejected.

When a server is connected securely to by a client, the client does not need to provide a certificate to the server. A Geneos component will provide this certificate if it has one but other clients (web browsers, API clients) may not. If the client does provided a certificate and the server has been provided with a chain of signing certificates, then the server will verify the client’s certificate against its certificate chain and reject the connection if the certificate cannot be verified. If the client does not provide a certificate, the server has nothing to check and will accept the secure connection.

Some Geneos servers can be configured to require that certificates are provided for certain types of connection. Currently the gateway and netprobe provide this functionality.

The netprobe can be configured to ensure that all incoming EMF2 connections to be established with a certificate. This allows the netprobes to ensure that they only accept connections from machines whose SSL certificate has been signed by the certificate authority used by the netprobe to verify certificates.

This is enabled by using the command line option with EMF2 as the argument:

-require-ssl-certificate-for emf2

If this setting is enabled then all Geneos components connecting to the netprobe will be required to provide a valid certificate. (There is one exception which is netprobes connection using the XMLRPC protocol which can be enabled as part of the Message Tracker plugin).

The gateway can be configured to ensure that require certificates for different incoming Geneos components. These are; netprobes, secondary gateways and importing gateways. The requirements are controlled from a section of the gateway setup. (In the Operating Enviroment section under the Connections tab)

Note: If either gateway or netprobe require certificates for an incoming connection of a specific type,

then all insecure connections of the specified type will be rejected as insecure connections do not provide a certificate.

Debugging Client Connections

If an error occurs connecting to a secure server, the server log is the place to look to understand the issue. The server is sent a message, and even if the message is sent to the incorrect port it can try to identify the type and place a useful message in its log.

Server log messages for a client trying to connect insecurely to a server using its secure port. This will occur both for SSL aware clients that have been mis-configured and old clients that predate the SSL enhancement:

ERROR: SSLConnector:accept SSL Error:[1408f10b] SSL3_GET_RECORD:wrong version number
WARN: SSLConnector:accept Possible attempt to connect insecurely to a secure port

Server log messages for a client trying to connect securely to a server using its insecure port:

ERROR: CONNECTOR ERROR: Secure Connection from '...' received on insecure port. Dropping connection

If the server being connected to is old and predates SSL support then the server will not print any log message. The SSL handshake will time out because the server has not responded correctly to the client. In this case there will be an error in the client log of the form:

ERROR: SSLConnector SSL Connection(=>127.0.0.1:19067) failed to complete handshake in 70s : Dropping connection

Web services

Certain Geneos Components provide web services. These web services can be accessed via HTTP or HTTPS depending on whether the service is published on a secure or an insecure port. Any web service that is published via the same port that is used for EMF connections will inherit the security settings from the EMF settings. So, for example if a netprobe is started with the -secure flag, then its web services will also be secure. The ORB debug pages will only be viewable via HTTPS. XML-API connections to the probe will need to use HTTPS rather than HTTP.

Hot Standby Gateways

Hot standby gateways connect to one another using Geneos’ EMF protocol. If one gateway is configured to use TLS security then both must be. If a secondary gateway is configured to listen on a secure port, then when it is connecting to the primary gateway it will attempt to connect securely. If the primary gateway is not also configured to listen on a secure port then the gateway will fail to connect.

Examples

Two setups are provided >HERE< to show how to setup Secure communications between Geneos components. In order to try these examples you will need a gateway, netprobe and Active console, all of version 3.1.X or higher. You will also need a copy of openssl to generate a set of certificates.

Following this example you will generate a pair of self signed certificates, however in production it is assumed that genuine certificates signed by a recognised certificate authority will be used.

Example 1

Example 1 has a netprobe and gateway running on the same computer, and a Active Console connected to them. All 3 componenets will need to be version 3.1.X or better. The assumption is that you are using the linux version of the gateway and netprobe. If you use a different version, please replace the names of the gateway and netprobe binaries in the commands below with the ones appropriate for your system.

First create a pair of certificates using openssl. One for the netprobe (np.pem) and one for the gateway (gw.pem).

Generate your own Self signed Certificate Authority

> openssl genrsa -out rootCA.key 2048
> openssl req -x509 -new -nodes -key rootCA.key -days 1024 -out rootCA.pem -subj "/CN=certificateAuthority"

Generate a certificate for your Netprobe

> openssl genrsa -out np.key 2048
> openssl req -new -key np.key -out np.csr -subj "/CN=netprobe"
> openssl x509 -req -in np.csr -CA rootCA.pem -CAkey rootCA.key -CAcreateserial -out np.crt -days 30 > cat np.crt np.key > np.pem

Generate a certificate for your Gateway

> openssl genrsa -out gw.key 2048
> openssl req -new -key gw.key -out gw.csr -subj "/CN=gateway"
> openssl x509 -req -in gw.csr -CA rootCA.pem -CAkey rootCA.key -CAcreateserial -out gw.crt -days 30 > cat gw.crt gw.key > gw.pem

Next copy the setup (eg1.setup.xml) into the gateway directory and the newly generated certificate (gw.pem) and the certificate of the CA (rootCA.pem). Now start the gateway as follows (please replace <LICD-HOST> by the location of your licence daemon);

> gateway2.linux -setup eg1.setup.xml -licd-host <LICD-HOST> -ssl-certificate ./gw.pem -ssl-certificate-chain ./rootCA.pem

This will start a directory running on the default port. You can connect to this gateway using the Active Console. Note that you will have to tick the “Secure Connection” option in the gateway connections dialog as the gateway has been configured only to listen on a secure port. See below;

You should find that you are connected to the gateway (which will be red). Looking at the Gateway Managed Entity you will see in the client view that the Active Console has connected securely to the gateway.

Looking at the GW view you will see that there are two gateway ports are listed. The insecurePort is set to 7039, indicating that the gateway is listening insecurely on that default insecure port, while the securePort value is 7038 indicating that the gateway is listening securely on the default secure gateway port.

Now you can start the netprobe. Copy the netprobe certificate (np.pem) and the certificate of the CA (rootCA.pem) into the netprobe directory start the netprobe by typing;

> netprobe.linux -ssl-certificate ./np.pem -secure -ssl-certificate-chain ./rootCA.pem -secure

The gateway will now connect to the probe and go green. If you now look Probes view on the Gateway Managed Entity you will see that the gateway has connected to the probe and the connection is secure.

The netprobe is now connected and all the netprobe functionality is available to the user via the gateway and the Active Console.

Example 2

Example 2 has a netprobe and gateway running on the same computer, and a Active Console connected to them. The difference is that the netprobe is configured to be a self announcing netprobe and so the netprobe will not require a certificate in order to perform secure communications. Agian all 3 componenets will need to be version 3.1.X or better. The assumption is that you are using the linux version of the gateway and netprobe. If you use a different version, please replace the names of the gateway and netprobe binaries in the commands below with the ones appropriate for your system.

Please follow the instructions from Example 1 to create the key pairs (you will only be using the gw.pem key pair).

Next copy the setup (eg2.setup.xml) into the gateway directory and the newly generated certificate (gw.pem). Now start the gateway as follows (please replace <LICD-HOST> by the location of your licence daemon);

> gateway2.linux -setup eg2.setup.xml -licd-host <LICD-HOST> -ssl-certificate ./gw.pem

This will start a directory running on the default port. You can connect to this gateway using the Active Console. Note that you will have to tick the “Secure Connection” option in the gateway connections dialog as the gateway has been configured only to listen on a secure port. See below;

You should find that you are connected to the gateway. Looking at the Gateway Managed Entity you will see in the client view that the Active Console has connected securely to the gateway.

Looking at the GW view you will see that there are two gateway ports are listed, just as in example 1. The insecurePort is set to 7039, indicating that the gateway is listening insecurely on that default insecure port, while the securePort value is 7038 indicating that the gateway is listening securely on the default secure gateway port.

Now you can start the self-announcing netprobe. Copy the netprobe setup from (eg2_np.setup.xml) into the netprobe directory and start the netprobe by typing;

> netprobe.linux -setup eg2_np.setup.xml

Looking at eg2.np.xml you will see that the gateway section specifies to connect securely to the gateway on port 7038.

<?xml version="1.0" encoding="ISO-8859-1"?>
<netprobe
   compatibility="1"
   xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
   xsi:noNamespaceSchemaLocation="http://schema.itrsgroup.com/GA2011.2-110303/netprobe.xsd">
   <selfAnnounce>
      <enabled>true</enabled>
      <retryInterval>20</retryInterval>
      <probeName>Probe</probeName>
      <managedEntity>
         <name>ME</name>
         <types>
            <type>HW</type>
         </types>
      </managedEntity>
      <gateways>
         <gateway>
            <hostname>localhost</hostname>
            <port>7038</port>
            <secure>true</secure>
         </gateway>
      </gateways>
   </selfAnnounce>
</netprobe

The Netprobe will connect to the gateway and a new row will appear in the Probes view indicating that a new Self-Announcing probe has connected and that it is connected securely.

Get the fingerprint and subject of an SSL Certificate

When connecting to the Gateway an SSL Identity may be required for authentication if an SSL Certificate is used. The identity can be the fingerprint or subject of an SSL Certificate. How to generate a fingerprint or subject (using Webslinger as an example) is shown below:

Fingerprint generation

> openssl x509 -in webslinger.pem -fingerprint -noout | sed 's/://g'
SHA1 Fingerprint=78B6CE43DF1D8C7A62BEAFFC988AFFA1EFC9DC7C

The fingerprint is all characters after '='. Copy this to the Gateway config.

Subject generation

> openssl x509 -in webslinger.pem -subject -noout
subject= /CN=Webslinger/C=UK/L=London/O=ITRS

The subject is all characters after "subject= ". Ensure there are no leading or trailing spaces. Copy this to the Gateway config.