Elasticsearch Setup for TeamConnect 6.2
These installation instructions pertain to TeamConnect 6.2 and include information on Elasticsearch 7.1.1 as well as the optional Elasticsearch SearchGuard.
Before you begin:
- Read the Installation Overview
- If you are upgrading from a previous version, review the Upgrade Considerations for Elasticsearch.
- Read the Elasticsearch Administration Guide and Best Practices.
- Review the Installation Requirements for Elasticsearch, including supported versions of Elasticsearch.
- Make sure you are using Java 1.8.
- Users upgrading from previous versions of Elasticsearch and/or TeamConnect must re-index their instances. Refer to the Upgrade Considerations for specific instructions.
- If using Tomcat and Elasticsearch, the following files need to be added to the tomcat/lib directory:
- bcpg-jdk15on-1.61.jar
- bcprov-jdk15on-1.61.jar
Note:
- Mitratech does not support multi-tenant Elasticsearch configurations.
For more detailed information on Elasticsearch, clusters and nodes, and global search indexing functions outside of TeamConnect, visit the Elasticsearch help and the latest support matrix for Elasticsearch compatibility.
Upgrading Elasticsearch
Important! If you have an existing TeamConnect installation with Elasticsearch and you need to upgrade Elasticsearch, delete your search indices before proceeding.
Installing Elasticsearch
The following instructions apply to both Linux and Microsoft Windows installation of Elasticsearch. For installations with Search Guard, see Installing Search Guard.
- Download the appropriate installation file from the following website: https://www.elastic.co/downloads/past-releases/elasticsearch-7-1-1. TeamConnect 6.2 is only certified against Elasticsearch 7.1.1.
- Extract the files into a directory (for example for Linux, /opt/elasticsearch/ or for Windows C:\...\Apache\Elasticsearch). Note that the elasticsearch.yml file will be located in the config subdirectory (for example, /opt/elasticsearch-7.1.1/config/elasticsearch.yml).
- Run the following command from the root Elasticsearch directory to install the ingest attachment plug-in:
bin\elasticsearch-plugin install ingest-attachment - If using Search Guard, install Search Guard for TeamConnect 6.2 and configure the Search Guard properties.
- Configure the Elasticsearch properties and the TeamConnect properties.
- Open the appropriate ports in the firewall (http.port and transport.tcp.port).
- To run Elasticsearch, navigate to the bin directory of your Elasticsearch installation and start the elasticsearch.bat file. (Linux users may have to run it from Elasticseach's root directory.)
- Restart TeamConnect.
- Verify the configuration and creating the new indices.
If you encounter any bootstrap check failures, see Elasticsearch Bootstrap Checks.
Installing Search Guard
For the most recent instructions for installing and configuring Search Guard, see Installing Search Guard for TeamConnect 6.2.
Configuring Elasticsearch Properties
The following configuration details cover a basic installation of Elasticsearch for TeamConnect 6.2. If your situation requires a more robust configuration or if you are unable to proceed through the following configuration, contact Mitratech Support.
Many properties can be configured during the installation; however, viewing the configuration file in a text editor provides better visibility and custom configuration options.
To modify the Elasticsearch configuration file:
- If using Search Guard, make sure you have placed the signing authority, node keystore, and truststore files into the elasticsearch-7.1.1/config directory. For example, these files may be signing-ca.crt, node-0-keystore.jks, and truststore.jks.
- Using a text editor, open the elasticsearch.yml file located in elasticsearch-7.1.1/config.
- For Linux, use a text editor such as vim or nano.
- For Windows, use a text editor of your choice.
- Referencing information in the Elasticsearch Configuration Properties table, uncomment and set a value to configure the following:
- Clusters and nodes - cluster.name property (for example, Cluster).
- Network connections
- Unicast discovery
- Search Guard properties (if using Search Guard)
Elasticsearch Configuration Properties (elasticsearch.yml)
The following properties are commented out by default; to use these properties, remove the comment hashtag and modify the default value if necessary.
Important: If you are upgrading from Elasticsearch 5.3.0 to 7.1.1, the discovery.zen.ping.unicast.hosts property has been deprecated; use the value from this property for the new discovery.seed_hosts and cluster.initial_master_nodes properties.
Note: Your Linux host might not start if all Elasticsearch specifications are not met. Users can work around this issue by adding the following line to the config/elasticsearch.yml file:
bootstrap.system_call_filter: false
| Property | Required | Recommendations | |
|---|---|---|---|
| XPack | xpack.security.enabled: false | x | Must be set to false. |
| Cluster | cluster.name | x | Use a descriptive name for your cluster, such as elasticsearch. This setting must match the elasticsearch.server.cluster.name property in the teamconnect.properties file. |
| Node | node.name | x | The name of this specific node. If using Search Guard, this name should match your keystore certificate name. For example, set node.name: node-0 in the elasticsearch.yml file and name your keystore file node-0-keystore.jks. |
| node.att.rack | Adds custom attributes to the node | ||
| node.master | Allows the node to be master eligible. If needed, this property must be added manually. For further detail, see Nodes. | ||
| node.data | Allows the node to store data. | ||
| Paths | path.data | Path to directory where to store the data (separate multiple locations by comma) | |
| path.logs | Path to log files. If this value is not set, log information will not be saved anywhere. | ||
| Memory | bootstrap.memory_lock | x (Requited if on Linux system or if using Search Guard) | Locks the memory on startup. If the server is in a Linux environment or if using Search Guard, set this property to true. It locks the memory for Elasticsearch so that the JVM does not start swapping memory. |
| Network | network.host | x | Sets the bind address to a specific IP (IPv4 or IPv6). This property should match the server listed in the elasticsearch.server.location property in the teamconnect.properties file. |
| http.host | Defaults to 9200. Set for a custom port for HTTP. | ||
| transport.host | Defaults to 9300. Set for a custom port | ||
| transport.tcp.port | This property should match the port listed in the elasticsearch.server.location property in teamconnect.properties file. | ||
| Discovery | discovery.seed_hosts: ["host1", "host2"] | If clustering nodes, pass an initial list of hosts to perform discovery when this node is started. The default list of hosts is ["127.0.0.1", "[::1]"] If you are upgrading from Elasticsearch 5.3.0 to 7.1.1, the discovery.zen.ping.unicast.hosts property has been deprecated; use the value from this property for the new discovery.seed_hosts and cluster.initial_master_nodes properties. |
|
| cluster.initial_master_nodes: ["node-1", "node-2"] | If clustering nodes, bootstrap the cluster using an initial set of master-eligible nodes. If you are upgrading from Elasticsearch 5.3.0 to 7.1.1, the discovery.zen.ping.unicast.hosts property has been deprecated; use the value from this property for the new discovery.seed_hosts and cluster.initial_master_nodes properties. |
||
| Gateway | gateway.recover_after_nodes: 3 | Block initial recovery after a full cluster restart until N nodes are started. | |
| Various | action.destructive_requires_name: true | Require explicit names when deleting indices. |
| Property | Required | Recommendations | |
|---|---|---|---|
| Search Guard Properties Note: Only use if you are using Search Guard |
searchguard.enterprise_modules_enabled | x | false This must be set in order to use the free Search Guard Community edition. |
| searchguard.ssl.transport.keystore_type | x | jks | |
| searchguard.ssl.transport. keystore_filepath |
x | The name of the node and reference the name of your node certificate. | |
| searchguard.ssl.transport. keystore_password |
x | The password used when generating the keystore. Passwords are stored as clear text. |
|
| searchguard.ssl.transport.truststore_type | x | jks | |
| searchguard.ssl.transport. truststore_filepath |
x | Use truststore.jks as the value. | |
| searchguard.ssl.transport. truststore_password |
x | The password used when generating the keystore. | |
| searchguard.ssl.transport. keystore_alias |
x | Set the alias to match the alias of your node's keystore | |
| searchguard.ssl.transport. enforce_hostname_verification |
x | Use false for the value. | |
| searchguard.authcz. admin_dn |
x | This setting configures the admin certificate that you can use with sgadmin.This setting references the certificate name of your certificate (which is set to "esdev" in the example). The format is "CN=<client_username>,OU=client, O=client,L=Test,C=DE": - '*' where <client_username> is the username (for example, "esdev") and references the certificate name of your certificate (for example, esdev-keystore.jks) |
|
| searchguard.ssl.http.enabled | x | true | |
| searchguard.ssl.http.keystore_type | x | jks | |
| searchguard.ssl.http.keystore_filepath | x | The name of the node keystore, references the name of the node certificate. For example, node-0-keystore.jks | |
| searchguard.ssl.http.keystore_password | x | The keystore password used when generating the keystore. For example, changeit | |
| searchguard.ssl.http.keystore_alias | x | Set the alias to match the alias name of your node's keystore. | |
| searchguard.ssl.http.truststore_type | x | jks | |
| searchguard.ssl.http.truststore_password | x | Password used when generating the truststore. For example, changeit | |
| searchguard.ssl.http.truststore_alias | x | Set the alias to match the alias name of your truststore's keystore. | |
| searchguard.ssl.http.truststore_filepath | x | truststore.jks |
Configuring TeamConnect Properties
The teamconnect.properties file can be configured for Global Search optimization. This information can be entered automatically into the teamconnect.properties file during the installation of TeamConnect or manually entered after installation.
To modify the TeamConnect configuration file:
- If using Search Guard, make sure you have placed the client certificate, client keystore and truststore files (for example, signing-ca.crt, esdev-keystore.jks and truststore.jks) in a directory on your PC that is accessible by TeamConnect (for example, config/certs).
- Using a text editor, open the teamconnect.properties file in the TeamConnect .WAR file.
- For Linux, use a text editor such as vim or nano.
- For Windows, use a text editor of your choice.
- Referencing information in the TeamConnect Configuration Properties table, uncomment and set a value to configure the following:
- Elasticsearch server location
- Elasticsearch server cluster name
- Indexing frequency
- Primary and replica shards
- Elasticsearch SSL (only if using Search Guard)
TeamConnect Configuration Properties (teamconnect.properties)
If you want to change any of the following properties and the property is commented out, remove the comment hashtag and modify the default value if necessary.
To configure the properties after TeamConnect is already installed, modify the properties in the teamconnect.properties file in the TeamConnect .WAR file and then redeploy the .WAR file.
| Property | Required | Recommendations and Comments | |
|---|---|---|---|
| Elasticsearch | elasticsearch.server.location | x | IP address and port number of the Elasticsearch server. |
| elasticsearch.server.cluster.name | x | Name of the cluster that can be found in the Elasticsearch configuration file. | |
| elasticsearch.index.frequency | x | Frequency (in seconds) in which TeamConnect looks for changes to existing, enabled indexed items and automatically updates them. For example, if set to 10, the index will be scanned for modifications, additions, and entry removal every ten seconds. Default value is 10. | |
| elasticsearch.index.shards.primary | x | The number of primary shards for the index. An ideal maximum shard size is 40-50 GB. For example, if an index size is 500 GB, you would have at least 10 primary shards. Default value is 1. | |
| elasticsearch.index.shards.replica | x | The number of replica shards for the index. The number of replica shards defaults to a value of '1'. Entering in any invalid character here will thereby result in 1 replica shard. Default value is 0. | |
| elasticsearch.client.bulkProcessor. concurrentRequests |
The number of concurrent requests to be used for indexing. This number refers to the number of threads running on the ElasticSearch server. Default value is 4. | ||
| elasticsearch.client.bulkProcessor. bulkSize |
x | The bulk size for flushing requests to Elasticsearch in MB. Default value is 25. | |
| elasticsearch.encryption.enabled | x | The property for enabling/disabling Elasticsearch security during transit. If not using Search Guard, set to NO to disable Search Guard. If using Search Guard or using SSL, set to YES to enable Search Guard. Default value is NO. |
|
|
Elasticsearch SSL Note: Only use these properties if you are using Search Guard |
elasticsearch.client.keystore. password |
x | The password for your Elasticsearch KeyStore file after being encypted with encrypt.jar. |
| elasticsearch.client.truststore. password |
x | The password for your Elasticsearch TrustStore file after being encypted with encrypt.jar. | |
| elasticsearch.keystore.path | x | The actual location and certificate name of your certificate. The file path and certificate name of your certificate. For example, config/certs/esdev-keystore. | |
| elasticsearch.truststore.path | x | The file path for your Elasticsearch TrustStore file. You can use absolute path or relative path. For example, config/certs/truststore.jks. | |
| elasticsearch.transport.username | x | The username paired with Search Guard for basic transport authentication. This "username" user is also used when populating the sg_internal_users.yml and sg_roles_mapping.yml files. |
|
| elasticsearch.transport.password | x | The password paired with Search Guard for basic transport authentication after being encypted with encrypt.jar. This password is used when populating the sg_internal_users.yml file. |
|
| elasticsearch.client.notification. |
The email set here will receive certificate expiration notices. |
Note: The concurrent requests and bulk size for flushing requests are not required and do not have default values.
Verifying the Configuration and Creating Indices
After Elasticsearch has been configured:
- Restart the application and clear the cache.
- Log into TeamConnect and navigate to the Global Search Index Tool from the Tools menu.
- Below the Object Indexing Status table, the UUID should display.
- Click the Test Connection button. If successful, a "Success" message appears.
- Select the objects you want to index and then click Index Selected button.