Elasticsearch Setup for TeamConnect 6.1
These installation instructions pertain to TeamConnect 6.1 and include information on Elasticsearch as well as the optional Elasticsearch SearchGuard.
Before you begin:
- Read the Installation Overview
- If you are upgrading from a previous version, pay attention to the Upgrade Considerations for Elasticsearch.
- Read the Elasticsearch Administration Guide and Best Practices
- Review the Installation Requirements for Elasticsearch, including supported versions of Elasticsearch
- Make sure you are using Java 1.8.
- Users upgrading from previous versions of Elasticsearch and/or TeamConnect must re-index their instances. Refer to the Upgrade Considerations for specific instructions.
Note:
- Mitratech does not support multi-tenant Elasticsearch configurations.
For more detailed information on Elasticsearch, clusters and nodes, and global search indexing functions outside of TeamConnect, visit the Elasticsearch help and the latest support matrix for Elasticsearch compatibility.
Installing Elasticsearch
The following instructions apply to both Linux and Microsoft Windows installation of Elasticsearch. If your system is unable to perform this type of installation, contact Mitratech Support at support@mitratech.com. For installations with Search Guard, see Installing Search Guard.
- Download the appropriate installation file from the following website: https://www.elastic.co/downloads/pas...icsearch-5-3-0. TeamConnect is only certified against Elasticsearch 5.3.0.
- Extract the files into a directory (for example for Linux, /opt/elasticsearch/ or for Windows C:\...\Apache\Elasticsearch). Note that the elasticsearch.yml file will be located in the config subdirectory (for example, /opt/elasticsearch-5.3.0/config/elasticsearch.yml).
- Run the following command from the root Elasticsearch directory to install the mapper attachment plug-in:
bin\elasticsearch-plugin install mapper-attachments
This plug-in provides the 'attachment' field type to Elasticsearch, thereby allowing it to index to content of documents like PDF files and Microsoft Word files. - If using Search Guard, install Search Guard and configure the Search Guard properties.
- Configure the Elasticsearch properties and the TeamConnect properties.
- Open the ports that are being used in the firewall (http.port and transport.tcp.port).
- To run Elasticsearch, navigate to the bin directory of your Elasticsearch installation and start the elasticsearch.bat file. (Linux users may have to run it from Elasticseach's root directory.)
If you encounter any bootstrap check failures, see Elasticsearch Bootstrap Checks.
Installing Search Guard
For the most recent instructions for installing and configuring Search Guard, see Installing Search Guard.
Configuring Elasticsearch Properties
The following configuration details cover a basic, functional installation of Elasticsearch for TeamConnect 6.1. If your situation requires a more robust configuration or if you are unable to proceed through the following configuration, contact Mitratech Support.
Many properties can be configured during the installation; however, viewing the configuration file in a text editor provides better visibility and custom configuration options. The configuration file is the same for both Windows and Linux.
To modify the Elasticsearch configuration file:
- If using Search Guard, make sure you have placed the signing authority, node keystore and truststore files (for example, signing-ca.crt, node-0-keystore.jks, and truststore.jks) into the elasticsearch-5.3.0/config directory.
- Using a text editor, open the elasticsearch.yml file located in elasticsearch-5.3.0/config.
- For Linux, use a text editor such as vim or nano.
- For Windows, use a text editor of your choice.
- Referencing information in the Elasticsearch Configuration Properties table, uncomment and set a value to configure the following:
- Clusters and nodes - cluster.name property (for example, Cluster).
- Network connections
- Unicast discovery
- Search Guard properties (if using Search Guard)
Elasticsearch Configuration Properties (elasticsearch.yml)
The following properties are commented out; in order to use these properties, remove the comment hashtag and modify the default value if necessary.
Note: Your Linux host might not start if all Elasticsearch specifications are not met. It is not highly recommended, but users can work around this issue by adding the following line to the config/elasticsearch.yml file:
bootstrap.system_call_filter: false
| Property | Required | Recommendations | |
|---|---|---|---|
| Cluster | cluster.name | x | Use a descriptive name for your cluster, such as elasticsearch. This setting must match the elasticsearch.server.cluster.name property in the teamconnect.properties file. |
| Node | node.name | x | The name of this specific node. If using Search Guard, this name should match your keystore certificate name. For example, set node.name: node-0 in the elasticsearch.yml file and name your keystore file node-0-keystore.jks. |
| node.att.rack | Adds custom attributes to the node | ||
| node.master | Allows the node to be master eligible. If needed, this property must be added manually. For further detail, see Nodes. | ||
| node.data | Allows the node to store data. | ||
| Paths | path.data | Path to directory where to store the data (separate multiple locations by comma) | |
| path.logs | Path to log files. If this value is not set, log information will not be saved anywhere. | ||
| Memory | bootstrap.memory_lock | x (Requited if on Linux system or if using Search Guard) | Locks the memory on startup. If the server is in a Linux environment or if using Search Guard, set this property to true. It locks the memory for Elasticsearch so that the JVM does not start swapping memory. |
| Network | network.host | x | Sets the bind address to a specific IP (IPv4 or IPv6). This property should match the server listed in the elasticsearch.server.location property in the teamconnect.properties file. |
| http.port | This property should match the port listed in the elasticsearch.server.location property in teamconnect.properties file. | ||
| transport.tcp.port | This property should match the port listed in the elasticsearch.server.location property in teamconnect.properties file. | ||
| Discovery | discovery.zen.ping.unicast.hosts | x | Provide the entry points into the cluster by specifying the server locations here. The default list of hosts is ["127.0.0.1", "[::1]"] |
| discovery.zen. minimum_master_nodes |
Prevent split brain by configuring the majority of nodes (total number of master-eligible nodes / 2 + 1) As a best practice, have at least 3 nodes and set minimum_master_nodes to 2. If there are only 2 nodes and one goes down (and minimum_master_nodes is set to 2), there will not be enough master nodes available and new data sent to Elasticsearch will not be processed. |
| Property | Required | Recommendations | |
|---|---|---|---|
| Search Guard Properties Note: Only use if you are using Search Guard |
searchguard.ssl.transport. keystore_filepath |
x | The name of the node and reference the name of your node certificate. |
| searchguard.ssl.transport. keystore_password |
x | The password used when generating the keystore. Passwords are stored as clear text. |
|
| searchguard.ssl.transport. truststore_filepath |
x | Use truststore.jks as the value. | |
| searchguard.ssl.transport. truststore_password |
x | The password used when generating the keystore. | |
| searchguard.ssl.transport. enforce_hostname_verification |
x | Use false for the value. | |
| searchguard.ssl.transport. keystore_alias |
x | Set the alias to match the alias of your node's keystore (See node.name) | |
| searchguard.authcz. impersonation_dn |
x | This setting configures the admin certificate that you can use with sgadmin.This setting references the certificate name of your certificate (which is set to "esdev" in the example). The format is "CN=<client_username>,OU=client, O=client,L=Test,C=DE": - '*' where <client_username> is the username (for example, "esdev") and references the certificate name of your certificate (for example, esdev-keystore.jks) |
Configuring TeamConnect Properties
The teamconnect.properties file can be configured for Global Search optimization. This information can be entered automatically into the teamconnect.properties file during the installation of TeamConnect or manually entered after installation.
To modify the TeamConnect configuration file:
- If using Search Guard, make sure you have placed the client certificate, client keystore and truststore files (for example, signing-ca.crt, esdev-keystore.jks and truststore.jks) in a directory on your PC that is accessible by TeamConnect (for example, config/certs).
- Using a text editor, open the teamconnect.properties file in the TeamConnect .WAR file.
- For Linux, use a text editor such as vim or nano.
- For Windows, use a text editor of your choice.
- Referencing information in the TeamConnect Configuration Properties table, uncomment and set a value to configure the following:
- Elasticsearch server location
- Elasticsearch server cluster name
- Indexing frequency
- Primary and replica shards
- Elasticsearch SSL (only if using Search Guard)
TeamConnect Configuration Properties (teamconnect.properties)
If you want to change any of the following propertiesand the property is commented out, remove the comment hashtag and modify the default value if necessary.
To configure the properties after TeamConnect is already installed, modify the properties in the teamconnect.properties file in the TeamConnect .WAR file and then redeploy the .WAR file.
| Property | Required | Recommendations and Comments | |
|---|---|---|---|
| Elasticsearch | elasticsearch.server.location | x | IP address and port number of the Elasticsearch server |
| elasticsearch.server.cluster.name | x | Name of the cluster that can be found in the Elasticsearch configuration file. | |
| elasticsearch.index.frequency | x | Frequency (in seconds) in which TeamConnect looks for changes to existing, enabled indexed items and automatically updates them. For example, if set to 10, the index will be scanned for modifications, additions, and entry removal every ten seconds. Default value is 10. | |
| elasticsearch.index.shards.primary | x | The number of primary shards for the index. An ideal maximum shard size is 40-50 GB. For example, if an index size is 500 GB, you would have at least 10 primary shards. Default value is 1. | |
| elasticsearch.index.shards.replica | x | The number of replica shards for the index. The number of replica shards defaults to a value of '1'. Entering in any invalid character here will thereby result in 1 replica shard. Default value is 0. | |
| elasticsearch.client.bulkProcessor. concurrentRequests |
The number of concurrent requests to be used for indexing. This number refers to the number of threads running on the ElasticSearch server. Default value is 4. | ||
| elasticsearch.client.bulkProcessor. bulkSize |
x | The bulk size for flushing requests to Elasticsearch in MB. Default value is 25. | |
| elasticsearch.encryption.enabled | x | The property for enabling/disabling Elasticsearch security during transit. If not using Search Guard, set to NO to disable Search Guard. If using Search Guard or using SSL, set to YES to enable Search Guard. Default value is NO. |
|
|
Elasticsearch SSL Note: Only use these properties if you are using Search Guard |
elasticsearch.client.keystore. password |
x | The password for your Elasticsearch KeyStore file after being encypted with encrypt.jar. |
| elasticsearch.client.truststore. password |
x | The password for your Elasticsearch TrustStore file after being encypted with encrypt.jar. | |
| elasticsearch.keystore.path | x | The actual location and certificate name of your certificate. The file path and certificate name of your certificate. For example, config/certs/esdev-keystore. | |
| elasticsearch.truststore.path | x | The file path for your Elasticsearch TrustStore file. You can use absolute path or relative path. For example, config/certs/truststore.jks. | |
| elasticsearch.transport.username | x | The username paired with Search Guard for basic transport authentication. This "username" user is also used when populating the sg_internal_users.yml and sg_roles_mapping.yml files. |
|
| elasticsearch.transport.password | x | The password paired with Search Guard for basic transport authentication after being encypted with encrypt.jar. This password is used when populating the sg_internal_users.yml file. |
|
| elasticsearch.client.notification. |
The email set here will receive certificate expiration notices. |
Note: The concurrent requests and bulk size for flushing requests are not required and do not have default values.
Verifying the Configuration and Creating Indices
After Elasticsearch has been configured:
- Restart the application and clear the cache.
- Log into TeamConnect and navigate to the Global Search Index Tool from the Tools menu.
- Below the Object Indexing Status table, the UUID should display.
- Click the Test Connection button. If successful, a "Success" message appears.
- Select the objects you want to index and then click Index Selected button.