Elasticsearch Setup for TeamConnect 7.0
Before you begin:
- If you are upgrading from a previous version, review the Upgrade Considerations for Elasticsearch.
- Read the Elasticsearch Administration Guide and Best Practices.
- Review the Installation Requirements for Elasticsearch, including supported versions of Elasticsearch.
- Make sure you are using Java 1.8.
- Users upgrading from previous versions of Elasticsearch and/or TeamConnect must re-index their instances. Refer to the Upgrade Considerations for specific instructions.
Note:
- Mitratech does not support multi-tenant Elasticsearch configurations.
For more detailed information on Elasticsearch, clusters and nodes, and global search indexing functions outside of TeamConnect, visit the Elasticsearch help and the latest support matrix for Elasticsearch compatibility.
Java (JVM) Version
Elasticsearch is built using Java, and includes a bundled version of OpenJDK from the JDK maintainers (GPLv2+CE) within each distribution. The bundled JVM is the recommended JVM and is located within the jdk directory of the Elasticsearch home directory.
To use your own version of Java, set the JAVA_HOME environment variable. If you must use a version of Java that is different from the bundled JVM, we recommend using a supported LTS version of Java. Elasticsearch will refuse to start if a known-bad version of Java is used. The bundled JVM directory may be removed when using your own JVM.
Upgrading Elasticsearch
Important! If you have an existing TeamConnect installation with Elasticsearch and you need to upgrade Elasticsearch, delete your search indices before proceeding.
Installing Elasticsearch
The following instructions apply to both Linux and Microsoft Windows installation of Elasticsearch.
- Download the appropriate installation file from the following website: https://www.elastic.co/downloads/past-releases/elasticsearch-7-16-3
- TeamConnect 7.0 is only certified against Elasticsearch 7.16.3.
- Extract the files into a directory (for example for Linux, /opt/elasticsearch/ or for Windows C:\...\Apache\Elasticsearch). Note that the elasticsearch.yml file will be located in the config subdirectory (for example, /opt/elasticsearch-7.16.3/config/elasticsearch.yml).
- Run the following command from the root Elasticsearch directory to install the ingest attachment plug-in:
- Navigate to elasticsearch-7.16.3/bin (Install ingest-attachment plugin for document search). Open Command prompt and run the elasticsearch-plugin install ingest-attachment command inside elasticsearch-7.16.3/bin folder and move according to instruction on command prompt.
- Configure the Elasticsearch properties and the TeamConnect properties.
- Open the appropriate ports in the firewall (http.port and transport.tcp.port).
- To run Elasticsearch, navigate to the bin directory of your Elasticsearch installation and start the elasticsearch.bat file. (Linux users may have to run it from Elasticseach's root directory.)
- Restart TeamConnect.
- Verify the configuration and creating the new indices.
If you encounter any bootstrap check failures, see Elasticsearch Bootstrap Checks.
Enabling Xpack
Xpack is optional for ElasticSearch implementation as Xpack enables SSL between all ElasticSearch and TeamConnect servers.
To enable Xpack, and thus SSL, visit this page: Enabling Elasticsearch Xpack Security (TCE 7.0).
Configuring Elasticsearch Properties
The following configuration details cover a basic installation of Elasticsearch for TeamConnect 7.0. If your situation requires a more robust configuration or if you are unable to proceed through the following configuration, contact Mitratech Support.
Many properties can be configured during the installation; however, viewing the configuration file in a text editor provides better visibility and custom configuration options.
To modify the Elasticsearch configuration file:
- Using a text editor, open the elasticsearch.yml file located in elasticsearch-7.16.3/config.
- For Linux, use a text editor such as vim or nano.
- For Windows, use a text editor of your choice.
- Referencing information in the Elasticsearch Configuration Properties table, uncomment and set a value to configure the following:
- Clusters and nodes - cluster.name property (for example, Cluster).
- Network connections
- Unicast discovery
- Xpack properties (if using Xpack). For detailed instructions on enabling Xpack visit this article: Enabling Elasticsearch Xpack
Elasticsearch Configuration Properties (elasticsearch.yml)
The following properties are commented out by default; to use these properties, remove the comment hashtag and modify the default value if necessary.
Note: Your Linux host might not start if all Elasticsearch specifications are not met. Users can work around this issue by adding the following line to the config/elasticsearch.yml file:
bootstrap.system_call_filter: false
| Property | Required | Recommendations | |
|---|---|---|---|
| XPack | xpack.security.enabled: false | x | Must be set to false. This will be set to true if you are using Xpack security. |
| Cluster | cluster.name | x | Use a descriptive name for your cluster, such as elasticsearch. This setting must match the elasticsearch.server.cluster.name property in the teamconnect.properties file. |
| cluster.max_shards_per_node | Default 1000, increase as needed | ||
| Node | node.name | x | The name of this specific node. |
| node.attr.rack | Adds custom attributes to the node | ||
| node.master | Allows the node to be master eligible. If needed, this property must be added manually. For further detail, see Nodes. | ||
| node.data | Allows the node to store data. | ||
| Paths | path.data | Path to directory where to store the data (separate multiple locations by comma) | |
| path.logs | Path to log files. If this value is not set, log information will not be saved anywhere. | ||
| Memory | bootstrap.memory_lock | x (Required on Linux) | Set to True. Locks the memory on startup. If the server is in a Linux environment, set this property to true. It locks the memory for Elasticsearch so that the JVM does not start swapping memory. Set heap size to half the memory available on the system. |
| Network | network.host | x | Sets the bind address to a specific IP (IPv4 or IPv6). This property should match the server listed in the elasticsearch.server.location property in the teamconnect.properties file. |
| http.host | Defaults to 9200. Set for a custom port for HTTP. | ||
| transport.tcp.port | Defaults to 9300. This property should match the port listed in the elasticsearch.server.location property in teamconnect.properties file. | ||
| Discovery | discovery.seed_hosts: ["host1", "host2"] | If clustering nodes, pass an initial list of hosts to perform discovery when this node is started. The default list of hosts is ["127.0.0.1", "[::1]"] If you are upgrading from Elasticsearch 5.3.0 to 7.10.2, the discovery.zen.ping.unicast.hosts property has been deprecated; use the value from this property for the new discovery.seed_hosts and cluster.initial_master_nodes properties. |
|
| cluster.initial_master_nodes: ["node-1", "node-2"] | If clustering nodes, bootstrap the cluster using an initial set of master-eligible nodes. If you are upgrading from Elasticsearch 5.3.0 to 7.10.2, the discovery.zen.ping.unicast.hosts property has been deprecated; use the value from this property for the new discovery.seed_hosts and cluster.initial_master_nodes properties. |
||
| discovery.zen.minimum_master_nodes | Defines the number of available master nodes. Equals the number of IP addresses identified for cluster.initial_master_nodes | ||
| Gateway | gateway.recover_after_nodes: 3 | Block initial recovery after a full cluster restart until N nodes are started. | |
| Various | action.destructive_requires_name: true | Require explicit names when deleting indices. |
| Property | Required | Recommendations | |
|---|---|---|---|
|
Xpack Properties Detailed instructions to Enabled Elasticsearch Xpack |
bootstrap.system_call_filter | x | false |
| xpack.security.enabled | x | true | |
| xpack.security.transport.ssl.enabled | x | true | |
| xpack.security.transport.ssl.verification_mode | x | certificate | |
| xpack.security.transport.ssl.keystore.path | x | /opt/elasticsearch/elasticsearch-7.16.3/config/certs/elastic-certificates.p12 | |
| xpack.security.transport.ssl.truststore.path | x | /opt/elasticsearch/elasticsearch-7.16.3/config/certs/elastic-certificates.p12 | |
| xpack.security.http.ssl.enabled | x | true | |
| xpack.security.http.ssl.keystore.path | x | /opt/elasticsearch/elasticsearch-7.16.3/config/certs/elastic-certificates.p12 | |
| xpack.security.http.ssl.truststore.path | x | /opt/elasticsearch/elasticsearch-7.16.3/config/certs/elastic-certificates.p12 | |
| xpack.security.http.ssl.client_authentication | x | optional |
Configuring TeamConnect Properties
The teamconnect.properties file can be configured for Global Search optimization. This information can be entered automatically into the teamconnect.properties file during the installation of TeamConnect or manually entered after installation.
To modify the TeamConnect configuration file:
- Using a text editor, open the teamconnect.properties file in the TeamConnect .WAR file.
- For Linux, use a text editor such as vim or nano.
- For Windows, use a text editor of your choice.
- Referencing information in the TeamConnect Configuration Properties table below, uncomment and set a value to configure the following:
- Elasticsearch server location
- Elasticsearch server cluster name
- Indexing frequency
- Primary and replica shards
- Elasticsearch SSL (only if using XPack)
TeamConnect Configuration Properties (teamconnect.properties)
If you want to change any of the following properties and the property is commented out, remove the comment hashtag and modify the default value if necessary.
To configure the properties after TeamConnect is already installed, modify the properties in the teamconnect.properties file in the TeamConnect .WAR file and then redeploy the .WAR file.
Note: The elasticsearch port needs to be set to 9200. If it was set to 9300, make sure you change it or the integration will not work.
| Property | Required | Recommendations and Comments | |
|---|---|---|---|
| Elasticsearch | elasticsearch.server.location | x | IP address and port number of the Elasticsearch server. 10.100.200.234:9200 OR 10.100.200.234:9200, 10.100.200.235:9200 (continue to add ' , IP:port ' for additional servers) |
| elasticsearch.server.cluster.name | x | Name of the cluster that can be found in the Elasticsearch configuration (.yml) file. | |
| elasticsearch.index.frequency | x | Frequency (in seconds) in which TeamConnect looks for changes to existing, enabled indexed items and automatically updates them. For example, if set to 10, the index will be scanned for modifications, additions, and entry removal every ten seconds. Default value is 10. | |
| elasticsearch.index.shards.primary | x | The number of primary shards for the index. An ideal maximum shard size is 40-50 GB. For example, if an index size is 500 GB, you would have at least 10 primary shards. Default value is 1. | |
| elasticsearch.index.shards.replica | x | The number of replica shards for the index. The number of replica shards defaults to a value of '1'. Entering in any invalid character here will thereby result in 1 replica shard. Default value is 0. | |
| elasticsearch.client.bulkProcessor. concurrentRequests |
The number of concurrent requests to be used for indexing. This number refers to the number of threads running on the ElasticSearch server. Default value is 4. | ||
| elasticsearch.client.bulkProcessor. bulkSize |
x | The bulk size for flushing requests to Elasticsearch in MB. Default value is 25. | |
| elasticsearch.protocol | x | Defaults to https. | |
| elasticsearch.bypass.host.setting | x | NO | |
| elasticsearch.hostname | x | Set hostname or IP for Elasticsearch. | |
| elasticsearch.port | x | Defaults to 9200. Leave it at 9200. If it is set to 9300, change it to 9200. | |
| elasticsearch.encryption.enabled.xpack | x | The property for enabling/disabling Elasticsearch security during transit. If not using Xpack, set to NO. If using Xpack, set to YES. Default value is NO. |
|
| Set SSL parameters for Elasticsearch if Xpack is enabled. If Xpack is set to NO, these parameters will be ignored. | elasticsearch.client.keystore.password | Password for your Elasticsearch KeyStore file. | |
| elasticsearch.client.truststore.password | Password for your Elasticsearch TrustStore file. | ||
| elasticsearch.keystore.path | File path for your Elasticsearch KeyStore file. | ||
| elasticsearch.truststore.path | File path for your Elasticsearch TrustStore file. | ||
| elasticsearch.xpack.ssl.username | Username used for Xpack password protection authentication. | ||
| elasticsearch.xpack.ssl.password | Password used for Xpack password protection authentication. |
Note: The concurrent requests and bulk size for flushing requests are not required and do not have default values.
Verifying the Configuration and Creating Indices
After Elasticsearch has been configured:
- Restart the application and clear the cache.
- Log into TeamConnect and navigate to the Global Search Index Tool from the Tools menu.
- Below the Object Indexing Status table, the UUID should display.
- Click the Test Connection button. If successful, a "Success" message appears.
- Select the objects you want to index and then click Index Selected button.