How Indexing Works
Indexing
This page provides reference material on how TeamConnect works with Global Search indexing. For instructions on building indices, see Global Search Index Tool.
Starting in TeamConnect 6.1, TeamConnect Objects each have their own Index. Details on the move to an index per object can be found here.
An index is a database-like structure which defines the correlating teamconnect object as a searchable json object. Each index represents a single object type, meaning you will have the same number of indices as teamconnect objects being indexed.
System Objects
All system objects are indexed, except for Accounts, Admin Settings, Expense, Group and User. Following is the list of all System Fields which are indexed. The table shows which fields are indexed and searchable. Note: there are system fields that are automatically added to new custom objects when they are created. That list of system fields is represented under the generic custom object example, "Project", in the table. If you need to be able to search for a field that is not currently searchable, please submit a support ticket.
Object Name | Field Name | Indexed | Searchable |
Appointments |
notes |
X | X |
project.gid |
X | X | |
_index |
X | X | |
_feature |
X | ||
displayName |
X | X | |
subject |
X | X | |
project |
X | ||
categories.name |
X | X | |
project.name |
X | X | |
denyUserAcl, |
X | X | |
_ignored |
X | X | |
modifiedOn |
X | X | |
allDayEvent |
X | X | |
emailNotificationActive |
X | X | |
emailAlertNumDaysAdvance |
X | X | |
_data_stream_timestamp |
X | ||
id |
X | x | |
categories |
X | ||
_version |
X | ||
_routing |
X | X | |
visibility |
X | X | |
allowUserAcl |
X | X | |
_type |
X | X | |
_seq_no |
X | X | |
createdOnBehalfOf |
X | X | |
createdBy |
X | X | |
_field_names, |
X | X | |
allowGroupAcl |
X | X | |
denyGroupAcl |
X | X | |
location |
X | X | |
_source |
X | ||
_id |
X | X | |
categories.treePosition |
X | X | |
Contacts |
note |
X | X |
_index |
X | X | |
_feature |
X | ||
displayName |
X | X | |
categories.name |
X | X | |
denyUserAcl |
X | X | |
title |
X | X | |
idNumber |
X | X | |
_ignored |
X | X | |
modifiedOn |
X | X | |
_data_stream_timestamp |
X | ||
driverLicense |
X | X | |
company |
X | ||
company.name |
X | X | |
id |
X | X | |
categories |
X | ||
socialSecurity |
X | X | |
_version |
X | ||
primaryEmail |
X | X | |
_routing |
X | X | |
visibility |
X | X | |
allowUserAcl |
X | X | |
_type |
X | X | |
taxNumber |
X | X | |
primaryPhoneNumber |
X | X | |
company.gid |
X | X | |
_seq_no |
X | X | |
createdOnBehalfOf |
X | X | |
createdBy |
X | X | |
_field_names |
X | X | |
allowGroupAcl |
X | X | |
name |
X | X | |
denyGroupAcl |
X | X | |
_source |
X | ||
_primaryAddress |
X | X | |
_id |
X | X | |
categories.treePosition | X | X | |
Documents |
extension |
X | X |
_index |
X | X | |
_feature |
X | ||
displayName |
X | X | |
description |
X | X | |
categories.name |
X | X | |
denyUserAcl |
X | X | |
content |
X | X | |
_ignored |
X | X | |
modifiedOn |
X | X | |
attachment |
X | ||
_data_stream_timestamp |
X | ||
id |
X | X | |
categories |
X | ||
_version |
X | ||
_routing |
X | X | |
visibility |
X | X | |
allowUserAcl |
X | X | |
_type |
X | X | |
_seq_no |
X | X | |
filename |
X | X | |
size |
X | X | |
createdOnBehalfOf |
X | X | |
createdBy |
X | X | |
_field_names |
X | X | |
allowGroupAcl |
X | X | |
attachment.content |
X | X | |
name |
X | X | |
denyGroupAcl |
X | X | |
_source |
X | ||
_id |
X | X | |
categories.treePosition |
X | X | |
Histories |
_index |
X | X |
_feature |
X | ||
displayName |
X | X | |
description |
X | X | |
categories.name |
X | X | |
denyUserAcl |
X | X | |
_ignored |
X | X | |
modifiedOn |
X | X | |
_data_stream_timestamp |
X | ||
id |
X | X | |
categories |
X | ||
_version |
X | ||
_routing |
X | X | |
visibility |
X | X | |
allowUserAcl |
X | X | |
_type |
X | X | |
_seq_no |
X | X | |
createdOnBehalfOf |
X | X | |
createdBy |
X | X | |
_field_names |
X | X | |
allowGroupAcl |
X | X | |
denyGroupAcl |
X | X | |
_source |
X | ||
_id |
X | X | |
categories.treePosition |
X | X | |
Invoices |
notes |
X | X |
_index |
X | X | |
_feature |
|||
lineItems.timekeeper.gid |
X | X | |
displayName |
X | X | |
categories.name |
X | X | |
lineItems.activity.treePosition |
X | X | |
denyUserAcl |
X | X | |
submittedCurrency.treePosition |
X | X | |
lineItems.timekeeper.name |
X | X | |
lineItems.project.gid |
X | X | |
vendor.gid |
X | X | |
lineItems.activity |
X | ||
_ignored |
X | X | |
lineItems |
X | ||
submittedCurrency.name |
X | X | |
modifiedOn |
X | X | |
lineItems.activity.name |
X | X | |
lineItems.isAppealed |
X | X | |
_data_stream_timestamp |
X | ||
vendor |
X | ||
invoiceNumber |
X | X | |
submittedCurrency.localizedNames |
X | X | |
id |
X | X | |
categories |
X | ||
lineItems.note |
X | X | |
receivedDate |
X | X | |
_version |
X | ||
_routing |
X | X | |
visibility |
X | X | |
allowUserAcl |
X | X | |
vendor.name |
X | X | |
_type |
X | X | |
lineItems.activity.localizedNames |
X | X | |
lineItems.project.name |
X | X | |
_seq_no |
X | X | |
lineItems.project |
X | ||
submittedCurrency |
X | ||
createdOnBehalfOf |
X | X | |
lineItems.hasAdjustments |
X | X | |
createdBy |
X | X | |
_field_names |
X | X | |
lineItems.timekeeper |
X | ||
allowGroupAcl |
X | X | |
denyGroupAcl |
X | X | |
_source |
X | ||
_id |
X | X | |
categories.treePosition |
X | X | |
Project |
embeddedObjects.displayName |
X | X |
_index |
X | X | |
_feature |
X | ||
displayName |
X | X | |
mainAssignee |
X | ||
involvedParties |
X | ||
categories.name |
X | X | |
denyUserAcl |
X | X | |
_ignored |
X | X | |
involvedParties.contact |
X | ||
modifiedOn |
X | X | |
mainAssignee.gid |
X | X | |
embeddedObjects.contact.name |
X | X | |
mainAssignee.name |
X | X | |
_data_stream_timestamp |
X | ||
embeddedObjects.contact |
X | ||
involvedParties.contact.gid |
X | X | |
id |
X | X | |
categories |
X | ||
matterId |
X | X | |
_version |
X | ||
_routing |
X | X | |
visibility |
X | X | |
involvedParties.contact.name |
X | X | |
embeddedObjects.gid |
X | X | |
allowUserAcl |
X | X | |
_type |
X | X | |
embeddedObjects.contact.gid |
X | X | |
embeddedObjects.idNumber |
X | X | |
embeddedObjects |
X | ||
involvedParties.gid |
X | X | |
_seq_no |
X | X | |
createdOnBehalfOf |
X | X | |
createdBy |
X | X | |
_field_names |
X | X | |
allowGroupAcl |
X | X | |
name |
X | X | |
denyGroupAcl |
X | X | |
_source |
X | ||
_id |
X | X | |
categories.treePosition |
X | X | |
Tasks |
note |
X | X |
assignee.gid |
X | X | |
activity.name |
X | X | |
project.gid |
X | X | |
_index |
X | X | |
activity |
X | ||
_feature |
X | ||
displayName |
X | X | |
subject |
X | X | |
assignee.name |
X | X | |
project |
X | ||
categories.name |
X | X | |
project.name |
X | X | |
denyUserAcl |
X | X | |
_ignored |
X | X | |
modifiedOn |
X | X | |
emailNotificationActive |
X | X | |
emailAlertNumDaysAdvance |
X | X | |
_data_stream_timestamp |
X | ||
contact |
X | ||
activity.treePosition |
X | X | |
id |
X | X | |
categories |
X | ||
_version |
X | ||
_routing |
X | ||
visibility |
X | X | |
allowUserAcl |
X | X | |
_type |
X | X | |
contact.gid, |
X | X | |
contact.name |
X | X | |
_seq_no |
X | X | |
createdOnBehalfOf |
X | X | |
createdBy |
X | X | |
_field_names |
X | X | |
allowGroupAcl |
X | X | |
denyGroupAcl |
X | X | |
_source |
X | ||
_id |
X | X | |
assignee |
X | ||
categories.treePosition |
X | X | |
activity.localizedNames |
X | X |
Many system fields are indexed but not all are searchable. These fields can be used as filters. For more information, visit Using Filters with Global Search.
Custom Object
Any Custom Object can be indexed. Administrators control that and it is defined in Setup.
Including Custom Objects in Global Search
For each Custom Object added in Setup, under the General tab you will have the option to “Remove from Global Search”. Check this box if you do not want to index any fields in this object.
Indexing Custom Fields
For each custom field created, except Label Only, you will have the option to “Exclude from Global Search”. If you don’t exclude the whole object from Global Search, you can control on a field level what is indexed and searchable. Select NO if you do not want to exclude a field from Global Search.
When new custom objects and custom fields are created, they need to be mapped to the index to become searchable (if you select to include them in Global Search in TeamConnect SetUp). This is controlled in the Global Search Index Tool.
Notes:
- Only custom Boolean fields appear in the search filter dropdown options
- For information on configuring filter displays, see Global Search Filters.
- The first time a custom filter display is added to an object, the filter display might not appear until the object's index is rebuilt.
Object and Field Mapping
Object mapping is the process of translating the TeamConnect object and the fields picked for indexing into a json object that will represent the structure of the object and its fields in Elasticsearch. When objects and fields are mapped to the index, they become searchable.
NOTE: Some field types take up more than 1 space in the index.
Field Types map as follows:
Field Type |
# of Spots in the Index |
Check Box |
2 |
Custom Object |
1 |
Date Field |
1 |
Involved |
1 |
List |
5 |
Memo Text |
2 |
Multi-Value List |
5 |
Number Field |
1 |
Text |
2 |
Label Only |
0 |
Indexing Limit per Object
Elasticsearch sets a 1000 field per object limit to avoid exponential memory usage. The 1000 number includes indexed system fields AND custom fields. As noted by the multipliers in the above table, some fields take up more than 1 space in the index. This means that even if an object has less than 1000 fields, it may still hit the 1000 field limit when indexing and cause the index to fail.
How do I know how many fields I have for each object?
To calculate the space each object will take, query the database for the number of fields by field type for a given object. An example of how to calculate whether your index will hit the limit looks like the following:
Field Type |
Count |
Multiplier |
Spaces in the Index |
Check Box |
22 |
x2 |
44 |
Custom Object |
5 |
x1 |
5 |
Date Field |
15 |
x1 |
15 |
Involved |
77 |
x1 |
77 |
List |
114 |
x5 |
570 |
Memo Text |
33 |
x2 |
66 |
Multi-Value List |
1 |
x5* |
5 |
Number Field |
19 |
x1 |
19 |
Text |
38 |
x2 |
116 |
TOTAL |
344 |
917 |
*Multiplier varies based on the number of values in the multi-value list.
In this scenario, you have 344 fields on an object but they take up 917 spaces in the index. Your object will still index so, in this case, it is safe to proceed.
NEW! However, in TeamConnect 6.3.3, there is a new column in the Global Search Index Tool, called Field Count, that shows how many spaces are taken by the fields on an Object and how many spaces are available. Example:
Also note, the field limit is 5000 instead of 1000. In TeamConnect 6.3.3, administrators can adjust the field limit count between 1000-5000 to accommodate for more custom and system fields on each object. If the index fails because the number of fields exceeds the field count limit, an error will appear in the Global Search Index Tool telling the Admin which field(s) need a higher limit. Visit the Global Search Index Tool page for more details.
Extending a System Object with Custom Fields
When adding custom fields onto a system object, some objects are immediately ready to for searched (for example, Tasks and Contacts). Other objects require a manual re-index (ex: Appointments). For example:
- Creating a task, the task is indexed, and then a custom field is added: all existing tasks are updated with the custom field and can be searched on immediately after the refresh interval, no manual re-index is necessary.
- Adding a custom field to tasks, then creating a new task: all existing tasks are updated with the custom field, and the newly created task is indexed with the new custom field. All tasks and their custom fields can be searched on without re-indexing.
- Creating an appointment, the appointment is indexed, and then a custom field is added: all appointments require a full re-index for the custom field to be searched on. Any new appointments created after the custom field is created cannot be searched on.
- Adding a custom field to appointments, then creating a new appointment: all existing appointments must be re-indexed to search on the custom field, but can be searched on with their pre-existing searchable fields.
One index per object
In TeamConnect 6.0 and earlier versions, all TeamConnect was stored in one index in Elasticsearch. To improve performance, facilitate upgrades, and allow users to search during re-indexing, TeamConnect 6.1 indices now have homogeneous types (object entityTypes) and one index per object, along with a STATUS index.
In TeamConnect 6.0 and earlier versions, the "ElasticSearchUUID" system setting is stored in the database and the UUID value is the name of the index. The new TeamConnect 6.1 naming convention for the indices is in the format UUID-[uniqueCode for object]. For example, index name for Contacts will be <UUID>-cont. For this reason, existing clients must delete their index in the previous version of TeamConnect before upgrading, then rebuild the index after the upgrade. Existing clients with Search Guard must do the following when upgrading:
- Since these configurations added the index name (the UUID from the database) added to their certificate on the node, update the certificate to include a wildcard after the UUID name.
- In the SG_ROLES.YML file, the index names under each role refer to the ElasticsearchUUID from the database. At the end of each index name, add an asterisk at the end of the name, before the closing single quote.
For example, In TeamConnect 6.0 and earlier versions:
In TeamConnect 6.1:
API Indexers
To improve the speed of indexing, starting in TeamConnect 6.1 now uses 2.x objects rather than 3.x objects on all of the indexers for serialization. With 3.x objects, the LegacyBuilder classes loaded parts of the objects that are not needed for the indexer classes, resulting in excessive database access. Additionally, using 2.x objects eliminates the need for the recurring thread to convert the objects from 3.x.