Elasticsearch

Overview

This Destination Connector uses the Elasticsearch Bulk API that is compatible with Elasticsearch version 7 and 8. The bulk API performs multiple indexing operations in a single API call in order to reduce overhead and greatly increase indexing speed.

Prerequisites

If the Elasticsearch security features are enabled, you must have create, index and write privileges for the target index.
To automatically create an index with a bulk API request, you must have the auto_configure, create_index, or manage index privilege. Note that the Elasticsearch Connector supports both indexing into an existing index as well as a new index.
To make the result of a bulk operation visible to search using the refresh parameter, you must have the maintenance or manage index privilege.
To connect to your Elasticsearch cluster with default authentication and TLS security features enabled, you need to configure the client to use the generated root CA certificate that can be found in the certs directory in your Elasticsearch config location (e.g. $ES_CONF_PATH/certs/http_ca.crt).

Authentication

The following authentication options are supported by this connector:

Authentication Method	Supported	Documentation
None	yes	Endpoint must be `http://`
Username/Password	yes	HTTP basic authentication
API Key	yes	Token-based authentication services

Configuration

Field	Required	Description	Example
Server Endpoint	yes	IP Address or Hostname of Elasticsearch Cluster. Must start with `http://` or `https://`	`https://acmeco.ip.es.io:9200`
Index Name or Pattern	yes	Index Name with optional datetime pattern	`my-index` `my-index-%{+YYYY.MM.dd}` `my-index-%{+YYYY.MM.dd.HH.mm.SS}`
CA Certificate	no	Generated Root CA Certificate. Required when the endpoint is `https`	`-----BEGIN RSA PRIVATE KEY----` `MIIEczCCA1ugAwIBAgIB...` `-----END RSA PRIVATE KEY-----`