December 2024

Tarsal's December update introduces a new Kflow proxy, 5 new Destination Connectors, 4 new Source Connecters, along with other enhancements and bug fixes.

Kflow Proxy

Tarsal has released kflowd-proxy, a service that acts as a proxy for the kflowd eBPF agent. This service enables forwarding of logs and messages from multiple kflowd agents (via UDP) to a unified proxy which supports the following HTTPS destinations:

  • AWS S3 – Log storage and archival.
  • Tarsal kflow Source Connector – Direct ingestion into Tarsal’s pipeline.
  • Custom Webhook Destinations – Forward data to any webhook-compatible endpoint.

The kflowd-proxy can accept input from multiple kflowd agents concurrently, allowing for centralized log handling and more streamlined configuration. This reduces the need for per-agent destination logic, enabling a more efficient and maintainable logging setup.

For setup instructions, configuration details, and usage examples, refer to the kflow-proxy README.

Connectors

New Connectors

Connector Updates

ConnectorTypeChanges
AWS CloudTrailSourceAdded support for IAM role authentication
GitHubSourceAdded support for GitHub App authentication
SlackSourceFixed issue with the “test” functionality
Slack Enterprise GridSourceFixed issue with the “test” functionality
The t_email_address normalization field now pulls
from actor.user.email instead of entity.user.email

Enhancements

Min/Max Sync Time & Max # of Streams

Some vendor APIs have date/time restrictions or lookback limits on how far back they can store or query data. For example, Docusign only allows API retrieval of up to 7 days of audit logs, meaning this connector must run more often that once every 7 days in order to ensure no data is missed when ingesting.

Conversely, some connectors need more time to complete a run. For example, Duo Security has an API rate limit of 1 request per minute. Since the connector can ingest 7 streams, that means the connector needs a minimum of 7 minutes to run, not including any pagination attempts.

When a connector has one of these limitations enforced, you will see a message in the Sync Frequency description

Another limitation can be around the number of streams to ingest. Some APIs take a non trivial amount of time to retrieve data, and this can cause sync issues or other problems. For example, soon the Salesforce connector will support ingesting 60 streams. However, the syncs can timeout if 1 connector is attempting to ingest all 60 at once. In this case, the UI will restrict the number of streams that can be ingested by a single Salesforce connector. If all 60 need to be ingested, then multiple Salesforce connectors should be created to shard out the work.

When a connector has this limitation enforced, you will see a message in the UI like below

Additional Enhancements

  1. A new SAML Configuration option Want AuthnRequests has been added

Bug Fixes

  1. An issue with Snowflake using S3 staging not updating it's status to healthy has been resolved.
  2. Authentication pages now respect the light/dark theme settings, if remembered by the browser.
  3. Fixed an issue where the flow configuration form save button may remain disabled.
  4. Fixed an issue where Tarsal Audit logs were incorrectly recording the API's reverse proxy ip address instead of the client IP address when API keys are used to authenticate.