December 2024
Tarsal's December update introduces a new Kflow proxy, 5 new Destination Connectors, 4 new Source Connecters, along with other enhancements and bug fixes.
Kflow Proxy
Tarsal has released kflowd-proxy, a service that acts as a proxy for the kflowd eBPF agent. This service enables forwarding of logs and messages from multiple kflowd agents (via UDP) to a unified proxy which supports the following HTTPS destinations:
- AWS S3 – Log storage and archival.
- Tarsal kflow Source Connector – Direct ingestion into Tarsal’s pipeline.
- Custom Webhook Destinations – Forward data to any webhook-compatible endpoint.
The kflowd-proxy can accept input from multiple kflowd agents concurrently, allowing for centralized log handling and more streamlined configuration. This reduces the need for per-agent destination logic, enabling a more efficient and maintainable logging setup.
For setup instructions, configuration details, and usage examples, refer to the kflow-proxy README.
Connectors
New Connectors
Source Connectors | Destination Connectors |
---|---|
Buildkite | Axiom |
Duo Security | Elasticsearch |
Tarsal kflow | Google Cloud Storage |
Sentry | Panther (HTTP Source) |
Panther (S3 Source) |
Connector Updates
Connector | Type | Changes |
---|---|---|
AWS CloudTrail | Source | Added support for IAM role authentication |
GitHub | Source | Added support for GitHub App authentication |
Slack | Source | Fixed issue with the “test” functionality |
Slack Enterprise Grid | Source | Fixed issue with the “test” functionality The t_email_address normalization field now pullsfrom actor.user.email instead of entity.user.email |
Enhancements
Min/Max Sync Time & Max # of Streams
Some vendor APIs have date/time restrictions or lookback limits on how far back they can store or query data. For example, Docusign only allows API retrieval of up to 7 days of audit logs, meaning this connector must run more often that once every 7 days in order to ensure no data is missed when ingesting.
Conversely, some connectors need more time to complete a run. For example, Duo Security has an API rate limit of 1 request per minute. Since the connector can ingest 7 streams, that means the connector needs a minimum of 7 minutes to run, not including any pagination attempts.
When a connector has one of these limitations enforced, you will see a message in the Sync Frequency description
Another limitation can be around the number of streams to ingest. Some APIs take a non trivial amount of time to retrieve data, and this can cause sync issues or other problems. For example, soon the Salesforce connector will support ingesting 60 streams. However, the syncs can timeout if 1 connector is attempting to ingest all 60 at once. In this case, the UI will restrict the number of streams that can be ingested by a single Salesforce connector. If all 60 need to be ingested, then multiple Salesforce connectors should be created to shard out the work.
When a connector has this limitation enforced, you will see a message in the UI like below
Additional Enhancements
- A new SAML Configuration option
Want AuthnRequests
has been added
Bug Fixes
- An issue with Snowflake using S3 staging not updating it's status to healthy has been resolved.
- Authentication pages now respect the light/dark theme settings, if remembered by the browser.
- Fixed an issue where the flow configuration form save button may remain disabled.
- Fixed an issue where Tarsal Audit logs were incorrectly recording the API's reverse proxy ip address instead of the client IP address when API keys are used to authenticate.