Forward audit events to Splunk
APIsec forwards user-management audit events to your Splunk HTTP Event Collector (HEC) as they happen, so sign-ins and credential changes land in the same SIEM you already monitor.
What is forwarded: user-management audit events. Sign-ins, team and team membership changes, API client and personal access token changes, and account-linking events.
What is not forwarded: scan results, findings, and vulnerabilities. The audit event stream is the only source feeding this integration. To get findings into an external system, use the Jira or Azure DevOps integrations instead.
That distinction is the most common misunderstanding about this integration. Check it before anyone builds dashboards on top of the feed.
Events reach Splunk asynchronously. A slow or unreachable HEC endpoint never blocks or fails the action a user took in APIsec.
Prerequisites
On the Splunk side:
- HTTP Event Collector enabled on your Splunk instance or Splunk Cloud stack.
- An HEC token, and the index that token is permitted to write to.
- The HEC endpoint reachable from the public internet over HTTPS. APIsec calls out to your endpoint, so a listener reachable only inside your private network does not work.
On the APIsec side:
- An account with the ADMIN role. Users with the USER or VIEWER role do not see the integration.
- A cloud APIsec tenant. On-premises deployments do not forward audit events.
Prepare Splunk
-
Choose or create the destination index, for example
apisec_audit. -
Enable the HTTP Event Collector on your Splunk instance or stack.
-
Create an HEC token and grant it write access to that index.
-
Build the full collector URL, including the path:
https://splunk.example.com:8088/services/collector/eventnoteSplunk Cloud does not use this shape. Its collector runs on a separate
http-inputs-hostname on port 443, not on your search head on 8088. Take the URL from the Splunk Cloud HEC token page rather than adapting the example. -
Confirm the token works before involving APIsec:
curl https://splunk.example.com:8088/services/collector/event \
-H "Authorization: Splunk YOUR-HEC-TOKEN" \
-d '{"event":"apisec connectivity check","index":"apisec_audit"}'A successful call returns
{"text":"Success","code":0}.Run this without
-k. Adding it skips the certificate check, which hides exactly the TLS problems that stop APIsec later.
Create the connection in APIsec
Open SIEM Tools in the console and select the Splunk tile.
| Field | Required | Notes |
|---|---|---|
| Name | Yes | A label for this destination. Must be unique among your Splunk connections. |
| HEC URL | Yes | The full collector URL including the path. Must start with https://. |
| HEC Token | Yes | The token value from Splunk. Entered as a password field. |
| Splunk Index | Yes | The destination index. Must be one the token is allowed to write to. |
| Source | No | Overrides the source field. Defaults to apisec-usermgmt. |
| Sourcetype | No | Overrides the sourcetype field. Defaults to apisec:audit:usermgmt. |
| Allow self-signed certificates | No | Toggle, on by default. See the warning that follows. |
| Enabled | No | Toggle, on by default. Turn it off to pause forwarding without deleting the connection. |
Leave Source and Sourcetype blank unless you have a reason to change them. The defaults are stable values your dashboards and alerts can rely on.
Set the self-signed certificate toggle deliberately. Turn it on only if your Splunk instance serves HEC with a self-signed certificate. If your endpoint presents a certificate from a trusted public CA — the normal case on Splunk Cloud — leave it off so the receiving server's identity is verified.
Test, then save
Select Test Connection. APIsec sends one synthetic event of type SPLUNK_TEST_EVENT and reports the outcome, including Splunk's own response text rather than a translated message.
The test is not a dry run. It goes through the same delivery path as a real event and is indexed, so eventType=SPLUNK_TEST_EVENT is a reliable search for confirming the pipeline end to end.
Testing is rate limited to 5 per minute per tenant, and the permits refill steadily rather than as a burst — roughly one every 12 seconds. Two tests in quick succession return an error on the second, so wait about 12 seconds between attempts.
The test saves nothing. Once it passes, select Create Connection.
Confirm the connection is live
The connection list shows the destination with a green ONLINE badge.
Confirm delivery in Splunk itself — search your index for the events, as described next.
Search your index to confirm events arrive. Use the Sourcetype you configured — the default is shown here:
index=apisec_audit sourcetype="apisec:audit:usermgmt" | head 20
Events are JSON. Searching on a field such as eventType works only once that sourcetype is set up for JSON field extraction in Splunk. If raw events appear but field searches return nothing, that extraction is what is missing, not the integration.
Event format
Each audit event is one JSON object, gzip-compressed, posted to the collector with an Authorization: Splunk header.
{
"time": 1787667791.482,
"host": "apisec-usermgmt",
"source": "apisec-usermgmt",
"sourcetype": "apisec:audit:usermgmt",
"index": "apisec_audit",
"event": {
"eventId": "3f6c1a90-6d1e-4a1c-9d0b-2f9a7c4b8e11",
"tenantId": "tenant-abc",
"eventType": "API_CREDENTIAL_PAT_CREATED",
"occurredAt": "2026-08-25T14:23:11.482Z",
"actor": {
"userId": "user-1",
"email": "admin@example.com",
"ip": "203.0.113.10"
},
"target": {
"type": "PAT",
"id": "tok-7781"
},
"attributes": {
"actorRole": "ADMIN",
"name": "ci-pipeline-token"
},
"schemaVersion": 1
}
}
| Envelope field | Value |
|---|---|
time | Epoch seconds with millisecond precision, derived from occurredAt. Falls back to the time of forwarding when the source timestamp is missing or unparseable. |
host | Always the literal apisec-usermgmt. It identifies the producing service and does not change across deployments or releases, so alerts do not churn. |
source | Your Source value, or apisec-usermgmt when blank. |
sourcetype | Your Sourcetype value, or apisec:audit:usermgmt when blank. |
index | Your Splunk Index value. |
event | The audit event itself. |
Inside event:
| Field | Type | Notes |
|---|---|---|
eventId | string | Unique per event. Deduplicate on it, since redelivery is possible. |
tenantId | string | Your APIsec tenant. Filter on it when several tenants share an index. |
eventType | string | What happened. |
occurredAt | string | ISO 8601 UTC timestamp. |
actor | object | Who acted: userId, email, ip. Absent when APIsec cannot attribute the event, and individual members may be absent. |
target | object | What was acted on: type and id. Absent for events with no distinct target. |
attributes | object | Remainder, varying by event type. Treat the key set as open. |
schemaVersion | number | Currently 1. Additive changes do not raise it. |
Write searches against eventType, actor, and target. Treat attributes as supporting detail rather than a stable schema.
Event types
eventType | Meaning | target.type |
|---|---|---|
USER_LOGIN | A user signed in | varies |
ADD_TEAM_MEMBER | A user was added to a team | USER |
CHANGE_TEAM_MEMBER | A team member's permission changed | USER |
DELETE_TEAM_MEMBER | A user was removed from a team | USER |
DELETE_TEAM | A team was deleted | absent |
API_CREDENTIAL_PAT_CREATED | A personal access token was created | PAT |
API_CREDENTIAL_PAT_DELETED | A personal access token was deleted | PAT |
API_CREDENTIAL_CLIENT_CREATED | An OAuth API client was created | API_CLIENT |
API_CREDENTIAL_CLIENT_UPDATED | An OAuth API client was updated | API_CLIENT |
API_CREDENTIAL_CLIENT_DELETED | An OAuth API client was deleted | API_CLIENT |
ACCOUNT_LINKED | A social or SSO identity was linked to an account | varies |
SOCIAL_SIGNUP_ENRICHED | A social signup record was enriched | varies |
SPLUNK_TEST_EVENT | The synthetic event sent by Test Connection | absent |
eventType is a free-form string rather than a fixed enumeration, and this list is the set produced today. Build searches so an unrecognized eventType surfaces rather than disappearing, and avoid dashboards that assume the list is complete.
Team membership events record the acting admin as actor.userId and the affected user as target.id, with teamId in attributes.
Delivery behavior
Read this before you rely on the feed — it determines how you monitor the integration.
One request per event. Each audit event is delivered as its own gzip-compressed HTTPS request, so request volume tracks audit activity directly.
Retries. A delivery gets up to 6 attempts in total, with waits of 1, 2, 4, 8, and 16 seconds between them — about 31 seconds end to end. Retries apply to Splunk 5xx responses, connection timeouts, and refused connections.
If all 6 attempts fail, the event is not discarded: it returns to APIsec's internal queue and is delivered again later. That redelivery is why every event carries eventId — deduplicate on it.
Failures that are never retried. Two classes of failure are permanent, and the event is dropped:
- Any 4xx response from Splunk. That covers a bad token, a disallowed index, and a malformed request — and also 429, which Splunk returns when it throttles you. A throttled event is discarded, not slowed down.
- A TLS handshake failure, which happens only when strict verification is on. With the default self-signed toggle left on, no certificate problem produces one.
These failures are not retried, so the affected events do not arrive. Verify delivery in Splunk after every token rotation, index permission change, and certificate change.
Terminal failures still count toward the circuit breaker, so a destination returning 4xx to everything pauses for 60 seconds after every tenth event rather than failing continuously.
Circuit breaker. After 10 consecutive failures to one destination, APIsec stops sending to it for 60 seconds, then lets a single probe event through. If the probe succeeds, forwarding resumes; if not, the destination pauses for another 60 seconds.
Events that arrive while the breaker is open are queued for redelivery rather than dropped, so a recovered destination typically shows a gap followed by a burst as the backlog drains.
Multiple destinations. A tenant can hold several Splunk connections, and every audit event goes to all of them that are enabled. Circuit breakers are tracked per destination, so one broken endpoint never suppresses a healthy one.
Redelivery, unlike the circuit breaker, is shared. When one destination accepts an event and another hits a retryable failure, the event counts as delivered and is not retried — so the failing destination never receives that event. With more than one destination you get fan-out, not independent guaranteed delivery per destination.
Manage the connection
Rotate the HEC token. Open the connection and edit it. The token field shows PASSWORD MASKED rather than the stored token, which is never displayed again after you save it. Type the new token over the placeholder and save. To change other fields without touching the token, leave the placeholder alone — APIsec keeps the stored token when the field is unchanged.
Test Connection in edit mode tests whatever is in the field. Type a new token and it tests the new one; leave the placeholder untouched and it tests the token already stored. During a rotation, type the new token before testing — otherwise a green result is confirming the credential you are replacing.
Pause forwarding. Turn the Enabled toggle off. The connection and its settings are kept.
Delete the connection. Deleting stops forwarding to that destination permanently. Events generated while a connection is paused or deleted are not replayed afterward.
Troubleshoot
| Symptom | Likely cause | What to do |
|---|---|---|
| Test Connection fails with a 401 or 403 | Bad or revoked HEC token | Re-copy the token from Splunk and test again. |
| Test Connection fails with a 400 | The index is not allowed for this token, or the URL path is wrong | Confirm the token's allowed indexes, and that the URL ends in /services/collector/event. |
| Test Connection fails with a TLS error | Strict verification is on and the certificate is untrusted or the hostname does not match | Fix the certificate. Turning Allow self-signed certificates back on also clears the error, but it disables verification rather than fixing it. |
| Test passes, but no events arrive | Nothing has generated an audit event yet | Sign out and back in to produce a USER_LOGIN, then search again. |
| Test passes, raw events are in Splunk, but field searches return nothing | The sourcetype has no JSON field extraction | Configure JSON extraction for the sourcetype in Splunk. |
| Events are missing from a search but present in the index | Search time range versus time, which comes from occurredAt | Widen the range. Redelivered events can carry a timestamp older than their arrival. |
| Events stopped arriving | A revoked token, a changed index permission, or Splunk throttling | Run Test Connection to confirm the destination still accepts events. |
| Events arrive in bursts after a gap | The circuit breaker opened and recovered | Check your endpoint's availability over the gap. |
Limits
| Area | Current behavior |
|---|---|
| Backfill | None. Events produced while a connection is disabled, deleted, or not yet created are never sent later. Retryable failures on a live connection are different — those are redelivered. |
| Delivery semantics | At-least-once per event, to a single destination, for retryable failures only. Terminal failures are not redelivered, and with several destinations a success at one suppresses redelivery to the others. Deduplicate on eventId. |
| Ordering | Not guaranteed. Redelivery means an event can arrive after a later one. |
Related
- Activity Logs — the same audit events inside the APIsec console.
- Slack Notifications — scan and finding notifications, a separate channel.