> ## Documentation Index
> Fetch the complete documentation index at: https://docs.rad.security/llms.txt
> Use this file to discover all available pages before exploring further.

# Splunk Enterprise

> Configure Splunk Enterprise integration with RAD Security to query security data.

# Splunk Enterprise Integration Setup

This guide walks you through integrating Splunk Enterprise with RAD Security to enable querying security data from Splunk for enrichment and correlation within RAD Security.

Splunk Enterprise provides powerful search and analysis capabilities for security data. RAD Security integrates with Splunk to pull relevant security events and logs for correlation with runtime security detections.

## Prerequisites

Before you begin, ensure you have:

<Check>
  * Admin access to Splunk Enterprise
  * Splunk Enterprise instance (version 8.0 or later recommended)
  * Ability to create authentication tokens in Splunk
  * Access to RAD Security workspace with integration permissions
</Check>

<Warning>
  **Splunk Enterprise Only:** RAD Security's Splunk integration uses authentication tokens for querying data, which is only available in Splunk Enterprise. This feature is **not available** in Splunk Cloud free trials.
</Warning>

<Info>
  **Service User Recommended:** Create a dedicated service user for this integration rather than using a personal account. This ensures the integration continues working when team members leave or change roles.
</Info>

***

## Step 1: Enable Token Authentication

Configure authentication tokens to allow RAD Security to query data from Splunk.

<Steps>
  <Step title="Log in to Splunk Enterprise">
    Log in to Splunk Enterprise with administrator privileges
  </Step>

  <Step title="Enable Token Authentication">
    If token authentication is not already enabled:

    1. Navigate to **Settings > Authentication methods**
    2. Enable **Token authentication**
    3. Click **Save**

    <Note>
      Token authentication is typically enabled by default in Splunk Enterprise 7.3 and later.
    </Note>
  </Step>
</Steps>

***

## Step 2: Create Service User (Recommended)

Create a dedicated service user for the RAD Security integration to ensure continuity and proper access control.

<Steps>
  <Step title="Navigate to User Management">
    Go to **Settings > Users and authentication > Access controls > Users**
  </Step>

  <Step title="Create New User">
    1. Click **New User**
    2. Enter user details:
       * **Username** (e.g., "rad-security-service")
       * **Full name** (e.g., "RAD Security Integration")
       * **Email address** - Use a service email (e.g., `rad-security@company.com`)
       * **Password** - Create a secure password
  </Step>

  <Step title="Assign Roles">
    Assign the appropriate role(s) to the service user. At minimum, the user needs:

    * **user** role - Basic search capabilities
    * Custom role with read access to relevant indexes (optional, for more granular control)

    <Info>
      You can create a custom role with only the specific permissions needed for RAD Security queries if you prefer least-privilege access.
    </Info>
  </Step>

  <Step title="Save User">
    Click **Save** to create the service user
  </Step>
</Steps>

<Info>
  **Alternative:** You can use an existing service account if one is already configured with appropriate search permissions.
</Info>

***

## Step 3: Create Authentication Token

<Steps>
  <Step title="Navigate to Token Management">
    Go to **Settings > Tokens**
  </Step>

  <Step title="Create New Token">
    1. Click **New Token**
    2. Enter a **Token name** (e.g., "RAD Security Query Token")
    3. Select the **User** - Choose the service user created in Step 2
    4. Set **Audience** - Leave as default (system) or set to specific audience if required
    5. Set **Expiration time**:
       * Recommended: 90 days or less for security best practices
       * Or set to match your organization's token rotation policy

    <Note>
      Setting an expiration date requires you to rotate the token periodically, but it's a security best practice.
    </Note>
  </Step>

  <Step title="Generate Token">
    1. Click **Create**
    2. **Immediately copy the authentication token** that appears

    <Warning>
      **This is your only chance to view the token!** If you lose it, you'll need to revoke and create a new token. Store it securely in a password manager or secrets vault.
    </Warning>
  </Step>

  <Step title="Note API Endpoint URL">
    Your Splunk API endpoint URL will be in the format:

    `https://<splunk-host>:8089`

    Replace `<splunk-host>` with your Splunk server hostname or IP address.

    <Info>
      Port 8089 is the default Splunk management port. If your Splunk instance uses a different port, use that instead.
    </Info>
  </Step>
</Steps>

For detailed instructions, see [Splunk's token authentication documentation](https://docs.splunk.com/Documentation/Splunk/latest/Security/CreateAuthTokens).

***

## Step 4: Configure in RAD Security

Navigate to your RAD Security workspace and configure the Splunk Enterprise integration with the following parameters:

### Required Parameters

| Parameter                | Description                      | Example                           |
| ------------------------ | -------------------------------- | --------------------------------- |
| **Search Service URL**   | Splunk management API endpoint   | `https://splunk.company.com:8089` |
| **Search Service Token** | Authentication token from Step 3 | `your-auth-token-here`            |

<Note>
  The API URL should be your Splunk management port (default: 8089). This is **different** from the web interface port (8000) or HEC port (8088).
</Note>

<Info>
  **Port Configuration:** If your Splunk instance uses a non-standard management port, update the port number in the API URL accordingly.
</Info>

***

## Verify Integration

After completing the setup, verify your integration is working:

<Steps>
  <Step title="Check Connection Status">
    1. Navigate to **Data Sources > Integrations > SIEM** in RAD Security
    2. Locate your Splunk Enterprise integration
    3. Verify the connection status shows as **Connected**
  </Step>
</Steps>

<Check>
  Your Splunk Enterprise integration is now configured! RAD Security can query security data from Splunk for enrichment and correlation.
</Check>

## What Data Can Be Queried

RAD Security can query the following data types from Splunk:

<AccordionGroup>
  <Accordion title="Security Events" icon="shield-halved">
    * Firewall logs
    * IDS/IPS alerts
    * Web application firewall (WAF) events
    * Network traffic logs
    * VPN connection logs
  </Accordion>

  <Accordion title="Authentication & Access" icon="key">
    * Login attempts (successful and failed)
    * Authentication events from Active Directory, LDAP, etc.
    * Privilege escalation events
    * Account creation/modification/deletion
    * Session management events
  </Accordion>

  <Accordion title="Endpoint Data" icon="desktop">
    * EDR/antivirus alerts
    * Process execution logs
    * File system changes
    * Registry modifications (Windows)
    * System performance metrics
  </Accordion>

  <Accordion title="Cloud & Infrastructure" icon="cloud">
    * Cloud service logs (AWS CloudTrail, Azure Activity, GCP Audit)
    * Container runtime events
    * Kubernetes audit logs
    * Infrastructure changes
    * API access logs
  </Accordion>

  <Accordion title="Application Logs" icon="code">
    * Application errors and exceptions
    * API request/response logs
    * Database query logs
    * Custom application events
    * Performance metrics
  </Accordion>

  <Accordion title="Threat Intelligence" icon="radar">
    * Indicator of Compromise (IoC) matches
    * Threat feeds integrated in Splunk
    * Malware detections
    * Command and control (C2) communications
    * Suspicious domain lookups
  </Accordion>
</AccordionGroup>

## Use Cases

<CardGroup cols={2}>
  <Card title="Security Event Enrichment" icon="sparkles">
    Enrich RAD Security runtime detections with historical context from Splunk logs.
  </Card>

  <Card title="Cross-Platform Correlation" icon="diagram-venn">
    Correlate RAD Security runtime events with network, endpoint, and application logs stored in Splunk.
  </Card>

  <Card title="Threat Investigation" icon="magnifying-glass">
    Query Splunk for additional context when investigating security incidents detected by RAD Security.
  </Card>

  <Card title="Historical Analysis" icon="clock-rotate-left">
    Access historical security data from Splunk to establish baselines and identify anomalies.
  </Card>

  <Card title="IoC Validation" icon="shield-check">
    Validate indicators of compromise by querying Splunk's threat intelligence and historical data.
  </Card>

  <Card title="Compliance Evidence" icon="file-check">
    Pull compliance-relevant logs from Splunk to support RAD Security's audit and compliance workflows.
  </Card>
</CardGroup>

## Troubleshooting

<AccordionGroup>
  <Accordion title="Authentication Failed" icon="triangle-exclamation">
    **Possible causes:**

    * Token authentication not enabled in Splunk
    * Authentication token is incorrect or expired
    * Token was revoked
    * Service user account was disabled

    **Solution:**

    * Verify token authentication is enabled: **Settings > Authentication methods**
    * Check token expiration date in **Settings > Tokens**
    * Ensure token is copied correctly (no extra spaces)
    * Verify the service user account is still active
    * Generate a new token if the old one was revoked or expired
    * Test authentication with curl:
      ```bash theme={null}
      curl -k https://<host>:8089/services/auth/login \
        -H "Authorization: Bearer <token>" \
        -d "output_mode=json"
      ```
  </Accordion>

  <Accordion title="Connection Failed" icon="plug-circle-xmark">
    **Possible causes:**

    * Incorrect API endpoint URL or port
    * Firewall blocking port 8089
    * Splunk management port not accessible
    * SSL certificate issues

    **Solution:**

    * Verify API URL format: `https://<host>:8089`
    * Ensure firewall rules allow port 8089 from RAD Security
    * Check Splunk is listening on port 8089: `netstat -an | grep 8089`
    * Test connectivity: `telnet <host> 8089`
    * Verify SSL certificate is valid
    * Whitelist RAD Security IP addresses if needed
  </Accordion>

  <Accordion title="Insufficient Permissions" icon="shield-exclamation">
    **Possible causes:**

    * Service user lacks search role
    * No read access to required indexes
    * Custom role missing necessary capabilities
    * Index-level permissions not configured

    **Solution:**

    * Verify service user has at minimum the **user** role
    * Check index permissions: **Settings > Access controls > Roles**
    * Ensure user can search the indexes you need: `| eventcount summarize=false index=*`
    * Test search directly in Splunk UI as the service user
    * Grant additional capabilities if needed (list\_inputs, search, etc.)
  </Accordion>

  <Accordion title="Query Errors" icon="magnifying-glass-minus">
    **Possible causes:**

    * Using Splunk Cloud free trial (query API not available)
    * Invalid SPL (Search Processing Language) query
    * Query timeout
    * Search quota exceeded

    **Solution:**

    * Verify you're using **Splunk Enterprise** (query API not available in Cloud free trials)
    * Test queries directly in Splunk UI before using in RAD Security
    * Simplify complex queries or add time range restrictions
    * Check search job limits: **Settings > Server settings > Search**
    * Review Splunk search logs: `index=_internal source=*splunkd.log* search`
  </Accordion>

  <Accordion title="No Results Returned" icon="inbox">
    **Possible causes:**

    * User doesn't have access to the queried indexes
    * Time range doesn't include relevant data
    * Query syntax is incorrect
    * Index doesn't exist or is empty

    **Solution:**

    * Verify the service user can access the target indexes
    * Check if data exists in the time range: Use Splunk UI to verify
    * Test with a simple query: `index=_internal | head 10`
    * Ensure the index name is spelled correctly
    * Check for typos in field names or values
  </Accordion>

  <Accordion title="Token Expired" icon="clock">
    **Possible causes:**

    * Token reached configured expiration time
    * Token was manually revoked
    * Service user password changed (doesn't affect tokens but might be related)

    **Solution:**

    * Check token status in **Settings > Tokens**
    * Create a new token following Step 3
    * Update the token in RAD Security integration settings
    * Consider setting longer expiration or implementing token rotation workflow
    * Document token expiration dates for proactive rotation
  </Accordion>

  <Accordion title="SSL/TLS Certificate Errors" icon="certificate">
    **Possible causes:**

    * Self-signed certificate not trusted
    * Certificate validation failed
    * Certificate expired or hostname mismatch

    **Solution:**

    * For testing: You may need to configure RAD Security to accept self-signed certificates
    * For production: Use valid SSL certificates from a trusted CA
    * Verify certificate expiration: `openssl s_client -connect <host>:8089`
    * Check hostname matches certificate CN or SAN
    * Import self-signed certificate to trusted store if needed
  </Accordion>

  <Accordion title="Rate Limiting or Quota Exceeded" icon="gauge-high">
    **Possible causes:**

    * Too many concurrent searches
    * Search quota limits reached
    * API rate limiting enabled

    **Solution:**

    * Check search concurrency limits: **Settings > Server settings > Search**
    * Review search quota usage for the service user
    * Reduce query frequency in RAD Security if possible
    * Increase search quotas for the service user if necessary
    * Schedule heavy queries during off-peak hours
  </Accordion>
</AccordionGroup>

## Security Best Practices

<CardGroup cols={2}>
  <Card title="Use Service Accounts" icon="user-gear">
    Always use a dedicated service account with a service email, never a personal account tied to an individual.
  </Card>

  <Card title="Rotate Tokens Regularly" icon="rotate">
    Set token expiration periods and rotate before expiry. Recommended: 90 days or less.
  </Card>

  <Card title="Least Privilege Access" icon="shield-halved">
    Grant only the minimum search permissions required. Use custom roles for granular access control.
  </Card>

  <Card title="Enable SSL/TLS" icon="lock">
    Always use HTTPS for the API endpoint. Use valid SSL certificates in production environments.
  </Card>

  <Card title="Monitor Token Usage" icon="chart-line">
    Regularly review authentication token usage in Splunk to detect anomalous query activity.
  </Card>

  <Card title="Restrict Network Access" icon="network-wired">
    Configure firewall rules to only allow port 8089 connections from RAD Security IP addresses.
  </Card>

  <Card title="Secure Token Storage" icon="vault">
    Store tokens in a secure password manager or secrets vault. Never commit tokens to version control.
  </Card>

  <Card title="Audit Search Activity" icon="clipboard-list">
    Regularly review search activity by the service user to ensure queries align with expected behavior.
  </Card>

  <Card title="Index-Level Permissions" icon="database">
    Use index-level permissions to limit which data the service user can query.
  </Card>

  <Card title="Document Token Expiration" icon="calendar">
    Maintain documentation of token expiration dates to enable proactive rotation.
  </Card>
</CardGroup>

## Token Rotation

To rotate your Splunk authentication token:

<Steps>
  <Step title="Create New Token">
    1. Log in to Splunk Enterprise as admin
    2. Navigate to **Settings > Tokens**
    3. Create a new authentication token for the same service user
    4. Use a different label to distinguish it from the old token
  </Step>

  <Step title="Update RAD Security Integration">
    Update the authentication token in RAD Security integration settings
  </Step>

  <Step title="Verify Connectivity">
    Test that queries still work with the new token
  </Step>

  <Step title="Revoke Old Token">
    Once verified, revoke the old token in **Settings > Tokens**
  </Step>
</Steps>

<Info>
  **Token Rotation Schedule:** Set calendar reminders 1-2 weeks before token expiration to ensure uninterrupted service.
</Info>

***

## Additional Resources

<CardGroup cols={2}>
  <Card title="Authentication Tokens" icon="key" href="https://docs.splunk.com/Documentation/Splunk/latest/Security/CreateAuthTokens">
    Official Splunk guide to creating and managing authentication tokens
  </Card>

  <Card title="Splunk REST API" icon="code" href="https://docs.splunk.com/Documentation/Splunk/latest/RESTREF/RESTprolog">
    Complete reference for Splunk's REST API
  </Card>

  <Card title="Search Reference" icon="magnifying-glass" href="https://docs.splunk.com/Documentation/Splunk/latest/SearchReference/WhatsInThisManual">
    Splunk Search Processing Language (SPL) reference
  </Card>

  <Card title="User Access Control" icon="users-gear" href="https://docs.splunk.com/Documentation/Splunk/latest/Security/Aboutusersandroles">
    Learn about Splunk users, roles, and permissions
  </Card>
</CardGroup>

## Next Steps

<CardGroup cols={2}>
  <Card title="SIEM Integrations Overview" icon="chart-line" href="/rad-security/integrations/siem/overview">
    Explore other SIEM integration options
  </Card>

  <Card title="Evidence Room" icon="folder-open" href="/rad-security/platform/evidence-room">
    See how Splunk data enriches security findings
  </Card>

  <Card title="Runtime Security" icon="shield" href="/rad-security/integrations/runtime-security">
    Learn about RAD Security's runtime detection capabilities
  </Card>

  <Card title="Data Sources" icon="database" href="/rad-security/integrations/data-sources">
    Connect additional security data sources
  </Card>
</CardGroup>