Getting Data In

indexRouting with customFilters to route based on log content

Ezward
New Member

I would like to use indexRouting to move some log lines to a given index and have other log lines go to athe HEC's default index.  The log lines that I want to route are single-line json formatted as a HEC event.  Below is a pretty-printed example:

 

{
  "event":{
    "device":{
      "id":"dcef6f000bc7a6baffc0f0b5f000",
    },
    "logMessage":{
      "description":"Publishing to web socket",
      "domain":"WebSocketChannel",
      "severity":"debug"
    },
    "topic":"com.juneoven.dev.analytics"
  },
  "index":"analytics_logs_dev",
  "level":"INFO",
  "source":"dev.analytics",
  "sourcetype":"analytics-logs",
  "time":1630091106.076237
}

 

 

Other log lines are normal text logs (non-json formatted):

 

2021-08-27 19:09:14,295 INFO [tornado.access] 202 POST /1/analytics/log (10.110.4.224) 35.62ms

 

 

I see that there is a customFilter feature.  I am hoping that Ican  key off of the 'index' field in the HEC event to route these json log lines to their index and allow all other lines to go to the default index for the HEC.

Is that possible?  Is there some documentation that would help me?  Thanks.

Labels (3)
0 Karma

PickleRick
SplunkTrust
SplunkTrust

You can use a transform to rewrite the index metadata field of an event.

https://docs.splunk.com/Documentation/Splunk/latest/admin/transformsconf#KEYS:

So in a props.conf you do

[mysourcetype]
TRANSFORMS-redirect=redirect_to_index2

And in transforms.conf (assuming you want to redirect json events:

[redirect_to_index2]
REGEX = {.*}
FORMAT = index2
DEST_KEY = _MetaData:Index

 

EDIT: I'm not sure if {} don't need to be escaped in regex.

0 Karma
Got questions? Get answers!

Join the Splunk Community Slack to learn, troubleshoot, and make connections with fellow Splunk practitioners in real time!

Meet up IRL or virtually!

Join Splunk User Groups to connect and learn in-person by region or remotely by topic or industry.

Get Updates on the Splunk Community!

Why Splunk Customers Should Attend Cisco Live 2026 Las Vegas

Why Splunk Customers Should Attend Cisco Live 2026 Las Vegas     Cisco Live 2026 is almost here, and this ...

What Is the Name of the USB Key Inserted by Bob Smith? (BOTS Hint, Not the Answer)

Hello Splunkers,   So you searched, “what is the name of the usb key inserted by bob smith?”  Not gonna lie… ...

Automating Threat Operations and Threat Hunting with Recorded Future

    Automating Threat Operations and Threat Hunting with Recorded Future June 29, 2026 | Register   Is your ...