Splunk Search
Highlighted

Can you help me with my multi-field field extraction query?

Communicator

I have a field in an event that contains a number of separate individual fields. What would be the most efficient way to extract these so I can search them? Can this be done at index time using transforms? I'm aware of the makemv command, but these are not values of the same field; they are fields within a field.

I'm trying to extract the individual fields within the authsource field. So I would like something like authsourceauth, authsourcelevel, authsourceloginid, etc

Example:

2018-10-18 19:36:47.000, date="10/18/2018 19:36:47", id="166450895", action_at="1539891407534", action="CREATE", permission_id="106965896", customer_id="5495063", role_id="270", active="1", created_at="2018-10-18 19:36:47.0", auth_source="auth:shhkey-v1 | auth-level:internal | login-id:firstlast@company.com-1539295510177 | cus-id:651613 | user-id:65161 | origin-ip:123.123.228.35 | correlation-id:2282777c-25c8-4e1e-a823-4e495fa354ad | zk-group:app-users:api:web:user-traffic:us-east-1", hub_id="61861251", produced_at="1539891407829"
0 Karma
Highlighted

Re: Can you help me with my multi-field field extraction query?

SplunkTrust
SplunkTrust

Do you want these field extractions to be at index time or search time? (Splunk best practice is at search time, but it can be at index time.)

Given that search time is preferred, here is a rex that will do what you want, and the auto-field-extraction will be similar (this is a run-anywhere search that shows the concept by using your data and showing that it works, the only line you need to use is the rex line for the regular expression to use for the field extraction):

| makeresults 
| eval auth_source="auth:shhkey-v1 | auth-level:internal | login-id:firstlast@company.com-1539295510177 | cus-id:651613 | user-id:65161 | origin-ip:123.123.228.35 | correlation-id:2282777c-25c8-4e1e-a823-4e495fa354ad | zk-group:app-users:api:web:user-traffic:us-east-1"
| rex field=auth_source "auth:(?<auth>[^|]*) \| auth-level:(?<auth_level>[^|]*) \| login-id:(?<login_id>[^|]*) \| cus-id:(?<cus_id>[^|]*) \| user-id:(?<user_id>[^|]*) \| origin-ip:(?<origin_ip>[^|]*) \| correlation-id:(?<correlation_id>[^|]*) \| zk-group:(?<zk_group>[^|]*)"

If you need more help on this, just add your comments.

View solution in original post

0 Karma
Highlighted

Re: Can you help me with my multi-field field extraction query?

Communicator

Looks great, thanks! I was thinking index time but search time will work just fine.

0 Karma
Highlighted

Re: Can you help me with my multi-field field extraction query?

Explorer

If doing this at index-time really is requirement, you can always use a report on the part of the event that starts with auth_source=

the report option in your props file associated with your sourcetype the extracts at search time the pairs separated by the = sign

Check this answer for reference 🙂

https://answers.splunk.com/answers/9853/multivalue-field-regex-question.html#answer-9875

0 Karma
Speak Up for Splunk Careers!

We want to better understand the impact Splunk experience and expertise has has on individuals' careers, and help highlight the growing demand for Splunk skills.