I have a field in an event that contains a number of separate individual fields. What would be the most efficient way to extract these so I can search them? Can this be done at index time using transforms? I'm aware of the
makemv command, but these are not values of the same field; they are fields within a field.
I'm trying to extract the individual fields within the authsource field. So I would like something like authsourceauth, authsourcelevel, authsourceloginid, etc
2018-10-18 19:36:47.000, date="10/18/2018 19:36:47", id="166450895", action_at="1539891407534", action="CREATE", permission_id="106965896", customer_id="5495063", role_id="270", active="1", created_at="2018-10-18 19:36:47.0", auth_source="auth:shhkey-v1 | auth-level:internal | login-id:email@example.com | cus-id:651613 | user-id:65161 | origin-ip:22.214.171.124 | correlation-id:2282777c-25c8-4e1e-a823-4e495fa354ad | zk-group:app-users:api:web:user-traffic:us-east-1", hub_id="61861251", produced_at="1539891407829"
Do you want these field extractions to be at index time or search time? (Splunk best practice is at search time, but it can be at index time.)
Given that search time is preferred, here is a
rex that will do what you want, and the auto-field-extraction will be similar (this is a run-anywhere search that shows the concept by using your data and showing that it works, the only line you need to use is the
rex line for the regular expression to use for the field extraction):
| makeresults | eval auth_source="auth:shhkey-v1 | auth-level:internal | login-id:firstname.lastname@example.org | cus-id:651613 | user-id:65161 | origin-ip:126.96.36.199 | correlation-id:2282777c-25c8-4e1e-a823-4e495fa354ad | zk-group:app-users:api:web:user-traffic:us-east-1" | rex field=auth_source "auth:(?<auth>[^|]*) \| auth-level:(?<auth_level>[^|]*) \| login-id:(?<login_id>[^|]*) \| cus-id:(?<cus_id>[^|]*) \| user-id:(?<user_id>[^|]*) \| origin-ip:(?<origin_ip>[^|]*) \| correlation-id:(?<correlation_id>[^|]*) \| zk-group:(?<zk_group>[^|]*)"
If you need more help on this, just add your comments.
If doing this at index-time really is requirement, you can always use a report on the part of the event that starts with auth_source=
the report option in your props file associated with your sourcetype the extracts at search time the pairs separated by the = sign
Check this answer for reference 🙂