Splunk Search

How would I extract fields from raw data containing auto populated numbers in the fields I am trying to extract?

nolejj
Explorer

Hello Community,

How would I extract fields from raw data containing auto populated numbers in the fields I am trying to extract? The below example is field containing raw data. Notice the numbers inside the bracket.  The numbers are not the same for events and will auto change from 1 to 2 digits. For the below example, I would like to extract values for user_id, NAME, and Car. What would be the rex command?

Event 1 for _raw field:

user_id:[4] "peter1234" NAME:[10] "Peter" Car:[3] "Pinto"

Event 2 for _raw field:

user_id:[11] "peter1234" NAME:[5] "Peter" Car:[9] "Gremlin"

Thank You for any assistance.

Joe

Labels (2)
0 Karma
1 Solution

ITWhisperer
SplunkTrust
SplunkTrust

Run them as separate rex commands

| rex "user_id:\[\d+\]\s\"(?<user_id>[^\"]+)"
| rex "NAME:\[\d+\]\s\"(?<NAME>[^\"]+)"
| rex "Car:\[\d+\]\s\"(?<Car>[^\"]+)"

View solution in original post

nolejj
Explorer

Thank you @ITWhisperer . Works great when all the  events have same fields attempting to extract from the raw data. The rex command is limited when extracting and displaying multiple fields not found on events. For example, the below will fill data in the user_id field for all events

 

 

 

| rex "user_id:\[\d+\]\s\"(?<user_id>[^\"]+)

 

 

 

 But when I have an event that displays data not found on another event, the fields would only be extracted for certain event. Below is example:

Event 1 for _raw field (Notice the data for option):

session_id:[3] "33445" user_id:[4] "peter1234" option:[4] "67" NAME:[10] "Peter" Car:[3] "Pinto"

Event 2 for _raw field (option is not listed):

session_id:[3] "33445" user_id:[11] "peter1234" NAME:[5] "Peter" Car:[9] "Gremlin"

Event 3 for _raw field (option and NAME is not listed):

session_id:[3] "33445" user_id:[11] "peter1234" Car:[9] "Gremlin"

When I run the rex example to only extract user_id and NAME, data would only be filled for records similar to Event 1. Data similar to Event 2  or Event 3 would not display data even though user_id exists with the raw data for events. After further testing, I noticed extracted fields will only display on all events if the same extracted field displays in all events within the raw data. 

 

 

 

| rex "user_id:\[\d+\]\s\"(?<user_id>[^\"]+).+NAME:\[\d+\]\s\"(?<NAME>[^\"]+)"

 

 

Is there a way to run the rex command to display extracted data on all events even if certain events have no data? Based on the above example, I would want

event 1 to display the user_id data and NAME data

event 2 to display user_id data and NAME data

event 3 to display  user_id data.

Also, if I am extracting data for example OS$USERID:[11] "peter1234", would the rex command be

| rex "OS.USERID:\[\d+\]\s\"(?<os_userid>[^\"]+)

Joe

0 Karma

ITWhisperer
SplunkTrust
SplunkTrust

Run them as separate rex commands

| rex "user_id:\[\d+\]\s\"(?<user_id>[^\"]+)"
| rex "NAME:\[\d+\]\s\"(?<NAME>[^\"]+)"
| rex "Car:\[\d+\]\s\"(?<Car>[^\"]+)"

ITWhisperer
SplunkTrust
SplunkTrust
| rex "user_id:\[\d+\]\s\"(?<user_id>[^\"]+).+NAME:\[\d+\]\s\"(?<NAME>[^\"]+).+Car:\[\d+\]\s\"(?<Car>[^\"]+)"
Get Updates on the Splunk Community!

Synthetic Monitoring: Not your Grandma’s Polyester! Tech Talk: DevOps Edition

Register today and join TekStream on Tuesday, February 28 at 11am PT/2pm ET for a demonstration of Splunk ...

Instrumenting Java Websocket Messaging

Instrumenting Java Websocket MessagingThis article is a code-based discussion of passing OpenTelemetry trace ...

Announcing General Availability of Splunk Incident Intelligence!

Digital transformation is real! Across industries, companies big and small are going through rapid digital ...