Splunk Search

How would I extract fields from raw data containing auto populated numbers in the fields I am trying to extract?

nolejj
Explorer

Hello Community,

How would I extract fields from raw data containing auto populated numbers in the fields I am trying to extract? The below example is field containing raw data. Notice the numbers inside the bracket.  The numbers are not the same for events and will auto change from 1 to 2 digits. For the below example, I would like to extract values for user_id, NAME, and Car. What would be the rex command?

Event 1 for _raw field:

user_id:[4] "peter1234" NAME:[10] "Peter" Car:[3] "Pinto"

Event 2 for _raw field:

user_id:[11] "peter1234" NAME:[5] "Peter" Car:[9] "Gremlin"

Thank You for any assistance.

Joe

Labels (2)
0 Karma
1 Solution

ITWhisperer
SplunkTrust
SplunkTrust

Run them as separate rex commands

| rex "user_id:\[\d+\]\s\"(?<user_id>[^\"]+)"
| rex "NAME:\[\d+\]\s\"(?<NAME>[^\"]+)"
| rex "Car:\[\d+\]\s\"(?<Car>[^\"]+)"

View solution in original post

nolejj
Explorer

Thank you @ITWhisperer . Works great when all the  events have same fields attempting to extract from the raw data. The rex command is limited when extracting and displaying multiple fields not found on events. For example, the below will fill data in the user_id field for all events

 

 

 

| rex "user_id:\[\d+\]\s\"(?<user_id>[^\"]+)

 

 

 

 But when I have an event that displays data not found on another event, the fields would only be extracted for certain event. Below is example:

Event 1 for _raw field (Notice the data for option):

session_id:[3] "33445" user_id:[4] "peter1234" option:[4] "67" NAME:[10] "Peter" Car:[3] "Pinto"

Event 2 for _raw field (option is not listed):

session_id:[3] "33445" user_id:[11] "peter1234" NAME:[5] "Peter" Car:[9] "Gremlin"

Event 3 for _raw field (option and NAME is not listed):

session_id:[3] "33445" user_id:[11] "peter1234" Car:[9] "Gremlin"

When I run the rex example to only extract user_id and NAME, data would only be filled for records similar to Event 1. Data similar to Event 2  or Event 3 would not display data even though user_id exists with the raw data for events. After further testing, I noticed extracted fields will only display on all events if the same extracted field displays in all events within the raw data. 

 

 

 

| rex "user_id:\[\d+\]\s\"(?<user_id>[^\"]+).+NAME:\[\d+\]\s\"(?<NAME>[^\"]+)"

 

 

Is there a way to run the rex command to display extracted data on all events even if certain events have no data? Based on the above example, I would want

event 1 to display the user_id data and NAME data

event 2 to display user_id data and NAME data

event 3 to display  user_id data.

Also, if I am extracting data for example OS$USERID:[11] "peter1234", would the rex command be

| rex "OS.USERID:\[\d+\]\s\"(?<os_userid>[^\"]+)

Joe

0 Karma

ITWhisperer
SplunkTrust
SplunkTrust

Run them as separate rex commands

| rex "user_id:\[\d+\]\s\"(?<user_id>[^\"]+)"
| rex "NAME:\[\d+\]\s\"(?<NAME>[^\"]+)"
| rex "Car:\[\d+\]\s\"(?<Car>[^\"]+)"

ITWhisperer
SplunkTrust
SplunkTrust
| rex "user_id:\[\d+\]\s\"(?<user_id>[^\"]+).+NAME:\[\d+\]\s\"(?<NAME>[^\"]+).+Car:\[\d+\]\s\"(?<Car>[^\"]+)"
Get Updates on the Splunk Community!

Earn a $35 Gift Card for Answering our Splunk Admins & App Developer Survey

Survey for Splunk Admins and App Developers is open now! | Earn a $35 gift card!      Hello there,  Splunk ...

Continuing Innovation & New Integrations Unlock Full Stack Observability For Your ...

You’ve probably heard the latest about AppDynamics joining the Splunk Observability portfolio, deepening our ...

Monitoring Amazon Elastic Kubernetes Service (EKS)

As we’ve seen, integrating Kubernetes environments with Splunk Observability Cloud is a quick and easy way to ...