Splunk Search

Extract fields out of plain text log file and assign it to a field name

Sujithkumarkb
Observer

I want to extract the below values during index time
1. extract WDDZF4KB3JA469368 ,ABCDE4KB3JA469368 and so on and assign it to a field name called VINnumber.
2. extract ConfigureServiceStatusMBIISProvABCSImpl, QuerySubscriptionListSiebelProvABCSImpl ,VINValidationProcess and so on and assign it to field name called services.

ist item2019-12-17T19:35:20.571-05:00|ConfigureServiceStatusMBIISProvABCSImpl|MBIIS|2837822832|0396771d-0879-4246-b9e9-e31328260eb3-04d362c7|SUCCESS|VIN-WDDZF4KB3JA469368|

2019-12-17T19:35:20.571-05:00|ConfigureServiceStatusMBIISProvABCSImpl|MBIIS|2837822832|0396771d-0879-4246-b9e9-e31328260eb3-04d362c7|SUCCESS|VIN-ABCDE4KB3JA469368|

2019-12-17T19:35:18.076-05:00|QuerySubscriptionListSiebelProvABCSImpl|SIEBEL|2837822838|0396771d-0879-4246-b9e9-e31328260eb3-04d362c7|FAILED|VIN-WDDZF4KB3JA469368|AccId-NoValueFound|Business Fault:Agree# Missing

2019-12-17T19:35:18.587-05:00|VINValidationProcess|SIEBEL|2837822846|0396771d-0879-4246-b9e9-e31328260eb3-04d362c7|SUCCESS|VIN-WDDZF4KB3JA469368|

2019-12-17T19:35:18.076-05:00|QuerySubscriptionListSiebelProvABCSImpl|SIEBEL|2837822838|0396771d-0879-4246-b9e9-e31328260eb3-04d362c7|FAILED|VIN-WDDZF4KB3JA469368*|AccId-NoValueFound|Business Fault:Agree# Missing*

I understand search time extraction is better for performance .
I tried to extract during search time like below with a bit of luck , but data is

index=ABC sourcetype=XYZ  WDDZF4KB3JA469368 
| rex field=_raw "(?WDDZF4KB3JA469368.+)\"" 

but the value i am getting in the field is quite extra VIN= WDDZF4KB3JA469368,,,Cathy,"Wright",,"6278 Harbour Heights Pkwy

0 Karma
1 Solution

to4kawa
Ultra Champion
| makeresults 
| eval raw="2019-12-17T19:35:20.571-05:00|ConfigureServiceStatusMBIISProvABCSImpl|MBIIS|2837822832|0396771d-0879-4246-b9e9-e31328260eb3-04d362c7|SUCCESS|VIN-WDDZF4KB3JA469368|@2019-12-17T19:35:20.571-05:00|ConfigureServiceStatusMBIISProvABCSImpl|MBIIS|2837822832|0396771d-0879-4246-b9e9-e31328260eb3-04d362c7|SUCCESS|VIN-ABCDE4KB3JA469368|@2019-12-17T19:35:18.076-05:00|QuerySubscriptionListSiebelProvABCSImpl|SIEBEL|2837822838|0396771d-0879-4246-b9e9-e31328260eb3-04d362c7|FAILED|VIN-WDDZF4KB3JA469368|AccId-NoValueFound|Business Fault:Agree# Missing@2019-12-17T19:35:18.587-05:00|VINValidationProcess|SIEBEL|2837822846|0396771d-0879-4246-b9e9-e31328260eb3-04d362c7|SUCCESS|VIN-WDDZF4KB3JA469368|@2019-12-17T19:35:18.076-05:00|QuerySubscriptionListSiebelProvABCSImpl|SIEBEL|2837822838|0396771d-0879-4246-b9e9-e31328260eb3-04d362c7|FAILED|VIN-WDDZF4KB3JA469368|AccId-NoValueFound|Business Fault:Agree# Missing"
| makemv delim="@" raw
| mvexpand raw
| rex field=raw "(?<_time>[^\|]+)\|(?<services>[^\|]+)\|[^\|]+\|[^\|]+\|[^\|]+\|(?<result>[^\|]+)\|(?<VINnumber>[^\|]+)\|"
| table _time raw services VINnumber result

View solution in original post

0 Karma

to4kawa
Ultra Champion
| makeresults 
| eval raw="2019-12-17T19:35:20.571-05:00|ConfigureServiceStatusMBIISProvABCSImpl|MBIIS|2837822832|0396771d-0879-4246-b9e9-e31328260eb3-04d362c7|SUCCESS|VIN-WDDZF4KB3JA469368|@2019-12-17T19:35:20.571-05:00|ConfigureServiceStatusMBIISProvABCSImpl|MBIIS|2837822832|0396771d-0879-4246-b9e9-e31328260eb3-04d362c7|SUCCESS|VIN-ABCDE4KB3JA469368|@2019-12-17T19:35:18.076-05:00|QuerySubscriptionListSiebelProvABCSImpl|SIEBEL|2837822838|0396771d-0879-4246-b9e9-e31328260eb3-04d362c7|FAILED|VIN-WDDZF4KB3JA469368|AccId-NoValueFound|Business Fault:Agree# Missing@2019-12-17T19:35:18.587-05:00|VINValidationProcess|SIEBEL|2837822846|0396771d-0879-4246-b9e9-e31328260eb3-04d362c7|SUCCESS|VIN-WDDZF4KB3JA469368|@2019-12-17T19:35:18.076-05:00|QuerySubscriptionListSiebelProvABCSImpl|SIEBEL|2837822838|0396771d-0879-4246-b9e9-e31328260eb3-04d362c7|FAILED|VIN-WDDZF4KB3JA469368|AccId-NoValueFound|Business Fault:Agree# Missing"
| makemv delim="@" raw
| mvexpand raw
| rex field=raw "(?<_time>[^\|]+)\|(?<services>[^\|]+)\|[^\|]+\|[^\|]+\|[^\|]+\|(?<result>[^\|]+)\|(?<VINnumber>[^\|]+)\|"
| table _time raw services VINnumber result
0 Karma

Sujithkumarkb
Observer

@to4kawa , thanks .
But , What if i want to extract fields during index time ?

0 Karma

to4kawa
Ultra Champion
your search
|eval index_time=strftime(_indextime,"%F %T")
|table _time index_time services VINnumber result
0 Karma

Sujithkumarkb
Observer

@to4kawa , sorry may be i was not clear.
I meant using Props , how can i extract these fields using regex , because i do not see a pattern for the fields i want to extract.

0 Karma

to4kawa
Ultra Champion

I'm sorry , I don't know props.conf
If you provide new Question of props.conf with this sample data, another professional will solve it.

0 Karma
Get Updates on the Splunk Community!

Continuing Innovation & New Integrations Unlock Full Stack Observability For Your ...

You’ve probably heard the latest about AppDynamics joining the Splunk Observability portfolio, deepening our ...

Monitoring Amazon Elastic Kubernetes Service (EKS)

As we’ve seen, integrating Kubernetes environments with Splunk Observability Cloud is a quick and easy way to ...

Cloud Platform & Enterprise: Classic Dashboard Export Feature Deprecation

As of Splunk Cloud Platform 9.3.2408 and Splunk Enterprise 9.4, classic dashboard export features are now ...