Splunk Search

Should we perform a field extraction or field transformation on this sample log?

Path Finder

Hi,

We are looking transform fields from log events, can some one please help.

we need to translate to below codes:
I = Upload
j = Errored
k = Aborted
o = Successful
p =Errored
q = Aborted

In the below example "i" in bold is where we need to translate.

sat dec 13 10:01:17 2017 0 10.01.42.1 42288 /home/cat/wat/g1/T24_94291_20170113_093008_20161213110014510.txt b s **i** r user 0 *

Thanks

0 Karma

SplunkTrust
SplunkTrust

You can create a lookup table for above status abbreviation and status description values table as status_description_csv lookup definition via csv file.

status_abbr,status_description
i,Upload
j,Errored
k,Aborted
o,Successful
p,Errored
q,Aborted

You can then use interactive Field Extractor to Extract new fields from the Splunk Search screen (you can either use Regular Expressions or use space as a delimiter)

For example following is the regular expression for status_abbr field: ^(?:[^.\n]*.){4}\w+\s+\w+\s+\w+\s+(?P\w+).

 <Your Base search> 
| lookup status_description_csv status_abbr output status_description
| table _time status_abbr status_description _raw
| stats count by status_description
____________________________________________
| makeresults | eval message= "Happy Splunking!!!"
0 Karma

Path Finder

Hi , above regular expression will only work if we 4 ".", but for some of the events we have more "."
Below is one of the example, cna you please provide me the regex:

  sat dec 13 10:01:17 2017 0 10.01.42.1 42288 /home/cat/wat/g1/T24_94291_20170113_093008_20161213110014510.txt.gdp b s **i** r user 0 *

  sat dec 13 10:01:17 2017 0 10.01.42.1 42288 /home/cat/wat.1/g1/T24_94291_20170113_093008_20161213110014510.txt. b s **u** r user 0 *

  sat dec 13 10:01:17 2017 0 10.01.42.1 42288 /home/cat/wat/g1.abc.xls/T24_94291_20170113_093008_20161213110014510.cat b s **o** r user 0 *
0 Karma

SplunkTrust
SplunkTrust

Try with this regex

^(\S+\s+){11}(?<status_abbr>\w)

Path Finder

Hi Somesh,

Just found another isssue, this is picking when we have only 11 spaces , but for few other events it is different.

here below event has more than 11 spaces, is there a way we can include if it 11 or more spaces some thing using rex.

Mon Feb 14 09:35:26 2017 236 100.81.24.1 8189 /cat2/main/y2/jila/itfs/589 y2 jila FS.zip b s o r jila ssh 0 *
0 Karma

SplunkTrust
SplunkTrust

Since the nearby words are also single aphabets, it makes it difficult. Also, for other events, the status was 12th from start and 5th from last, but in your new samples it doesn't follow that as well. Assuming the status field can have values from range (I , j, k, o, p, q) and nearby characters don't include any of status values, give this a try

^(\S+\s+){11,12}(?<status_abbr>(I|j|k|o|p|q))\s
0 Karma

SplunkTrust
SplunkTrust

Is the "i" (or any other possible value) already being extracted as field? Do you want to modify raw data (non recommended due to overhead on indexing) or just need a field with translated status.

0 Karma

Path Finder

NO , we didn't extracted anything,

Yes we would prefer translated status on field would be great.

Thanks

0 Karma

Path Finder

I would recommend doing a field extraction using regex to capture the i,j,k,o,p and q into a field named code.

Then in your search use the case command to transform the letter code to the word.

| eval Status_Code=case(code==i,"Upload",code==j,"Errored",code==k,"Aborted" etc..... )
0 Karma