topic Re: REGEX to extract null/empty field as it has values in Splunk Search

REGEX to extract null/empty field as it has values

larmesto — Fri, 18 Mar 2016 19:35:52 GMT

Hello folks,

I was wondering if you could help me with an issue regarding to the field extraction technique.
I have this multiline log (below). My problem is that when I trying to extract the empty fields at the same time as the valued ones.

This is my LOG:

14/03/16 06:02:03 VSED CRPDRETR RETRANS B F XCOMJOB OK+A P $IJDPRT 02014
14/03/16 06:02:03 VSED CRPDRETR RETRANS B F XCOMJOB OK+A P ISTMMCHS 02014
14/03/16 06:02:03 VSED CRPDRETR RETRANS B F XCOMJOB WRITE ALL OK+A D OPE294 VCIP3.SORTWK1.F4 02014
14/03/16 06:02:03 VSED CRPDRETR RETRANS B F XCOMJOB READ OK+A ? PRD2.XCOM310.C230FI02.CONFI 02014
14/03/16 06:02:03 VSED CRPDRETR RETRANS B F XCOMJOB READ OK+A ? PRD2.XCOM310.C230FI02.CONFI 02014
14/03/16 06:02:03 VSED CRPDRETR RETRANS B F XCOMJOB READ OK+A ? PRD2.XCOM310.DESTMODE.CONFI 02014
14/03/16 06:02:03 VSED CRPDRETR RETRANS B F XCOMJOB READ OK+A ? PRD2.XCOM310.DESTMODE.CONFI 02014
14/03/16 06:02:03 VSED CRPDRETR RETRANS B F XCOMJOB OK+A P IJDANCHX 02014
14/03/16 06:02:03 VSED CRPDRETR RETRANS B F $JOBACCT OK+A P IESWAIT 02014
14/03/16 06:04:03 VSED CRPDRETR RETRANS B F $JOBACCT OK+A TRM 02014
14/03/16 08:17:04 VSED CRPDRETR DUMMY B F $JOBACCT READ OK+A q SUBMIT.VCIP3.RETRANS.CRPDRE 43327
14/03/16 08:17:04 VSED CRPDRETR RETRANS B F $JOBACCT OK+A T VMRDR00C 43327 VMRDR00C
14/03/16 08:17:04 VSED CRPDRETR RETRANS B F $JOBACCT READ OK+A e VMRDR00C 43327 VMRDR00C
14/03/16 08:17:04 VSED CRPDRETR RETRANS B F $JOBACCT OK+A VFX 43327 VMRDR00C
14/03/16 08:17:04 VSED CRPDRETR RETRANS B F $JOBACCT OK+A , VSED 43327
14/03/16 08:17:04 VSED CRPDRETR RETRANS B F $JOBACCT OK+A U CPU.VSED 43327
14/03/16 08:17:04 VSED CRPDRETR RETRANS B F $JOBACCT OK+A ? F3 43327
14/03/16 08:17:04 VSED CRPDRETR RETRANS B F $JOBACCT OK+A U VSEPART. 43327
14/03/16 08:17:04 VSED CRPDRETR RETRANS B F $JOBACCT READ OK+A e VMRDR00C 43327
14/03/16 08:17:04 VSED CRPDRETR RETRANS B F $JOBACCT OK+A INI NAME=TRANSM/USO COMUN 43327
14/03/16 08:17:04 VSED CRPDRETR RETRANS B F $JOBACCT READ OK+A ? PRD2.CONFIG.SETXCOM.PROC 43327

What I'm trying to do is to extract, for example, in the first row Column1=WRITE and Column2=ALL and so on.... With empty values or not.

Here's my current REGEX:
(\d+\/\d+\/\d+\s\d+:\d+:\d+)\s+([^\s]+)\s+([^\s]+)\s+([^\s]+)\s+([^\s]+\s+[^\s]+)\s+([^\s]+)([^\s]+)

As you can see in the image attached when is use the final \s+ the matches for 'spaces' breaks my entire formula. (See image)
I'm using regex101 site to test this.

Any help is welcome!
Best regards,

Re: REGEX to extract null/empty field as it has values

acharlieh — Fri, 18 Mar 2016 20:22:35 GMT

In your screenshot, that file looks to be tab delimited? would matching based on individual tab characters instead of any number of whitespace characters help?

Re: REGEX to extract null/empty field as it has values

larmesto — Fri, 18 Mar 2016 20:49:11 GMT

Thanks acharlieh
I've tryed this, but is not tab delimited. I believe the only option is to parse with static positions...

Re: REGEX to extract null/empty field as it has values

alemarzu — Fri, 18 Mar 2016 20:54:29 GMT

Are those fixed length fields ?

Take a look at this,

Edit: regex added

main search | rex field=_raw "\d{2}\/\d{2}\/\d{2}\s\d{2}:\d{2}:\d{2}\s(?<A_TEMP>.*?)\sOK" | rex field=A_TEMP "(?<a_group1>.{4})\s(?<a_group2>.{8})\s(?<a_group3>.{7})\s+(?<a_group4>.{3})\s+(?<a_group5>.*?)\s+(?<a_group6>.*)" | table _time a_group1 a_group2 a_group3 a_group4 a_group5 a_group6

Re: REGEX to extract null/empty field as it has values

somesoni2 — Fri, 18 Mar 2016 21:11:12 GMT

Give this a try

(\d+\/\d+\/\d+\s\d+:\d+:\d+)\s+([^\s]+)\s+([^\s]+)\s+([^\s]+)\s+([^\s]+\s+[^\s]+)\s+([^\s]+)\s+(.*)\s*OK\+A\s+([^\s]+)

Updated
Try this

(\d+\/\d+\/\d+\s\d+:\d+:\d+)\s+([^\s]+)\s+([^\s]+)\s+([^\s]+)\s+([^\s]+\s+[^\s]+)\s+([^\s]+)\s+(.*)\s*(OK\S*)\s+(\S+)\s+(.+\s+)+(\S+)$

Re: REGEX to extract null/empty field as it has values

larmesto — Mon, 21 Mar 2016 15:25:32 GMT

Thanks for your reply. The problem is that I need to use only plain regex for extraction because I'm using the props.conf file to achive this, before any use of search & reporting app.

regards,

Re: REGEX to extract null/empty field as it has values

larmesto — Mon, 21 Mar 2016 15:32:03 GMT

Hi somesoni2,
Thanks for your reply. I've give that regex a try but It only works for some lines.

Regards,

Re: REGEX to extract null/empty field as it has values

alemarzu — Mon, 21 Mar 2016 16:49:25 GMT

Somesoni's regex should work, the problem was your data sample, all the records ended with "OK+A" but it looks like not after all.

So, you have to change/remove \+A\s+after the OK from the regular expresion.