topic Re: Why can't I get the right field extractions from a PSV file using auto header in transforms.conf? in Splunk Search

Why can't I get the right field extractions from a PSV file using auto header in transforms.conf?

shariinPH — Wed, 25 Mar 2015 05:13:07 GMT

Hi all,

I have here a log file with a header and I'm using transforms.conf to extract the fields, but I'm not getting the right results.

for reference:

my log file consist of:

ARU|Portion|AR Text Sched|From Date|To Date|
02000000|02_AG0|SCAL MRU 02_AG0|02/01/20|12/31/20|
02001000|02_AG1|SCAL MRU 02_AG1|02/01/20|12/31/20|
02002000|02_AG2|SCAL MRU 02_AG2|02/01/20|12/31/20|
02003000|02_AG3|SCAL MRU 02_AG3|02/01/20|12/31/20|

I put props.conf both on:
C:\Program Files\Splunk\etc\system\local\props.conf
C:\Program Files\Splunk\etc\app\Maynilad\local\props.conf

[rbil_mrsched]
NO_BINARY_CHECK = true
SHOULD_LINEMERGE = false
category = Custom
disabled = false
pulldown_type = true
INDEXED_EXTRACTIONS = PSV
REPORT-AutoHeader = rbil_mrsched_trans

and in my transforms.conf
C:\Program Files\Splunk\etc\system\local\transfoms.conf

[rbil_mrsched_trans]
DELIMS= "|"
FIELDS="RbillARU","|","RbillPortion","|","RbillARTextSched","|","RbillFromDate","|","RbillToDate","|"

Values should be
RbillARU:

RbillPortion:

02_AG0
02_AG1
02_AG2
02_AG3

RbillARTextSched:

SCAL MRU 02_AG0
SCAL MRU 02_AG1
SCAL MRU 02_AG2
SCAL MRU 02_AG3

RbillFromDate:

02/01/20

RbillToDate:

12/31/20

but the results are:
02000000 for RbillARU (correct)
no values for RbillPortion
SCAL MRU 02_AG0 for RbillPortion (wrong this should be the result for RbillARTextSched)
12/31/20 for RbillARTextSched (wrong this should be the result for RbillToDate)
no values/result for RbillFromDate
no values/result for RbillToDate

Please help me with this. thanks

Re: Why can't I get the right field extractions from a PSV file using auto header in transforms.conf?

stephane_cyrill — Wed, 25 Mar 2015 09:03:58 GMT

Hi, I want to try But i need your file.
is it a csv?
can you send it to
cyrilleko@gmail.com

Re: Why can't I get the right field extractions from a PSV file using auto header in transforms.conf?

shariinPH — Wed, 25 Mar 2015 09:45:58 GMT

Its a txtfile .. i'll send you

Re: Why can't I get the right field extractions from a PSV file using auto header in transforms.conf?

stephane_cyrill — Wed, 25 Mar 2015 09:55:01 GMT

Hi , I have put your sample data in a text file that i indexed. you can use this regex to have your fields extracted as you like.

index=* sourcetype=txt | rex field=_raw "^(?P\\s+\\d+)\\|(?P[^\\|]+)\\|(?P[^\\|]+)\\|(?P[^\\|]+)\\|(?P[^\\|]+)"|table RbillARU RbillPortion RbillARTextSched RbillFromDate

Re: Why can't I get the right field extractions from a PSV file using auto header in transforms.conf?

stephane_cyrill — Wed, 25 Mar 2015 10:18:58 GMT

ok i will take a look.

Re: Why can't I get the right field extractions from a PSV file using auto header in transforms.conf?

shariinPH — Thu, 26 Mar 2015 02:33:46 GMT

hello, does it work on your machine?

Re: Why can't I get the right field extractions from a PSV file using auto header in transforms.conf?

s2_splunk — Thu, 26 Mar 2015 05:36:15 GMT

You have specified INDEXED_EXTRACTIONS = PSV, Splunk should do the right thing automatically.

You definitely don't need a transforms.conf (aside from it being incorrect), please review this documentation

Re: Why can't I get the right field extractions from a PSV file using auto header in transforms.conf?

shariinPH — Thu, 26 Mar 2015 06:44:03 GMT

with this i have to remove the configs on my transforms.conf?

Re: Why can't I get the right field extractions from a PSV file using auto header in transforms.conf?

s2_splunk — Thu, 26 Mar 2015 06:52:12 GMT

Yes, although it doesn't matter if it doesn't get referenced.
I also wouldn't specify anything in ../etc/system/local but instead put all your configurations for this in a separate app context. Whatever you decide, definitely only have it ONE place.

I would just try:

[rbil_mrsched]
NO_BINARY_CHECK = true
SHOULD_LINEMERGE = false
INDEXED_EXTRACTIONS = PSV

You should see that your events show up with the field names as defined in the header row of the PSV input file, assuming you specified sourcetype=rbil_mrsched in your inputs.conf.

If you don't like those field names, you can create field aliases on your search head, or use the rename command in your searches.