topic Re: index time extraction not working in Splunk Enterprise

index time extraction not working

aamer86 — Mon, 05 Jul 2021 16:15:35 GMT

Hi,

I have a HEC input on an indexer.

I am trying to send Palo Alto Traffic Logs over HEC

I have the this stanza in the props.conf

[source::hec]

pulldown_type = true

SHOULD_LINEMERGE = false

TIME_PREFIX = ^(?:[^,]*,){5}

MAX_TIMESTAMP_LOOKAHEAD = 100

#TRANSFORMS-sourcetype = pan_traffic

REPORT-trafic_fields = pan_trafic_fields

and this in transforms.conf

[pan_trafic_fields]

DELIMS = ","

FIELDS = "receive_time","serial_number","log_type","log_subtype","src_ip","dest_ip","rule","app","vsys","src_zone","dest_zone","src_interface","dest_interface","log_forwarding_profile","session_id","repeat_count","src_port","dest_port","transport","action","bytes","bytes_out","bytes_in","packets","start_time","duration","http_category","sequence_number","src_location","dest_location","packets_out","packets_in","session_end_reason","dvc_name","action_source","tunnel_id"

I am trying to test it with curl

curl -k "https://172.31.72.93:8088/services/collector/raw?cca3-f29f63e09fdc&sourcetype=pan:log" -H "Authorization: Splunk 92a1a276-eee8-XXXX-XXXX-11d002640ad0" -d '"2021/07/05 12:30:06",44A1B3FC68F5304,TRAFFIC,end,103.125.191.136,10.0.0.10,splunk,incomplete,vsys1,untrusted,trusted,ethernet1/3,ethernet1/2,log-forwarding-default,574277,1,52564,8088,tcp,allow,74,74,0,1,"2021/07/05 12:30:06",0,any,730218,"United States",10.0.0.0-10.255.255.255,1,0,aged-out,PA-VM,from-policy,0'

the Sourcetype is being recognised by Splunk as pan:traffic as expected but the parsing is not working on the indexers and no fields are showing in my search

am i missing something here

Re: index time extraction not working

maraman_splunk — Tue, 06 Jul 2021 07:22:50 GMT

Hi,

I think this should work with the transform that change sourcetype uncommented.

Then move the REPORTS stanza is in a sourcetype scope not a source

[pan:trafic]

REPORT-trafic_fields = pan_trafic_fields

(make sure this config is also present on sh so deploy the whole sh + idx)

that makes at least things much easier to debug with such things scoped at sourcetype level

Re: index time extraction not working

maraman_splunk — Tue, 06 Jul 2021 07:23:31 GMT

btw reports is a search time extraction

Re: index time extraction not working

Gene — Tue, 06 Jul 2021 09:13:44 GMT

Why don't you use app\addon for PaloAlto? It extracts fields without problems. Also according to your props.conf - sourcetype recognition is commented, so looks like it happens somewhere else.
#TRANSFORMS-sourcetype = pan_traffic

You can also try to download application and check config files there (easiest way) - so you will have some clue on how to modify your configs or copy them to your system.

Thank, Gene