Getting Data In

How do I edit props.conf on my indexer to prevent automatic sorting of events?

seetharamanss
Explorer

Hi,

I have a problem when indexing the events through a forwarder. The forwarder is listening to a log file with first event (first record) as header event. But I could see the events are not coming as is into the indexer. It is getting sorted on datetime.

Log file:

REQID~INPCD~RQTPCD~WKFWNM~INSTNM~NODENM~PRTNCT~STRTTS~ENDTS~DURMSCT~VOLCT~REVOLCT~APPNM~RULENO~NMSCORE~CONMSCORE~ADDRSCORE~PHSCORE~SSNSCORE~YOBSCORE~MATCHIND
4472276843203022016174103072                                ~RT ~RT ~node01~RUNINSTANCE4~PCIS_ABCDPB_01~1~2016-03-02 17:41:02.771~2016-03-02 17:41:03.072
~1~1~1~1 
4477576843003022016175856617                                ~RT ~RT ~node01~RUNINSTANCE2~PCIS_ABCDPB_01~1~2016-03-02 17:58:56.022~2016-03-02 17:58:56.617
~1~1~1~1                          ~NA~NA~NA~NA~NA~NA~NA~N

In indexer:

4477576843003022016175856617                                ~RT ~RT ~node01~RUNINSTANCE2~PCIS_ABCDPB_01~1~2016-03-02 17:58:56.022~2016-03-02 17:58:56.617
~1~1~1~1                          ~NA~NA~NA~NA~NA~NA~NA~N
4472276843203022016174103072                                ~RT ~RT ~node01~RUNINSTANCE4~PCIS_ABCDPB_01~1~2016-03-02 17:41:02.771~2016-03-02 17:41:03.072
~1~1~1~1 

Props.conf in indexer

NO_BINARY_CHECK=1
SHOULD_LINEMERGE=false
TIME_FORMAT=%Y-%m-%d
TRUNCATE=300000
pulldown_type = 1

Please advise the correct configuration to get the event as is from the log file.

0 Karma

renjith_nair
Legend

Splunk Indexes data as soon as they are received and they might not know when your file writing finishes. These events are persisted to disk in their original arrival order. However, in search results events are retrieved in inverse time order to show the latest events first.

You can sort events during search to see events in arrival order.

---
What goes around comes around. If it helps, hit it with Karma 🙂
0 Karma

seetharamanss
Explorer

The first record in the file is header record. In the search head,I'm trying to extract the fields by creating the props.conf and transforms.conf in $splunk_HOME/etc/apps/appname/local . But I couldn't see the fields getting extracted ,Is it because of order I'm not able to extract the fields ? Please advise.

props.conf

[12345678_abcd_Analytic_logs]
SHOULD_LINEMERGE=false
KV_MODE = none
CHECK_FOR_HEADER = TRUE
REPORT-ext = detail_abcd_fields
pulldown_type = 1

transforms.conf

[detail_abcd_fields]
DELIMS = "~"
FIELDS = REQ_ID,INP_CHAN_CD,REQ_TYP_CD,INFA_WKFLOW_NM,INFA_INSTNC_NM,INFA_NODE_NM,INFA_PRTN_CT,PRCS_STRT_TS,PRCS_END_TS,PRCS_DUR__CT,INP_VOL_CT,REFER_VOL_CT,APPL_NM
0 Karma

renjith_nair
Legend

The order of indexing should not affect the extraction process. From the above configuration, it more looks like an indexer time extraction and for that you have to place the configuration on indexers.

If you just want search time extractions, try with splunk web

http://docs.splunk.com/Documentation/Splunk/6.4.1/Knowledge/Managesearch-timefieldextractions

If you still want to do search time field extractions using config files, refer to http://docs.splunk.com/Documentation/Splunk/6.4.1/Knowledge/Createandmaintainsearch-timefieldextract...

Also remember, whenever you do configurations using files, then you need to restart splunk and with the web, it's not required.

---
What goes around comes around. If it helps, hit it with Karma 🙂
0 Karma
Get Updates on the Splunk Community!

Introduction to Splunk Observability Cloud - Building a Resilient Hybrid Cloud

Introduction to Splunk Observability Cloud - Building a Resilient Hybrid Cloud  In today’s fast-paced digital ...

Observability protocols to know about

Observability protocols define the specifications or formats for collecting, encoding, transporting, and ...

Take Your Breath Away with Splunk Risk-Based Alerting (RBA)

WATCH NOW!The Splunk Guide to Risk-Based Alerting is here to empower your SOC like never before. Join Haylee ...