If my log file does not have a header, how can I divide my raw data into 2 fields at index-time with the 1st space as the delimiter on each line?

[nitesh218ss]

Hi

20140902191418.351 TrxManagerFactory.CreateTrxManager Done
20140902191418.351 TransactionBaseMgr.Init
20140902191418.352 TransactionBaseMgr.Init Done
20140902191418.352 TransactionBaseMgr.ProcessTransaction
20140902191418.352 TransactionBaseMgr.InitTransaction
20140902191418.353 CardPaymentTrxMgr.IsNewTransaction
20140902191418.353 CardPaymentTrxMgr.IsNewTransaction Done
20140902191418.354 TransactionBaseMgr.GetTransaction
20140902191418.354 DL_TransactionsMgr.GetTransaction
20140902191418.371 DL_TransactionsMgr.GetTransaction Done

That is my log file. In this, there is no header name present, so i want this raw data divided into 2 field names time and Msg
The Msg (2nd field) starts after the first space.

Example:

time                    Msg 
20140902191418.352      TransactionBaseMgr.Init Done
20140902191418.371      DL_TransactionsMgr.GetTransaction Done

I am not able to set space as a delimiter because the 2nd field Msg has many spaces present in the values. Is it possible to do this at index-time?

[nitesh218ss · Accepted solution]

In Props.conf you add

FIELD_NAMES = datetime,Msg
FIELD_DELIMITER = tab

Then they divide your event(line) base on tab and provide name as datetime and Msg

[stephane_cyrill]

Hi I think this can help you:

index=????? sourcetype=???? | rex  field=_raw "^(?P<TIME>[^ ]+)\\s+(?P<MMESSAGE>.+)"|table TIME MMESSAGE

[ngatchasandra]

Hi,
In Inputs.conf , put the stanza like follow (you create an inputs.conf in
$SPLUNK_HOME/etc/system/local/ if it don't exist)

[monitor://........./your.csv]
sourcetype=yoursourcetype

In Props.conf put the stanza like follow (you create an props.conf in $SPLUNK_HOME/etc/system/local/ if it don't exist)

[yoursourcetype]
            FIELD_DELIMITER=\s
            TIMESTAMP_FIELDS=time,Msg
            HEADER_FIELD_DELIMITER=\s

[nitesh218ss]

FIELD_DELIMITER=\s that is not work because in msg have some space to they take separate field

i give you one line of log down
20140902191418.213 CardPaymentServices.Authorize Username and password supplied starting the Customer not Present Request

in this 20140902191418.213 is time and after one space after that msg(CardPaymentServices.Authorize Username and password supplied starting the Customer not Present Request )

[ngatchasandra]

Try with

FIELD_DELIMITER=\d+\s

[Tanefo]

hi,
please cant you send me your files. i would see your files initial structure. my mail: tiwa.romuald@yahoo.fr

[juvetm]

what do you want to do exactly can you explain to me more detail

[nitesh218ss]

I want in indexing time(when i add data ) i set 2 field 1st is timlog and 2nd is Msg
if you see in my log data in question
20140902191418.351 is time after that one space after that Msg(TrxManagerFactory.CreateTrxManager Done)
so i want after indexing(add data) the event result come in to field that is
timlog after Msg with default field(_time,host,sourcetype)

[juvetm]

what you ask is impossible to do that indexing time
note that went you are add data you are uploading the file so during that moment you can not give a header name at that moment because you are uploading a file

[nitesh218ss]

when we set Set Sourcetype after uploading that time any possibility to configure Set Sourcetype base on our requirement?

[juvetm]

what you ask me sir i am still think on it if it impossible at indexing time

[nitesh218ss]

i created reg for msg and time but hoe i put in header name in indexing time

reg for msg ^[^ \n]* (?P(\w+.+|.+))
reg for time ^(?P[^ ]+)