How do I configure line breaking in props.conf based on my sample logs?

[meenuvn]

Wanted to do custom line breaking for a sourcetype.
Logs looks like below. Currently every line is identified as an event, whereas I would need to split it based on m_username row
ie, Line 1-5 as one event and Line 6-10 as sec event and so on..

Tried adding the props.conf as

BREAK_ONLY_BEFORE = Trace: m_username: CAR_CoBrand_eReceipt_API
SHOULD_LINEMERGE = True

Also tried with

BREAK_ONLY_BEFORE = Trace: m_username: CAR_CoBrand_eReceipt_API
SHOULD_LINEMERGE = false

Both didnt help. Could you please help me understand what's going wrong?

Logs:

1. 6/16/2016 2:28:19 PM Trace: m_username: CAR_CoBrand_eReceipt_API
2. 6/16/2016 2:28:19 PM Trace: Line2
3. 6/16/2016 2:28:19 PM Trace: Line3
4. ----
5. ----
6. 6/16/2016 3:00:39 PM Trace: m_username: CAR_CoBrand_eReceipt_API
7. 6/16/2016 2:28:19 PM Trace: Line2
8. 6/16/2016 2:28:19 PM Trace: Line3
9. ----
10. ----

[aladda_splunk · Accepted solution]

Give this a try. Splunk appears to be picking up time format ok. You can try tweaking it as well to be certain

[ trace_sourcetype]
SHOULD_LINEMERGE=false
NO_BINARY_CHECK=true
MAX_TIMESTAMP_LOOKAHEAD=36
LINE_BREAKER=(\n\r)*(?:\d+\/\d+\/\d+\s\d+\:\d+:\d+\s\w+\sTrace: m_username: CAR_CoBrand_eReceipt_API)

[meenuvn]

Hi aladda,
Wanted to make sure what i'm trying is correct.
I'm trying to use this configuration in indexer and not in forwarder.
since its a cloud env, i got this added through support team and then did a splunk restart.
But the new events are still showing up as earlier and not with the new config!!

[aladda_splunk]

These would go on the Indexer(s) props.conf or the heavy forwarder if you have one between your universal forwarder and the indexer
Couple of questions
1) Did you create a new source_type using these parameters or did you update an existing sourcetype.
2) If you created a new sourcetype then you'll need to update your forwarders inputs to use that
3) If you updated an existing sourcetype, do you have multiple versions of that sourcetype stanza in different apps? The cloud team could tell you that by running a command on the indexers

I'd recommend trying new sourcetype and update one forwarder input to use that new sourcetype to see if it gives you what you're looking for. You can create sourcetypes from the UI in Splunkcloud. It pushes them down to the indexers automagically

[meenuvn]

1) I was trying to update an existing sourcetype.
2)Also This sourcetype is configured in just one corporate server and not used by any other apps too.
Let me try with a different sourcetype
One qn i wanted to ask is that i do have a Heavy Forwarder, but havent applied the props there..Just did it on cloud. Would that a prob?

[aladda_splunk]

Depends on what the hwf is used for. Is it doing just collection of data or also indexing i.e, indexandForward flag value on props.conf.

[meenuvn]

I got it working by adding config to HF. Thank You

[ryanoconnor]

I can definitely help. I have two questions for you though.

Are you consistently going to have two empty lines made up of 4 dashes?

There are 3 different timestamps for each "event". Which timestamp is going to be the most relevant?

[meenuvn]

Hi ryanoconnor,
Those are not empty lines. I meant to say there are a set of lines starting with "Trace: m_username: CAR_CoBrand_eReceipt_API".
Each line will have a sequence of timestamps in the order it occurs.
I want to start a new event "Trace: m_username: CAR_CoBrand_eReceipt_API".
Let me know if this clarify.