Line breaking does not work for events with the same timestamp

[jgaraygay]

Help please! Our data looks like the one below....

1377190800,ANAQUA_VMs,52940532,987100964550,Normal,0,161792,50,18623,4.29447,3.02706
1377190800,ANAQUA_VMs,ANAQUA_VMs-ETC,P,166810,47232,33,8

And here is our props.conf file. I believe we have tried all possible combinations of lin breaking parameters but none of them seem to work for us...

TIME_PREFIX = ^
TIME_FORMAT = %s
MAX_TIMESTAMP_LOOKAHEAD = 10
TZ = UTC
#LINE_BREAKER=([\r\n]+)
BREAK_ONLY_BEFORE_DATE = true
#SHOULD_LINEMERGE = false
#MUST_BREAK_AFTER = ([\r\n]+)

Thanks in advance!

[jgaraygay · Accepted solution]

Manually reloading the endpoints (/debug/refresh) did not work so I had to restart the Splunk daemon. I am running Splunk 5.0.3.1 build 167641.

[kristian_kolb]

This does not work?

TIME_PREFIX = ^
TIME_FORMAT = %s
MAX_TIMESTAMP_LOOKAHEAD = 20
TZ = UTC
SHOULD_LINEMERGE = false

Even though only a plain

SHOULD_LINEMERGE = false

should be enough

Are you sure that you're editing the correct props.conf file. I.e. where the parsing phase takes place;

http://wiki.splunk.com/Where_do_I_configure_my_Splunk_settings

/K

[kristian_kolb]

Yes, but the question still remains; Have you configured this on the correct Splunk instance? Depending on your setup, i.e. the chain of splunk instances that may be involved, you line-breaking configs should be on the highlighted instance.

a) file -> Heavy Forwarder-> Indexer
b) file -> Universal Forwarder ->Indexer
c) file -> Universal Forwarder ->Heavy Forwarder-> Indexer
d) file -> Indexer

Any clearer? Revisit the "where do i configure my splunk settings" link above for guidance.

/K

[jgaraygay]

The ones explicitly defined are coming from "e:\Program Files\Splunk\etc\apps\ko_props_transforms\local\props.conf" and the default ones from "e:\Program Files\Splunk\etc\system\default\props.conf"

[jgaraygay]

e:ProgramFilesSplunketcappsko_props_transformslocalprops.conf [recoverpoint_stats]
MAX_TIMESTAMP_LOOKAHEAD = 20
REPORT-rp_stats_fields = rp_stats_fields_P, rp_stats_fields_L, rp_stats_fields_R
SHOULD_LINEMERGE = false
TIME_FORMAT = %s
TIME_PREFIX = ^
TZ = UTC

e:\Program Files\Splunk\etc\system\default\props.conf
ANNOTATE_PUNCT = True
BREAK_ONLY_BEFORE = 
BREAK_ONLY_BEFORE_DATE = True
CHARSET = AUTO
DATETIME_CONFIG = \etc\datetime.xml
HEADER_MODE =

[sowings]

Btool btool btool!

Run $SPLUNK_HOME/bin/splunk cmd btool props list recoverpoint_stats --debug

This will tell you the app (or if Splunk > 5.0.3, the exact file) that contains the settings which apply for that type. The comment about hiding or showing the REPORT stanza is immaterial here; if there's no other props.conf containing that setting, the one you're editing will take precedence. If you've used the UI to edit the props, those entries are in the local/ subfolder, which contains any override settings. That is, there may be two versions of props.conf, and you're editing the wrong one.

[Ayn]

REPORT is search-time though, so if you have a setup with a search head and an indexer you need to put this on the indexer. If you're searching directly on your indexer you've put your configs in the right place though.

[jgaraygay]

Still didn't work.

[recoverpoint_stats]
TIME_PREFIX = ^
TIME_FORMAT = %s
MAX_TIMESTAMP_LOOKAHEAD = 20
TZ = UTC
SHOULD_LINEMERGE = false
REPORT-rp_stats_fields=rp_stats_fields_P, rp_stats_fields_L, rp_stats_fields_R

I believe I have the correct props.conf file because it also has a REPORT setting. And if I comment the #REPORT, I don't get the field extractions.