[ANSWERED by to4kawa]
props.conf should be
[yoursourcetype]
DATETIME_CONFIG = CURRENT
SHOULD_LINEMERGE = false
LINE_BREAKER = .*($)
I have a Catch 22 issue. I want three things to happen.
Imagine my file is empty and then within a microsecond these three lines are added to my file.
TEST1 Fri Apr 6 20:05:59 EDT 2020
TEST2 Fri Apr 3 20:04:30 EDT 2020
TEST3 Fri Apr 1 20:05:59 EDT 2020
I would like them all to be combined into a single event. And I want the timestamp to be set to current.
If I just just start monitoring the file without any props it combines the events into a single one just fine. BUT it will try to parse the timestamps and they will be all over the place. If I mod props.conf and set DATETIME_CONFIG=CURRENT it will set the time to the current one but then split the events into single lines.
So I am in a catch 22, I can have one or the other. Any ideas what inputs/props/transforms combo I can have that ignores all time stamps, and combines the events into a single one no matter what.
I honestly want something that checks if file has not been modified for 2 seconds and then combine everything new that was added into ONE event.
Thanks!
These settings should help. Adjust the TRUNCATE
and MAX_EVENTS
settings as appropriate for your data.
inputs.conf:
[monitor://my/file]
multiline_event_extra_waittime = true
time_before_close = 2
sourcetype = foo
index = bar
props.conf:
[foo]
DATETIME_CONFIG = current
SHOULD_LINEMERGE = true
TRUNCATE = 10000
MAX_EVENTS = 256
These settings should help. Adjust the TRUNCATE
and MAX_EVENTS
settings as appropriate for your data.
inputs.conf:
[monitor://my/file]
multiline_event_extra_waittime = true
time_before_close = 2
sourcetype = foo
index = bar
props.conf:
[foo]
DATETIME_CONFIG = current
SHOULD_LINEMERGE = true
TRUNCATE = 10000
MAX_EVENTS = 256
Hi Rich,
I just tried this.
INPUTS.CONF
[monitor:///tmp/test3.log]
disabled = false
sourcetype = mytest
multiline_event_extra_waittime = true
time_before_close = 2
PROPS.CONF
[mytest]
DATETIME_CONFIG = CURRENT
SHOULD_LINEMERGE = true
TRUNCATE = 10000
MAX_EVENTS = 256
I've added
TEST8 Fri Apr 6 21:05:59 EDT 2020
TEST8 Fri Apr 3 21:04:30 EDT 2020
TEST8 Fri Apr 1 21:05:59 EDT 2020
But they came as separate events on Splunk 8.
SHOULD_LINEMERGE = false
LINE_BREAKER = .*($)
if SHOULD_LINEMERGE = false
, LINE_BREAKER
works.
.*($)
is intended to be the end of the event.
That's perfect, thank you! The below came together as one event AND the timestamp is auto set to current.
4/4/20 7:09:02.000 PM
TEST11 Fri Apr 6 22:05:59 EDT 2020
TEST11 Fri Apr 3 22:04:30 EDT 2020
TEST11 Fri Apr 1 22:05:59 EDT 2020
Appreciate your time
your welcome
.*($)
is OK, I recognized it, too.
until the file is not updated for 2 seconds or more.
what's inputs.conf?
this is not pops.conf issue.