hi guys,
i am trying to merge these lines into a event
so far i tried
[cycledata]
EVENT_BREAKER = (CycleDataTask finished)
SHOULD_LINEMERGE = false
i got block of lines starting with CycleDataTask started and finishing with CycleDataTask finished and i want to group them into a single event for each started finished.
and MUST BREAK AFTER same regex
these is an example:
2019-05-09 13:29:02.3975 INFO CycleData - CycleDataTask started ________________________________________________________
2019-05-09 13:29:06.3746 INFO CycleData - Pool has NEW TICKETS:-> =
2019-05-09 13:29:06.3746 INFO CycleData - Pool has NEW TICKETS: ->
2019-05-09 13:29:06.3746 INFO CycleData - Pool has NEW TICKETS: ->
2019-05-09 13:29:06.3746 INFO CycleData - Pool has NEW TICKETS: ->
2019-05-09 13:29:06.8166 INFO CycleData - Pool has been updated succesfully.
2019-05-09 13:29:06.8166 INFO CycleData - Pool has been updated succesfully.
2019-05-09 13:29:06.8166 INFO CycleData - Pool has been updated succesfully.
2019-05-09 13:29:06.8166 INFO CycleData - Pool has been updated succesfully.
2019-05-09 13:29:06.8166 INFO CycleData - CycleDataTask finished _______________________________________________________
thank you
I took below sample data and verified the config that I mentioned
2019-05-09 14:41:02.3975 INFO CycleData - CycleDataTask started
2019-05-09 14:41:06.3746 INFO CycleData - Pool has NEW TICKETS:->
2019-05-09 14:41:06.3746 INFO CycleData - Pool has NEW TICKETS: ->
2019-05-09 14:41:06.3746 INFO CycleData - Pool has NEW TICKETS: ->
2019-05-09 14:41:06.3746 INFO CycleData - Pool has NEW TICKETS: ->
2019-05-09 14:41:06.8166 INFO CycleData - Pool has been updated succesfully.
2019-05-09 14:41:06.8166 INFO CycleData - Pool has been updated succesfully.
2019-05-09 14:41:06.8166 INFO CycleData - Pool has been updated succesfully.
2019-05-09 14:41:06.8166 INFO CycleData - Pool has been updated succesfully.
2019-05-09 14:41:06.8166 INFO CycleData - CycleDataTask finished
2019-05-10 14:41:02.3975 INFO CycleData - CycleDataTask started
2019-05-10 14:41:06.3746 INFO CycleData - Pool has NEW TICKETS:->
2019-05-10 14:41:06.3746 INFO CycleData - Pool has NEW TICKETS: ->
2019-05-10 14:41:06.3746 INFO CycleData - Pool has NEW TICKETS: ->
2019-05-10 14:41:06.3746 INFO CycleData - Pool has NEW TICKETS: ->
2019-05-10 14:41:06.8166 INFO CycleData - Pool has been updated succesfully.
2019-05-10 14:41:06.8166 INFO CycleData - Pool has been updated succesfully.
2019-05-10 14:41:06.8166 INFO CycleData - Pool has been updated succesfully.
2019-05-10 14:41:06.8166 INFO CycleData - Pool has been updated succesfully.
2019-05-10 14:41:06.8166 INFO CycleData - CycleDataTask finished
2019-05-08 14:41:02.3975 INFO CycleData - CycleDataTask started
2019-05-08 14:41:06.3746 INFO CycleData - Pool has NEW TICKETS:->
2019-05-08 14:41:06.3746 INFO CycleData - Pool has NEW TICKETS: ->
2019-05-08 14:41:06.3746 INFO CycleData - Pool has NEW TICKETS: ->
2019-05-08 14:41:06.3746 INFO CycleData - Pool has NEW TICKETS: ->
2019-05-08 14:41:06.8166 INFO CycleData - Pool has been updated succesfully.
2019-05-08 14:41:06.8166 INFO CycleData - Pool has been updated succesfully.
2019-05-08 14:41:06.8166 INFO CycleData - Pool has been updated succesfully.
2019-05-08 14:41:06.8166 INFO CycleData - Pool has been updated succesfully.
2019-05-08 14:41:06.8166 INFO CycleData - CycleDataTask finished
Config I have tried
LINE_BREAKER = CycleDataTask\sfinished([\r\n]*)
MUST_BREAK_AFTER = CycleDataTask\sfinished
SHOULD_LINEMERGE = true
TIME_PREFIX = ^
MAX_TIMESTAMP_LOOKAHEAD = 26
TIME_FORMAT = %Y-%m-%d %H:%M:%S.%4N
This is parsing as you are expecting, hopefully this helps
it's not working for me, as i said below, i believe there is something wrong going on with my cluster and config files.
as soon i find a solution and this work i'll mark it as working
thank you
this is how it looks like
i think there is something going on with my cluster, if i upload a txt sample, and i add the regex (finished) it merges it almost fine, but then, when i add to props.conf does not work at all.
For sample data
2019-05-09 14:41:02.3975 INFO CycleData - CycleDataTask started
2019-05-09 14:41:06.3746 INFO CycleData - Pool has NEW TICKETS:->
2019-05-09 14:41:06.3746 INFO CycleData - Pool has NEW TICKETS: ->
2019-05-09 14:41:06.3746 INFO CycleData - Pool has NEW TICKETS: ->
2019-05-09 14:41:06.3746 INFO CycleData - Pool has NEW TICKETS: ->
2019-05-09 14:41:06.8166 INFO CycleData - Pool has been updated succesfully.
2019-05-09 14:41:06.8166 INFO CycleData - Pool has been updated succesfully.
2019-05-09 14:41:06.8166 INFO CycleData - Pool has been updated succesfully.
2019-05-09 14:41:06.8166 INFO CycleData - Pool has been updated succesfully.
2019-05-09 14:41:06.8166 INFO CycleData - CycleDataTask finished
Please find solution
[cycledata]
LINE_BREAKER = CycleDataTask finished([\r\n]+)
SHOULD_LINEMERGE = false
cheers
this does not work
eh? Can you please verify if the sample data is like above?
Your example had all lines merged already, so no settings required. I have split that into individual lines
Please put your sample data again in a formatted way as it exactly occurs in your file (not in Splunk)
check now please
reading again my message i was not very clear, i edited it.
Hi
Just check if you gave (EVENT_BREAKER) instead of LINE_BREAKER attribute
[cycledata]
LINE_BREAKER = (CycleDataTask finished)
SHOULD_LINEMERGE = false
the above should work
Thanks
no it does not work
i almost find a solution adding the example in data file but now it cuts off the word 'finished' which i use as regex. and of course i need it in.
(finished)
Is the example a single event you want to break into multiple events or multiple events you want to make into a single event?
the example has been edited, and like that seems just one long line, instead i have multiple lines which i want to merge in a single event
it should start here:
2019-05-09 13:29:02.3975 INFO CycleData - CycleDataTask started ________________________________________________________
and finish here:
2019-05-09 13:29:06.8166 INFO CycleData - CycleDataTask finished _______________________________________________________