Getting Data In
×

Join the Conversation

Without signing in, you're just watching from the sidelines. Sign in or Register to connect, share, and be part of the Splunk Community.
Ask a Question

hot to merge multiple lines into a single event?

0xlc
Path Finder
‎05-09-2019 09:06 AM

hi guys,

i am trying to merge these lines into a event

so far i tried

[cycledata]
EVENT_BREAKER = (CycleDataTask finished)
SHOULD_LINEMERGE = false

i got block of lines starting with CycleDataTask started and finishing with CycleDataTask finished and i want to group them into a single event for each started finished.

and MUST BREAK AFTER same regex

these is an example:

2019-05-09 13:29:02.3975 INFO CycleData - CycleDataTask started ________________________________________________________
2019-05-09 13:29:06.3746 INFO CycleData - Pool has NEW TICKETS:-> =
2019-05-09 13:29:06.3746 INFO CycleData - Pool has NEW TICKETS: ->
2019-05-09 13:29:06.3746 INFO CycleData - Pool has NEW TICKETS: -> 
2019-05-09 13:29:06.3746 INFO CycleData - Pool has NEW TICKETS: -> 
2019-05-09 13:29:06.8166 INFO CycleData - Pool has been updated succesfully.
2019-05-09 13:29:06.8166 INFO CycleData - Pool has been updated succesfully.
2019-05-09 13:29:06.8166 INFO CycleData - Pool has been updated succesfully.
2019-05-09 13:29:06.8166 INFO CycleData - Pool has been updated succesfully.
2019-05-09 13:29:06.8166 INFO CycleData - CycleDataTask finished _______________________________________________________

thank you

0 Karma
Reply

sanjeev543
Communicator
‎05-10-2019 06:45 AM

I took below sample data and verified the config that I mentioned 

2019-05-09 14:41:02.3975 INFO CycleData - CycleDataTask started
 2019-05-09 14:41:06.3746 INFO CycleData - Pool has NEW TICKETS:-> 
 2019-05-09 14:41:06.3746 INFO CycleData - Pool has NEW TICKETS: ->
 2019-05-09 14:41:06.3746 INFO CycleData - Pool has NEW TICKETS: ->
 2019-05-09 14:41:06.3746 INFO CycleData - Pool has NEW TICKETS: ->
 2019-05-09 14:41:06.8166 INFO CycleData - Pool has been updated succesfully.
 2019-05-09 14:41:06.8166 INFO CycleData - Pool has been updated succesfully.
 2019-05-09 14:41:06.8166 INFO CycleData - Pool has been updated succesfully.
 2019-05-09 14:41:06.8166 INFO CycleData - Pool has been updated succesfully.
 2019-05-09 14:41:06.8166 INFO CycleData - CycleDataTask finished
 2019-05-10 14:41:02.3975 INFO CycleData - CycleDataTask started
 2019-05-10 14:41:06.3746 INFO CycleData - Pool has NEW TICKETS:-> 
 2019-05-10 14:41:06.3746 INFO CycleData - Pool has NEW TICKETS: ->
 2019-05-10 14:41:06.3746 INFO CycleData - Pool has NEW TICKETS: ->
 2019-05-10 14:41:06.3746 INFO CycleData - Pool has NEW TICKETS: ->
 2019-05-10 14:41:06.8166 INFO CycleData - Pool has been updated succesfully.
 2019-05-10 14:41:06.8166 INFO CycleData - Pool has been updated succesfully.
 2019-05-10 14:41:06.8166 INFO CycleData - Pool has been updated succesfully.
 2019-05-10 14:41:06.8166 INFO CycleData - Pool has been updated succesfully.
 2019-05-10 14:41:06.8166 INFO CycleData - CycleDataTask finished
 2019-05-08 14:41:02.3975 INFO CycleData - CycleDataTask started
 2019-05-08 14:41:06.3746 INFO CycleData - Pool has NEW TICKETS:-> 
 2019-05-08 14:41:06.3746 INFO CycleData - Pool has NEW TICKETS: ->
 2019-05-08 14:41:06.3746 INFO CycleData - Pool has NEW TICKETS: ->
 2019-05-08 14:41:06.3746 INFO CycleData - Pool has NEW TICKETS: ->
 2019-05-08 14:41:06.8166 INFO CycleData - Pool has been updated succesfully.
 2019-05-08 14:41:06.8166 INFO CycleData - Pool has been updated succesfully.
 2019-05-08 14:41:06.8166 INFO CycleData - Pool has been updated succesfully.
 2019-05-08 14:41:06.8166 INFO CycleData - Pool has been updated succesfully.
 2019-05-08 14:41:06.8166 INFO CycleData - CycleDataTask finished

Config I have tried

LINE_BREAKER = CycleDataTask\sfinished([\r\n]*)
 MUST_BREAK_AFTER  = CycleDataTask\sfinished
 SHOULD_LINEMERGE = true
 TIME_PREFIX = ^
 MAX_TIMESTAMP_LOOKAHEAD = 26
 TIME_FORMAT = %Y-%m-%d %H:%M:%S.%4N

This is parsing as you are expecting, hopefully this helps

0 Karma
Reply

0xlc
Path Finder
‎05-10-2019 06:55 AM

it's not working for me, as i said below, i believe there is something wrong going on with my cluster and config files.

as soon i find a solution and this work i'll mark it as working

thank you

this is how it looks like

link text

0 Karma
Reply

0xlc
Path Finder
‎05-10-2019 06:12 AM

i think there is something going on with my cluster, if i upload a txt sample, and i add the regex (finished) it merges it almost fine, but then, when i add to props.conf does not work at all.

0 Karma
Reply

koshyk
Super Champion
‎05-10-2019 03:59 AM

For sample data

2019-05-09 14:41:02.3975 INFO CycleData - CycleDataTask started
2019-05-09 14:41:06.3746 INFO CycleData - Pool has NEW TICKETS:-> 
2019-05-09 14:41:06.3746 INFO CycleData - Pool has NEW TICKETS: ->
2019-05-09 14:41:06.3746 INFO CycleData - Pool has NEW TICKETS: ->
2019-05-09 14:41:06.3746 INFO CycleData - Pool has NEW TICKETS: ->
2019-05-09 14:41:06.8166 INFO CycleData - Pool has been updated succesfully.
2019-05-09 14:41:06.8166 INFO CycleData - Pool has been updated succesfully.
2019-05-09 14:41:06.8166 INFO CycleData - Pool has been updated succesfully.
2019-05-09 14:41:06.8166 INFO CycleData - Pool has been updated succesfully.
2019-05-09 14:41:06.8166 INFO CycleData - CycleDataTask finished

Please find solution

[cycledata]
LINE_BREAKER = CycleDataTask finished([\r\n]+)
SHOULD_LINEMERGE = false

cheers

0 Karma
Reply

0xlc
Path Finder
‎05-10-2019 06:11 AM

this does not work

0 Karma
Reply

koshyk
Super Champion
‎05-10-2019 06:13 AM

eh? Can you please verify if the sample data is like above?
Your example had all lines merged already, so no settings required. I have split that into individual lines

Please put your sample data again in a formatted way as it exactly occurs in your file (not in Splunk)

0 Karma
Reply

0xlc
Path Finder
‎05-10-2019 06:26 AM

check now please

reading again my message i was not very clear, i edited it.

0 Karma
Reply

PowerPacked
Builder
‎05-09-2019 04:16 PM

Hi

Just check if you gave (EVENT_BREAKER) instead of LINE_BREAKER attribute

[cycledata]
LINE_BREAKER = (CycleDataTask finished)
SHOULD_LINEMERGE = false

the above should work

Thanks

0 Karma
Reply

0xlc
Path Finder
‎05-10-2019 02:05 AM

no it does not work

i almost find a solution adding the example in data file but now it cuts off the word 'finished' which i use as regex. and of course i need it in.

(finished)

0 Karma
Reply

richgalloway
SplunkTrust
SplunkTrust
‎05-09-2019 12:57 PM

Is the example a single event you want to break into multiple events or multiple events you want to make into a single event?

---
If this reply helps you, Karma would be appreciated.
0 Karma
Reply

0xlc
Path Finder
‎05-10-2019 01:37 AM

the example has been edited, and like that seems just one long line, instead i have multiple lines which i want to merge in a single event

it should start here:

2019-05-09 13:29:02.3975 INFO CycleData - CycleDataTask started ________________________________________________________

and finish here:

2019-05-09 13:29:06.8166 INFO CycleData - CycleDataTask finished _______________________________________________________

0 Karma
Reply
Got questions? Get answers!

Join the Splunk Community Slack to learn, troubleshoot, and make connections with fellow Splunk practitioners in real time!

Meet up IRL or virtually!

Join Splunk User Groups to connect and learn in-person by region or remotely by topic or industry.

Get Updates on the Splunk Community!

[Puzzles] Solve, Learn, Repeat: Matching cron expressions

This puzzle (first published here) is based on matching timestamps to cron expressions.All the timestamps ...

Design, Compete, Win: Submit Your Best Splunk Dashboards for a .conf26 Pass

Hello Splunkers,  We’re excited to kick off a Splunk Dashboard contest! We know that dashboards are a primary ...

May 2026 Splunk Expert Sessions: Security & Observability

Level Up Your Operations: May 2026 Splunk Expert Sessions Whether you are refining your security posture or ...