topic Re: How can I merge multiple mail log lines into one event in Getting Data In

How can I merge multiple mail log lines into one event

lars_meldgaard — Sat, 22 Oct 2016 19:34:40 GMT

I want to examine my mail server logfile. The logfile has many lines per logsession. Each line starts with a timestamp, then followed by a sessionid in [ and ]. Rest of line is data.
Each session is separated with ----------

Example lines
'Thu 2016-10-20 00:00:39: [39647:2] --> 250-ETRN
Thu 2016-10-20 00:00:39: [39647:2] --> 250-AUTH LOGIN CRAM-MD5 PLAIN
Thu 2016-10-20 00:00:39: [39647:2] --> 250-8BITMIME
Thu 2016-10-20 00:00:40: [39647:2] --> 250-STARTTLS
Thu 2016-10-20 00:00:41: [39647:2] --> 250 SIZE

Example Event separator:
Thu 2016-10-20 00:00:42: ----------

How can I make splunk Group all relevant lines into one event?
I'm running a Windows system with Splunk Light

Re: How can I merge multiple mail log lines into one event

maciep — Sun, 23 Oct 2016 12:48:09 GMT

This sounds like a good case to use the LINE_BREAKER setting in props.conf. Something like this

[your:sourcetype]
LINE_BREAKER = (\w{3}\s\d{4}\-\d{2}\-\d{2}\s\d{2}:\d{2}:\d{2}:\s\-+)
MAX_TIMESTAMP_LOOKAHEAD = 30

This will tell splunk to use those separators as line breakers instead of the default \r\n. As a result, they won't show up in your data.

Re: How can I merge multiple mail log lines into one event

lars_meldgaard — Tue, 29 Sep 2020 11:31:58 GMT

Sorry to be a bit slow here.

I'm not sure where props.conf is located.
From the web interface of Splunk, I have added the wildcard to monitor, so this is located in apps/_server_app_mail/local/inputs.conf

There is not existing props.conf file. Should I just add one in this directory?

And so your regex deletes the timestamps?

I was kinda looking for something so I can group by sessionid ([39647:2]) - (whatever is in the brackets).
End goal is to list how many sessions has a specific line associated
e.g.
Fri 2016-10-21 12:40:01: [55444:3] * zen.spamhaus.org - failed - 127.0.0.4

Hope you can elaborate a bit. Thanks

Re: How can I merge multiple mail log lines into one event

skoelpin — Sun, 23 Oct 2016 17:51:08 GMT

When you modify props.conf, it will be on the indexer, not the forwarder where inputs.conf is located. Your indexer is a central server where data is forwarded to, this is responsible for indexing and parsing the data, so it would make sense to put linebreaking rules where the data is parsed. Making changes to props.conf will not work retroactively and will ONLY apply those changes to data being indexed after the changes were applied. So you have 3 options

1) Apply the changes to props.conf and your older data will not be parsed correctly
2) Apply changes to props.conf and verify your linebreaking is correct, then delete the old data and reindex your data with the changes applied
3) Use a transaction or stats command to combine the events at search time without having to set any line breaking rules on your indexer

Are you looking for a permanent solution to group these into a single event?

Re: How can I merge multiple mail log lines into one event

lars_meldgaard — Sun, 23 Oct 2016 20:05:20 GMT

This is first time setting up the data source for splunk use. I'm looking for the best way to organize the data, so that queries will be as fast and easy to use in the future. I don't have any existing data in the Splunk server.
The Splunk data server and indexer is the same machine.
I'm looking for the location of the props.conf. Theres a lot in etc/apps and others in etc/system/default.
Which one is the correct file to change?

Re: How can I merge multiple mail log lines into one event

skoelpin — Sun, 23 Oct 2016 20:55:49 GMT

Ok this narrows it down, you should take suggestion 1 above, which is to set line_breaker in your props.conf

The props.conf file is located in %SPLUNK_HOME/etc/system/localon the indexer.

Insert this stanza into your props.conf file

[your:sourcetype]
 LINE_BREAKER = (\w{3}\s\d{4}\-\d{2}\-\d{2}\s\d{2}:\d{2}:\d{2}:\s\-+)
 MAX_TIMESTAMP_LOOKAHEAD = 30`

Replace "your:sourcetype" with the sourcetype name you specified in your inputs.conf which is on the forwarder under %SPLUNK_HOME/etc/system/local/inputs.conf

After you've made these changes on your indexer, restart the splunk service and verify the linebreaking is correct

Re: How can I merge multiple mail log lines into one event

lars_meldgaard — Tue, 29 Sep 2020 11:32:01 GMT

To Recap: put the suggested lines on splunk Indexing server in $SPLUNK$/etc/system/default/props.conf
To modify the actual input I had to add a few lines on the input of the forwarding server as well. In my case in SplunkUniversalForwarder\etc\apps_server_app_mail\local\inputs.conf

I needed
crcSalt =
ignoreOlderThan=3d
maybe because i tried a few times and need to clear the data

Re: How can I merge multiple mail log lines into one event

skoelpin — Sun, 23 Oct 2016 21:06:22 GMT

No. Do not put this under /default/props.conf

You want to place this under /etc/system/local/props.conf

Re: How can I merge multiple mail log lines into one event

lars_meldgaard — Sun, 23 Oct 2016 21:43:00 GMT

Ah just missed your post. Very good answer

Re: How can I merge multiple mail log lines into one event

lars_meldgaard — Sun, 23 Oct 2016 21:43:36 GMT

Hope I did this now. Thanks for your help