help with breaking events from router

[ranjitbrhm1]

Good Day All. I came across a log file which seems to be missing the carriage and ends. Can anyone assist me in breaking this log into events. Ideally I would like the events to be broken after the (1.1.1.1) in the log file. Any help is highly appreciated. Below is how the log file looks like. For some odd reason i cannot figure out why Splunk dont want to accept my regex for breaking events at (1.1.1.1)

Sep 6 09:13:00 RouterName cosco: Sep 6 14:12:56.872: %SYS-5-CONFIG_I: Configured from console by admin on vty0 (1.1.1.1) Sep 6 09:13:01 RouterName 83: Sep 6 14:12:57.872: %SYS-6-LOGGINGHOST_STARTSTOP: Logging to host 1.1.1.1 started - CLI initiated Sep 6 09:14:42 RouterName 84: Sep 6 14:14:39.048: %SYS-5-CONFIG_I: Configured from console by admin on vty0 (1.1.1.1) Sep 6 09:18:13 RouterName 85: Sep 6 14:18:10.047: %SYS-5-CONFIG_I: Configured from console by admin on vty0 (1.1.1.1) Sep 6 09:20:44 RouterName 86: Sep 6 14:20:35.991: %SYS-5-CONFIG_I: Configured from console by admin on vty0 (1.1.1.1) Sep 6 09:20:45 RouterName 87: Sep 6 14:20:41.991: %SYS-6-LOGGINGHOST_STARTSTOP: Logging to host 1.1.1.1 stopped - CLI initiated Sep 6 09:20:45 RouterName 88: Sep 6 14:20:41.991: %SYS-6-LOGGINGHOST_STARTSTOP: Logging to host 1.1.1.1 started - CLI initiated Sep 6 09:25:12 RouterName 89: %SYS-5-CONFIG_I: Configured from console by admin on vty0 (1.1.1.1) Sep 6 12:42:16 RouterName 90: %SYS-5-CONFIG_I: Configured from console by admin on vty0 (1.1.1.1) Sep 6 12:42:47 RouterName 91: %SYS-5-CONFIG_I: Configured from console by admin on vty0 (1.1.1.1) Sep 6 12:44:52 RouterName 92: %SYS-5-CONFIG_I: Configured from console by admin on vty0 (1.1.1.1) Sep 7 06:20:59 RouterName 93: SSH2 0: Unexpected message received Sep 7 07:02:56 RouterName 94: SSH2 0: Unexpected mesg type received Sep 7 13:18:06 RouterName 95: %SYS-5-CONFIG_I: Configured from console by admin on vty0 (1.1.1.1) Sep 7 13:18:06 RouterName 96: %SEC-6-IPACCESSLOGP: list 120 denied udp 10.0.0.66(137) -> 10.0.0.11(137), 33 packets Sep 6 09:13:00 RouterName 82: Sep 6 14:12:56.872: %SYS-5-CONFIG_I: Configured from console by admin on vty0 (1.1.1.1) Sep 6 09:13:01 RouterName 83: Sep 6 14:12:57.872: %SYS-6-LOGGINGHOST_STARTSTOP: Logging to host 1.1.1.1 started - CLI initiated Sep 6 09:14:42 RouterName 84: Sep 6 14:14:39.048: %SYS-5-CONFIG_I: Configured from console by admin on vty0 (1.1.1.1) Sep 6 09:18:13 RouterName 85: Sep 6 14:18:10.047: %SYS-5-CONFIG_I: Configured from console by admin on vty0 (1.1.1.1) Sep 6 09:20:44 RouterName 86: Sep 8 14:20:35.991: %SYS-5-CONFIG_I: Configured from console by admin on vty0 (1.1.1.1) Sep 6 09:20:45 RouterName 87: Sep 6 14:20:41.991: %SYS-6-LOGGINGHOST_STARTSTOP: Logging to host 1.1.1.1 stopped - CLI initiated Sep 6 09:20:45 RouterName 88: Sep 6 14:20:41.991: %SYS-6-LOGGINGHOST_STARTSTOP: Logging to host 1.1.1.1 started - CLI initiated Sep 6 09:25:12 RouterName 89: %SYS-5-CONFIG_I: Configured from console by admin on vty0 (1.1.1.1) Sep 6 12:42:16 RouterName 90: %SYS-5-CONFIG_I: Configured from console by admin on vty0 (1.1.1.1) Sep 6 12:42:47 RouterName 91: %SYS-5-CONFIG_I: Configured from console by admin on vty0 (1.1.1.1) Sep 6 12:44:52 RouterName 92: %SYS-5-CONFIG_I: Configured from console by admin on vty0 (1.1.1.1) Sep 7 06:20:59 RouterName 93: SSH2 0: Unexpected message received Sep 7 07:02:56 RouterName 94: SSH2 0: Unexpected mesg type received Sep 7 13:18:06 RouterName 95: %SYS-5-CONFIG_I: Configured from console by admin on vty0 (1.1.1.1) Sep 7 13:18:06 RouterName 96: %SEC-6-IPACCESSLOGP: list 120 denied udp 10.0.0.66(137) -> 10.0.0.11(137), 33 packets

[kmorris_splunk]

A good practice for data sources with line breaking issues out of the box, is to get a sample and ingest through the GUI. At least up to the point of creating a sourcetype (you can cancel the ingest after saving the new sourcetype). This allows you to get a preview of the data. The two things you need to get correct when ingesting data is: timestamping and linebreaking. The data preview allows you to see if Splunk is timestamping correctly and if it is linebreaking in the appropriate place.

The data preview also allows you to change settings like those in the post by @harsmarvania57. You can immediately see the effects of the changes and know if you have everything correct. Once the data looks good in the preview, you can save it as a sourcetype, which you would reference in your inputs.conf.

A good tool to help figure out the regex for the LINE_BREAKER is regex101.com.

This isn't an answer, you already have a good one, but hopefully it is helpful information.

[skoelpin]

To extend on this explanation. You should always reuse sourcetypes when possible. Logging format will be relative to the sourcetype, so if you were to create multiple sourcetypes for existing formats, you are doubling up the work to write the props and maintain them. It's a good practice to use the punct field to identify what existing sourcetypes may have the format you are wanting to add

[harsmarvania57]

Hi @ranjitbrhm1,

You can try below configuration on Indexer/Heavy Forwarder whichever comes first from UF.

props.conf

[yoursourcetype]
SHOULD_LINEMERGE=false
LINE_BREAKER=(\s)\w{3}\s\d{1,2}\s\d{2}\:\d{2}\:\d{2}\s
TIME_FORMAT=%b %e %H:%M:%S
MAX_TIMESTAMP_LOOKAHEAD=14