Splunk Enterprise

Can't get events to split at desired breaks

circleup
Explorer

We are forwarding some logs to Splunk Light Cloud using a Universal Forwarder.

The inputs.conf on the forwarder successfully sends the logs and specifies a custom sourcetype.

Within the Splunk Light Cloud ui, I am trying to edit the source type to do event breaks at a regex. However, it seems no matter what I use for the regex, it has no effect on the logs. Logs continue to be somewhat arbitrarily broken up, sometimes by timestamp and other times not at all.

Our logs look like:

nginx stderr | 2016/08/02 11:03:03 [info] 24#0: *5390 client closed connection while waiting for request, client: 10.0.30.16, server: 0.0.0.0:8000
nginx stdout | 10.0.30.16 - - [02/Aug/2016:11:03:15 +0000] "GET /health-check/ HTTP/1.1" 200 234 "-" "ELB-HealthChecker/1.0" "-" - 0.001 0.001 .
nginx stdout | 10.0.31.218 - - [02/Aug/2016:11:03:15 +0000] "GET /health-check/ HTTP/1.1" 200 234 "-" "ELB-HealthChecker/1.0" "-" - 0.001 0.001 .
nginx stdout | 10.0.30.16 - - [02/Aug/2016:11:03:45 +0000] "GET /health-check/ HTTP/1.1" 200 234 "-" "ELB-HealthChecker/1.0" "-" - 0.001 0.001 .
nginx stdout | 10.0.31.218 - - [02/Aug/2016:11:03:45 +0000] "GET /health-check/ HTTP/1.1" 200 234 "-" "ELB-HealthChecker/1.0" "-" - 0.001 0.001 .

This particular example all shows as a single event in the UI. But I'm trying to split at each nginx stderr | type prefix. Patterns I've tried so far:
- \S+ \S+ \|, ^\S+ \S+ \|, ([\r\n]+)\S+ \S+ \|
- .*\|, ^.*\|
- stderr and ^.*stderr (just to try and see if I can get it to do anything)

None of these appear to have any effect on event splitting.

Looking in the "Advanced" section of the editor, the pattern corresponds to the BREAK_ONLY_BEFORE setting. But it won't let me manually specify other settings without overriding them on save.

Anyone have any ideas? Thanks!

0 Karma
1 Solution

circleup
Explorer

Finally figured it out. Appears the missing piece was I had to manually go to the menu and select "Restart".

I falsely assumed that the web UI would automatically handle reloading sourcetypes after they've been edited. That does not appear to be the case.

A warning message (e.g. "These changes will not take effect until the server has been restarted" or "Would you like to restart to make these changes take effect?") would be nice.

View solution in original post

0 Karma

circleup
Explorer

Finally figured it out. Appears the missing piece was I had to manually go to the menu and select "Restart".

I falsely assumed that the web UI would automatically handle reloading sourcetypes after they've been edited. That does not appear to be the case.

A warning message (e.g. "These changes will not take effect until the server has been restarted" or "Would you like to restart to make these changes take effect?") would be nice.

0 Karma
Get Updates on the Splunk Community!

Exporting Splunk Apps

Join us on Monday, October 21 at 11 am PT | 2 pm ET!With the app export functionality, app developers and ...

Cisco Use Cases, ITSI Best Practices, and More New Articles from Splunk Lantern

Splunk Lantern is a Splunk customer success center that provides advice from Splunk experts on valuable data ...

Build Your First SPL2 App!

Watch the recording now!.Do you want to SPL™, too? SPL2, Splunk's next-generation data search and preparation ...