Getting Data In

Line breaking powershell output

Path Finder

I've attempted multiple times mixing up LINE_BREAKER, BREAK_ONLY_BEFORE, SHOULD_LINEMERGE, BREAK_ONLY_BEFORE_DATE, nothing seems to work - I've obviously missed something.

Sample:
...........................
EventDate : 2017-05-19 14:20
Host : Server8
InstanceName : oracle
CPU_Percent : 15.19

EventDate : 2017-05-19 14:20
Host : Server8
InstanceName : oracle
CPU_Percent : 12.40

EventDate : 2017-05-19 14:20
Host : Server8
InstanceName : powershell
CPU_Percent : 0.19
...........................

The data comes in as sourcetype Windows:processmonitor from a UF and I currently have the following in my indexer props.conf:
...........................
[Windows:processmonitor]

LINE_BREAKER = EventDate

BREAK_ONLY_BEFORE = EventDate

SHOULD_LINEMERGE = false

TIME_FORMAT = %Y-%m-%d %H:%M
BREAK_ONLY_BEFORE_DATE = true
...........................

Any advice greatly appreciated!

0 Karma

Esteemed Legend

Try this:

[Windows:processmonitor]
LINE_BREAKER = ([\r\n]+)EventDate
SHOULD_LINEMERGE = false
TIME_PREFIX = EventDate\s*:\s*
TIME_FORMAT = %Y-%m-%d %H:%M
MAX_TIMESTAMP_LOOKAHEAD = 16
0 Karma

Path Finder

Thanks all, however no luck so far in either case, unfortunately.

I did however notice in the health check that there were issues in the 'Event-processing issues' section, relating to events for this sourcetype being too long in bytes, and event max was also an issue.
...............

Message

Some recently ingested events are triggering event-processing warnings and indicate the presence of one or more of these scenarios:
1. Lines in the event are too long, exceeding props.conf / TRUNCATE
2. There are too many lines per event, exceeding props.conf / MAX_EVENTS
3. The extraction of event time stamps was partially or completely unsuccessful
These event-processing issues can have a negative impact on the performance of data ingestion.
Suggested Action

Check the events that are triggering these warnings. Adjust event-processing settings as needed to ensure their proper ingestion.
...................

So I added TRUNCATE and MAX_EVENTS to the stanza, to result in the following:
.......................
[procmonitor]
TRUNCATE = 15000
MAX_EVENTS = 300
LINE_BREAKER = ([\r\n]+)EventDate
SHOULD_LINEMERGE = false
TIME_PREFIX = EventDate\s*:\s*
TIME_FORMAT = %Y-%m-%d %H:%M
MAX_TIMESTAMP_LOOKAHEAD = 16
.......................
Note I also changed the sourcetype name just to be sure there was no issue there.

Now the length and count issues are no longer showing up, but the data is the same one big event.

I'm wondering if what I see on the screen and what Splunk is looking at are two different things. I'm also wondering if I should modify my script to make the format more digestible to Splunk, somehow - perhaps xml.

0 Karma

Builder

Hello,

What is missing is SHOULD_LINEMERGE = true, The following stanza in props.conf should work fine:

[Windows:processmonitor]
SHOULD_LINEMERGE = true
TIME_FORMAT = %Y-%m-%d %H:%M

Regards

0 Karma
State of Splunk Careers

Access the Splunk Careers Report to see real data that shows how Splunk mastery increases your value and job satisfaction.

Find out what your skills are worth!