Getting Data In

Line breaking powershell output

a_splunk_user
Path Finder

I've attempted multiple times mixing up LINE_BREAKER, BREAK_ONLY_BEFORE, SHOULD_LINEMERGE, BREAK_ONLY_BEFORE_DATE, nothing seems to work - I've obviously missed something.

Sample:
...........................
EventDate : 2017-05-19 14:20
Host : Server8
InstanceName : oracle
CPU_Percent : 15.19

EventDate : 2017-05-19 14:20
Host : Server8
InstanceName : oracle
CPU_Percent : 12.40

EventDate : 2017-05-19 14:20
Host : Server8
InstanceName : powershell
CPU_Percent : 0.19
...........................

The data comes in as sourcetype Windows:processmonitor from a UF and I currently have the following in my indexer props.conf:
...........................
[Windows:processmonitor]

LINE_BREAKER = EventDate

BREAK_ONLY_BEFORE = EventDate

SHOULD_LINEMERGE = false

TIME_FORMAT = %Y-%m-%d %H:%M
BREAK_ONLY_BEFORE_DATE = true
...........................

Any advice greatly appreciated!

0 Karma

woodcock
Esteemed Legend

Try this:

[Windows:processmonitor]
LINE_BREAKER = ([\r\n]+)EventDate
SHOULD_LINEMERGE = false
TIME_PREFIX = EventDate\s*:\s*
TIME_FORMAT = %Y-%m-%d %H:%M
MAX_TIMESTAMP_LOOKAHEAD = 16
0 Karma

a_splunk_user
Path Finder

Thanks all, however no luck so far in either case, unfortunately.

I did however notice in the health check that there were issues in the 'Event-processing issues' section, relating to events for this sourcetype being too long in bytes, and event max was also an issue.
...............

Message

Some recently ingested events are triggering event-processing warnings and indicate the presence of one or more of these scenarios:
1. Lines in the event are too long, exceeding props.conf / TRUNCATE
2. There are too many lines per event, exceeding props.conf / MAX_EVENTS
3. The extraction of event time stamps was partially or completely unsuccessful
These event-processing issues can have a negative impact on the performance of data ingestion.
Suggested Action

Check the events that are triggering these warnings. Adjust event-processing settings as needed to ensure their proper ingestion.
...................

So I added TRUNCATE and MAX_EVENTS to the stanza, to result in the following:
.......................
[procmonitor]
TRUNCATE = 15000
MAX_EVENTS = 300
LINE_BREAKER = ([\r\n]+)EventDate
SHOULD_LINEMERGE = false
TIME_PREFIX = EventDate\s*:\s*
TIME_FORMAT = %Y-%m-%d %H:%M
MAX_TIMESTAMP_LOOKAHEAD = 16
.......................
Note I also changed the sourcetype name just to be sure there was no issue there.

Now the length and count issues are no longer showing up, but the data is the same one big event.

I'm wondering if what I see on the screen and what Splunk is looking at are two different things. I'm also wondering if I should modify my script to make the format more digestible to Splunk, somehow - perhaps xml.

0 Karma

aakwah
Builder

Hello,

What is missing is SHOULD_LINEMERGE = true, The following stanza in props.conf should work fine:

[Windows:processmonitor]
SHOULD_LINEMERGE = true
TIME_FORMAT = %Y-%m-%d %H:%M

Regards

0 Karma
Get Updates on the Splunk Community!

Introducing the 2024 SplunkTrust!

Hello, Splunk Community! We are beyond thrilled to announce our newest group of SplunkTrust members!  The ...

Introducing the 2024 Splunk MVPs!

We are excited to announce the 2024 cohort of the Splunk MVP program. Splunk MVPs are passionate members of ...

Splunk Custom Visualizations App End of Life

The Splunk Custom Visualizations apps End of Life for SimpleXML will reach end of support on Dec 21, 2024, ...