Getting Data In

Log file is monitored, but all lines are one event in Splunk?

richtate
Path Finder

I have a log file that Splunk is monitoring.  The problem is, I think, that a custom python script runs and outputs the results at one time to the log file.  The forwarder it taking the entire entry from the script as one event, but I need each line to be an event.   How do I configure the forwarder to parse the output to the log file? 

Here is what I have configured:

input.conf:

[monitor://D:\Tools\DailyChecks\Reports\Actionable_report_output_PROD.txt]
index=test_7d
sourcetype=Ibm:BigFix:DailyChecks
disabled=0

props.conf:

[Ibm:BigFix:DailyChecks]
EVENT_BREAKER_ENABLE=false
EVENT_BREAKER=([\r\n]+)
SHOULD_LINEMERGE=false
LINE_BREAKER=([\r\n]+)
NO_BINARY_CHECK=false
CHARSET=UTF-8
MAX_TIMESTAMP_LOOKAHEAD=30
# disabled=false
TZ=UTC

Sample of the log file:

-------------Report for-----------PROD
DETAIL Take Action==> Number of encryption certificates of bes license: [0]

FAIL Take Action==> 1.7.6: Actionsite Size Check Actionsite Size Check

FAIL Take Action==> ActionSite Size is too large: ['63733 KB']

DETAIL Take Action==> Total Stopped/Expired Action count (more than 30 days old): [['Total: 96', 'Single Top-Level:4', 'Baseline Component: 92']]

FAIL Take Action==> 1.10.5: Duplicate Computers (by Computer Name) Check for Duplicated Computers

FAIL Take Action==> There are at least 100 duplicates of the following computers: ['PL-MTL-P-151', 'PL-MTL-P-41', 'SIMICS-MACHINE', 'SIMICSLESS-VM', 'localhost.localdomain', 'simics-vm061', 'simics-vm062', 'simics-vm063', 'simics-vm064', 'simics-vm065', 'simics-vm066', 'simics-vm067']

0 Karma
1 Solution

richgalloway
SplunkTrust
SplunkTrust

Index-time properties for parsing data must be installed on the first Splunk instance that parses the data.  Usually, this is an indexer, but might be a heavy forwarders. (In rare cases it can be a UF, but that is not the case here.)

The conf files must be in a 'default' or 'local' directory 2 levels subordinate to %SPLUNK_HOME%\Program Files\SplunkUniversalForwarder\etc\apps.  The intervening level is an app name.

Give your props.conf settings to your Splunk Team and let them worry where to put them.

---
If this reply helps you, Karma would be appreciated.

View solution in original post

richgalloway
SplunkTrust
SplunkTrust

The props look right, but are they in the right place?  If by "forwarder" you mean Heavy Forwarder (HF) then the props must be there.  If the forwarder is a Universal Forwarder then the props must be on the indexers, unless the data passes through a HF, in which case the HF also must  have the settings.

Don't forget to restart the indexer/HF after changing props.conf.

---
If this reply helps you, Karma would be appreciated.
0 Karma

richtate
Path Finder

Thanks, it is a Universal Forwarder running on the same server as the log files. The conf files are located in "C:\Program Files\SplunkUniversalForwarder\etc\apps". But they also need to be on the indexer? How would I inquire from our Splunk team where to look?

0 Karma

richgalloway
SplunkTrust
SplunkTrust

Index-time properties for parsing data must be installed on the first Splunk instance that parses the data.  Usually, this is an indexer, but might be a heavy forwarders. (In rare cases it can be a UF, but that is not the case here.)

The conf files must be in a 'default' or 'local' directory 2 levels subordinate to %SPLUNK_HOME%\Program Files\SplunkUniversalForwarder\etc\apps.  The intervening level is an app name.

Give your props.conf settings to your Splunk Team and let them worry where to put them.

---
If this reply helps you, Karma would be appreciated.

richtate
Path Finder

Sorry for the delay, still waiting on our Splunk admins to answer my request to copy the props.conf file to the indexer.

0 Karma
Get Updates on the Splunk Community!

Splunkbase | Splunk Dashboard Examples App for SimpleXML End of Life

The Splunk Dashboard Examples App for SimpleXML will reach end of support on Dec 19, 2024, after which no new ...

Understanding Generative AI Techniques and Their Application in Cybersecurity

Watch On-Demand Artificial intelligence is the talk of the town nowadays, with industries of all kinds ...

.conf24 | Registration Open!

Hello, hello! I come bearing good news: Registration for .conf24 is now open!   conf is Splunk’s rad annual ...