Getting Data In

How to apply line breakers to data from universal forwarder?

Poojitha
Path Finder

Hi All,

I have setup a universal forwarder in windows machine to monitor static file which is in json format.

The logs are being forwarded but the point is it is forwarded as single event like below :

 

 

{"Env": "someenv12”, "Name": "test12”, "feature": "TestFeature12”, "logLevel": "info", "Id": "1234", "date": 1652187242.57, "productName": “testproduct”,  "process_name": “test process, "pid": 695, "process_status": "sleeping", "process_cpu_usage": 0.0, "process_ram_usage": 0.0, "metric_type": "system_process"}
{"Env": "someenv1”3, "Name": "test13”, "feature": "TestFeature12”, "logLevel": “error”, "Id": "234", "date": 1652187342.57, "productName": “testproduct12”,  "process_name": “test process, "pid": 685, "process_status": "sleeping", "process_cpu_usage": 0.0, "process_ram_usage": 0.0, "metric_type": “application_process}
{"Env": "someenv14”, "Name": "test14”, "feature": "TestFeature13”, “info”: “error”, "Id": "2344", "date": 1672187342.57, "productName": “testproduct13”,  "process_name": “test process, "pid": 695, "process_status": "sleeping", "process_cpu_usage": 0.0, "process_ram_usage": 0.0, "metric_type": “security”}

 

 

This entire thing is coming as one event.

I have applied line breakers in props.conf file :

 

 

[test_sourcetype]
SHOULD_LINEMERGE =false
NO_BINARY_CHECK=true
BREAK_ONLY_BEFORE={"Env"
MUST_BREAK_AFTER=\"\}
TIME_PREFIX=date
TIMEFORMAT=%s%4N
MAX_TIMESTAMP_LOOKAHEAD = 14

 

 


I have added it under /SplunkUniversalForwarder/etc/apps/splunk_TA_windows app/local/props.

None of my line breaking is getting applied , please help me on this.

Should I add props.conf under default folder ?

Regards,
NVP

Labels (2)
0 Karma

richgalloway
SplunkTrust
SplunkTrust

The universal forwarder cannot perform line breaking.  That is done by the indexer or HF.

Try using these settings in props.conf:

[test_sourcetype]
SHOULD_LINEMERGE =false
NO_BINARY_CHECK=true
LINE_BREAKER=()\{"Env"
TIME_PREFIX=date
TIMEFORMAT=%s%4N
MAX_TIMESTAMP_LOOKAHEAD = 14
---
If this reply helps you, Karma would be appreciated.
0 Karma

Poojitha
Path Finder


@richgalloway  - Thanks for your response .

So, this props.conf I will be applying in the path I have mentioned right ?

I mean inside the TA - /SplunkUniversalForwarder/etc/apps/splunk_TA_windows app/local/props.

 

Tags (1)
0 Karma

richgalloway
SplunkTrust
SplunkTrust

As I mentioned, the props.conf settings must be on an indexer or Heavy Forwarder so the path as given is incorrect.  It should be $SPLUNK_HOME/etc/apps/splunk_TA_windows app/local/props.conf, where $SPLUNK_HOME usually is /opt/splunk, but can change depending on where you installed Splunk.

---
If this reply helps you, Karma would be appreciated.
0 Karma

Poojitha
Path Finder

Apologies for my repeated question.

I understand this folder path $SPLUNK_HOME usually is /opt/splunk, so this will be our linux environment file path. 

I also understand the parsing/line_breaking cannot be done at universal forwarder.

In my case, I have installed the Universal forwarder on windows machine , I  dont have opt/path there, so it means I have to add it either on heavy forwarder or indexer ?

0 Karma

richgalloway
SplunkTrust
SplunkTrust

These props settings have no effect on a UF so we can ignore it in this case.

The settings go on the indexers, which is on a Linux server in your environment so the /opt/splunk path applies.  If your Splunk is installed in a different directory (/Splunk is not uncommon) then use that instead.

---
If this reply helps you, Karma would be appreciated.

Poojitha
Path Finder

Thanks for clarifying 🙂 

0 Karma
Get Updates on the Splunk Community!

What's new in Splunk Cloud Platform 9.1.2312?

Hi Splunky people! We are excited to share the newest updates in Splunk Cloud Platform 9.1.2312! Analysts can ...

What’s New in Splunk Security Essentials 3.8.0?

Splunk Security Essentials (SSE) is an app that can amplify the power of your existing Splunk Cloud Platform, ...

Let’s Get You Certified – Vegas-Style at .conf24

Are you ready to level up your Splunk game? Then, let’s get you certified live at .conf24 – our annual user ...