Getting Data In

I want to index a logfile without breaking up - regardless the content

Starlette
Contributor

Is there a config to index a full logfile regardless the content? I tried MAX_EVENTS=3000 only but it looks that this one needs a BREAK variable as well.

Tags (1)
0 Karma
1 Solution

gkanapathy
Splunk Employee
Splunk Employee

Use:

TRUNCATE = 9999999999999
LINE_BREAKER = (?!)

You could use:

MAX_EVENTS = 99999999
BREAK_ONLY_BEFORE = (?!)

But the former is much more efficient for Splunk to perform. (?!) is a PCRE regular expression that will always fail to match.

View solution in original post

jambajuice
Communicator

I'm trying to do the same thing. I want Splunk to index a bunch of individual rule files. My props.conf looks like this:

[nessus_plugins] TRUNCATE = 0 LINE_BREAKER = (?!) SHOULD_LINEMERGE = false

But Splunk still keeps breaking the file into chunks. I can't figure out the logic of it either. Sometimes it will split the file after 16 events or so. Other times it'll do a couple hundred lines...

Any thoughts?

Craig

0 Karma

Nicholas_Key
Splunk Employee
Splunk Employee

There is a much elegant solution to address your issue Starlette.

In your props.conf, have this stanza

[your_currently_defined_stanza]
TRUNCATE = 0
LINE_BREAKER = (?!)

The reason of using this is because we'll never know how long is long enough. So TRUNCATE = 0 tells Splunk not to break the lines at all.

Look for TRUNCATE in the following webpage http://docs.splunk.com/Documentation/Splunk/5.0/Admin/Propsconf

gkanapathy
Splunk Employee
Splunk Employee

Use:

TRUNCATE = 9999999999999
LINE_BREAKER = (?!)

You could use:

MAX_EVENTS = 99999999
BREAK_ONLY_BEFORE = (?!)

But the former is much more efficient for Splunk to perform. (?!) is a PCRE regular expression that will always fail to match.

Get Updates on the Splunk Community!

Splunk Search APIを使えば調査過程が残せます

   このゲストブログは、JCOM株式会社の情報セキュリティ本部・専任部長である渡辺慎太郎氏によって執筆されました。 Note: This article is published in both Japanese ...

Integrating Splunk Search API and Quarto to Create Reproducible Investigation ...

 Splunk is More Than Just the Web Console For Digital Forensics and Incident Response (DFIR) practitioners, ...

Congratulations to the 2025-2026 SplunkTrust!

Hello, Splunk Community! We are beyond thrilled to announce our newest group of SplunkTrust members!  The ...