Getting Data In

I want to index a logfile without breaking up - regardless the content

Starlette
Contributor

Is there a config to index a full logfile regardless the content? I tried MAX_EVENTS=3000 only but it looks that this one needs a BREAK variable as well.

Tags (1)
0 Karma
1 Solution

gkanapathy
Splunk Employee
Splunk Employee

Use:

TRUNCATE = 9999999999999
LINE_BREAKER = (?!)

You could use:

MAX_EVENTS = 99999999
BREAK_ONLY_BEFORE = (?!)

But the former is much more efficient for Splunk to perform. (?!) is a PCRE regular expression that will always fail to match.

View solution in original post

jambajuice
Communicator

I'm trying to do the same thing. I want Splunk to index a bunch of individual rule files. My props.conf looks like this:

[nessus_plugins] TRUNCATE = 0 LINE_BREAKER = (?!) SHOULD_LINEMERGE = false

But Splunk still keeps breaking the file into chunks. I can't figure out the logic of it either. Sometimes it will split the file after 16 events or so. Other times it'll do a couple hundred lines...

Any thoughts?

Craig

0 Karma

Nicholas_Key
Splunk Employee
Splunk Employee

There is a much elegant solution to address your issue Starlette.

In your props.conf, have this stanza

[your_currently_defined_stanza]
TRUNCATE = 0
LINE_BREAKER = (?!)

The reason of using this is because we'll never know how long is long enough. So TRUNCATE = 0 tells Splunk not to break the lines at all.

Look for TRUNCATE in the following webpage http://docs.splunk.com/Documentation/Splunk/5.0/Admin/Propsconf

gkanapathy
Splunk Employee
Splunk Employee

Use:

TRUNCATE = 9999999999999
LINE_BREAKER = (?!)

You could use:

MAX_EVENTS = 99999999
BREAK_ONLY_BEFORE = (?!)

But the former is much more efficient for Splunk to perform. (?!) is a PCRE regular expression that will always fail to match.

Get Updates on the Splunk Community!

Data Management Digest – December 2025

Welcome to the December edition of Data Management Digest! As we continue our journey of data innovation, the ...

Index This | What is broken 80% of the time by February?

December 2025 Edition   Hayyy Splunk Education Enthusiasts and the Eternally Curious!    We’re back with this ...

Unlock Faster Time-to-Value on Edge and Ingest Processor with New SPL2 Pipeline ...

Hello Splunk Community,   We're thrilled to share an exciting update that will help you manage your data more ...