Getting Data In

I want to index a logfile without breaking up - regardless the content

Starlette
Contributor

Is there a config to index a full logfile regardless the content? I tried MAX_EVENTS=3000 only but it looks that this one needs a BREAK variable as well.

Tags (1)
0 Karma
1 Solution

gkanapathy
Splunk Employee
Splunk Employee

Use:

TRUNCATE = 9999999999999
LINE_BREAKER = (?!)

You could use:

MAX_EVENTS = 99999999
BREAK_ONLY_BEFORE = (?!)

But the former is much more efficient for Splunk to perform. (?!) is a PCRE regular expression that will always fail to match.

View solution in original post

jambajuice
Communicator

I'm trying to do the same thing. I want Splunk to index a bunch of individual rule files. My props.conf looks like this:

[nessus_plugins] TRUNCATE = 0 LINE_BREAKER = (?!) SHOULD_LINEMERGE = false

But Splunk still keeps breaking the file into chunks. I can't figure out the logic of it either. Sometimes it will split the file after 16 events or so. Other times it'll do a couple hundred lines...

Any thoughts?

Craig

0 Karma

Nicholas_Key
Splunk Employee
Splunk Employee

There is a much elegant solution to address your issue Starlette.

In your props.conf, have this stanza

[your_currently_defined_stanza]
TRUNCATE = 0
LINE_BREAKER = (?!)

The reason of using this is because we'll never know how long is long enough. So TRUNCATE = 0 tells Splunk not to break the lines at all.

Look for TRUNCATE in the following webpage http://docs.splunk.com/Documentation/Splunk/5.0/Admin/Propsconf

gkanapathy
Splunk Employee
Splunk Employee

Use:

TRUNCATE = 9999999999999
LINE_BREAKER = (?!)

You could use:

MAX_EVENTS = 99999999
BREAK_ONLY_BEFORE = (?!)

But the former is much more efficient for Splunk to perform. (?!) is a PCRE regular expression that will always fail to match.

Get Updates on the Splunk Community!

[Puzzles] Solve, Learn, Repeat: Dynamic formatting from XML events

This challenge was first posted on Slack #puzzles channelFor a previous puzzle, I needed a set of fixed-length ...

Enter the Agentic Era with Splunk AI Assistant for SPL 1.4

  🚀 Your data just got a serious AI upgrade — are you ready? Say hello to the Agentic Era with the ...

Stronger Security with Federated Search for S3, GCP SQL & Australian Threat ...

Splunk Lantern is a Splunk customer success center that provides advice from Splunk experts on valuable data ...