Getting Data In

Reading large Files using Splunk UF

akshatj2
Path Finder

Hi,

I am currently trying to read logs file of size 10Gb. I have changed thruput to 0 but still takes about 30 min-1 hr for Splunk to finish reading the file. Is there a way to increase the reading speed further for splunk UF.

0 Karma

skalliger
SplunkTrust
SplunkTrust

Can you show us your props.conf and inputs.conf stanza? Have you tried to set EVENT_BREAKER as well as EVENT_BREAKER_ENABLE in the props.conf for that sourcetype? I don't know of any performance tests done using those settings, but I'd try setting

EVENT_BREAKER_ENABLE = true
EVENT_BREAKER = \r\n

Skalli

0 Karma
Register for .conf21 Now! Go Vegas or Go Virtual!

How will you .conf21? You decide! Go in-person in Las Vegas, 10/18-10/21, or go online with .conf21 Virtual, 10/19-10/20.