Getting Data In

Stripping header from input file

jhallman
Explorer

Is there a way to strip the header from a data input? This is coming from a universal forwarder

example

this is garbage
this is also garbage
end of garbage
HEADER DB_NAME DB_ID IO
timestamp test_db 1 100000
..
timestamp last_db 10 500000

I want to not index the first 4 lines (3 starting with > and the column heading line)

0 Karma

ogdin
Splunk Employee
Splunk Employee

Use Header-based Index-time field extractions:

http://docs.splunk.com/Documentation/Splunk/latest/Data/Extractfieldsfromfileheadersatindextime

In your example above, you could use HEADER_FIELD_LINE_NUMBER=4 or if there is garbage before the field names in the header FIELD_HEADER_REGEX=HEADER\s(.*)

davecroto
Splunk Employee
Splunk Employee

hello world two
hello world
pet,phone,street
cow,999-9999,taylor
dog,777-7878,balor
cat,656-5637,main
pig,878-1212,pine

transforms.conf

[HEADER_NULLQ]
REGEX= (pet|world)
DEST_KEY=queue
FORMAT=nullQueue

props.conf

[your sourcetype]
SHOULD_LINEMERGE = False
pulldown_type = 1
TRANSFORMS-HEADER_NULLQ=HEADER_NULLQ

Got questions? Get answers!

Join the Splunk Community Slack to learn, troubleshoot, and make connections with fellow Splunk practitioners in real time!

Meet up IRL or virtually!

Join Splunk User Groups to connect and learn in-person by region or remotely by topic or industry.

Get Updates on the Splunk Community!

Optimize AI at Scale: Must-Attend Observability Sessions at .conf26

conf26   With nearly 70 breakout sessions on the docket, the .conf26 Observability track is designed to help ...

How much can you really learn in 3 minutes?

Observability can certainly be hard to understand – there's a lot of jargon and buzzwords and it seems to ...

Event Series: The Agentic SOC: Trust Before Autonomy

AI is fundamentally changing security operations, but true progress requires more than just automation—it ...