Getting Data In

How to edit my current props and transforms.conf to eliminate the first 10 lines of my sample log?

TheJagoff
Communicator

I have a log that I want to throw the first 9 lines to the bit bucket, but I can’t seem to get the transforms.conf to do it.
It’s doing the opposite of what I want it to do – it’s eliminating everything below the time\tthread… line

Here’s the the first 12 lines of the log… named trace.log

traceVersion=4.3.0
hostName=server1
clusterName=ADS
cpuType=INTEL
osType=LINUXRDH
buildType=gold
initCause=normal
initTime=Tue Aug 16 19:34:24 2016 (1471376064) schema=
time\tthreadID\tlevel\tunitName\tmessage
Aug 16 19:34:24:133     0xB76D06C0      3       srkDefault      SrkInitTime = 1471376064
Aug 16 19:34:24:143     0xB76D06C0      1       srkDefault      Watchdog Sucessfully Started
Aug 16 19:34:35:123     0xB76D06C0      0       srkDefault      Changing Trace Unit Launcher to Level 3

Here’s the props.conf

[trace-sourcetype]
TIME_PREFIX = ^
TIME_FORMAT = %b %d %H:%M:%S:%3N
MAX_TIMESTAMP_LOOKAHEAD = 19
SHOULD_LINEMERGE = false
LINE_BREAKER = ([\r\n]+)\w{3}\s\d{2}\s\d{2}:\d{2}:\d{2}:\d{3}
TRUNCATE = 999999
TRANSFORMS-t1=eliminate_headers
category = Custom
pulldown_type = true

Here’s the transforms.conf

[eliminate_headers]
REGEX=^(?:trace|hostName|clusterName|cpuType|osType|buildType|initCause|initTime|time)\s
DEST_KEY=queue
FORMAT=nullQueue

As always, any and all help is much appreciated.

Thank you

0 Karma

skoelpin
SplunkTrust
SplunkTrust

You can use SEDCMD command which is like sed..

You need to set SEDCMD in your props.conf since this needs to happen at index-time

[trace-sourcetype]
 TIME_PREFIX = ^
 TIME_FORMAT = %b %d %H:%M:%S:%3N
 MAX_TIMESTAMP_LOOKAHEAD = 19
 SHOULD_LINEMERGE = false
 LINE_BREAKER = ([\r\n]+)\w{3}\s\d{2}\s\d{2}:\d{2}:\d{2}:\d{3}
 TRUNCATE = 999999
 TRANSFORMS-t1=eliminate_headers
 category = Custom
 pulldown_type = true
 SEDCMD-strip-detail-msg = time\tthreadID\tlevel\tunitName\tmessage

http://docs.splunk.com/Documentation/Splunk/4.3.1/Data/Anonymizedatausingconfigurationfiles

0 Karma
Get Updates on the Splunk Community!

Enterprise Security Content Update (ESCU) | New Releases

In January, the Splunk Threat Research Team had one release of new security content via the Splunk ES Content ...

Expert Tips from Splunk Professional Services, Ensuring Compliance, and More New ...

Splunk Lantern is a Splunk customer success center that provides advice from Splunk experts on valuable data ...

Observability Release Update: AI Assistant, AppD + Observability Cloud Integrations & ...

This month’s releases across the Splunk Observability portfolio deliver earlier detection and faster ...