Getting Data In

How to edit my current props and transforms.conf to eliminate the first 10 lines of my sample log?

TheJagoff
Communicator

I have a log that I want to throw the first 9 lines to the bit bucket, but I can’t seem to get the transforms.conf to do it.
It’s doing the opposite of what I want it to do – it’s eliminating everything below the time\tthread… line

Here’s the the first 12 lines of the log… named trace.log

traceVersion=4.3.0
hostName=server1
clusterName=ADS
cpuType=INTEL
osType=LINUXRDH
buildType=gold
initCause=normal
initTime=Tue Aug 16 19:34:24 2016 (1471376064) schema=
time\tthreadID\tlevel\tunitName\tmessage
Aug 16 19:34:24:133     0xB76D06C0      3       srkDefault      SrkInitTime = 1471376064
Aug 16 19:34:24:143     0xB76D06C0      1       srkDefault      Watchdog Sucessfully Started
Aug 16 19:34:35:123     0xB76D06C0      0       srkDefault      Changing Trace Unit Launcher to Level 3

Here’s the props.conf

[trace-sourcetype]
TIME_PREFIX = ^
TIME_FORMAT = %b %d %H:%M:%S:%3N
MAX_TIMESTAMP_LOOKAHEAD = 19
SHOULD_LINEMERGE = false
LINE_BREAKER = ([\r\n]+)\w{3}\s\d{2}\s\d{2}:\d{2}:\d{2}:\d{3}
TRUNCATE = 999999
TRANSFORMS-t1=eliminate_headers
category = Custom
pulldown_type = true

Here’s the transforms.conf

[eliminate_headers]
REGEX=^(?:trace|hostName|clusterName|cpuType|osType|buildType|initCause|initTime|time)\s
DEST_KEY=queue
FORMAT=nullQueue

As always, any and all help is much appreciated.

Thank you

0 Karma

skoelpin
SplunkTrust
SplunkTrust

You can use SEDCMD command which is like sed..

You need to set SEDCMD in your props.conf since this needs to happen at index-time

[trace-sourcetype]
 TIME_PREFIX = ^
 TIME_FORMAT = %b %d %H:%M:%S:%3N
 MAX_TIMESTAMP_LOOKAHEAD = 19
 SHOULD_LINEMERGE = false
 LINE_BREAKER = ([\r\n]+)\w{3}\s\d{2}\s\d{2}:\d{2}:\d{2}:\d{3}
 TRUNCATE = 999999
 TRANSFORMS-t1=eliminate_headers
 category = Custom
 pulldown_type = true
 SEDCMD-strip-detail-msg = time\tthreadID\tlevel\tunitName\tmessage

http://docs.splunk.com/Documentation/Splunk/4.3.1/Data/Anonymizedatausingconfigurationfiles

0 Karma
Get Updates on the Splunk Community!

Splunk Mobile: Your Brand-New Home Screen

Meet Your New Mobile Hub  Hello Splunk Community!  Staying connected to your data—no matter where you are—is ...

Introducing Value Insights (Beta): Understand the Business Impact your organization ...

Real progress on your strategic priorities starts with knowing the business outcomes your teams are delivering ...

Enterprise Security (ES) Essentials 8.3 is Now GA — Smarter Detections, Faster ...

As of today, Enterprise Security (ES) Essentials 8.3 is now generally available, helping SOC teams simplify ...