Getting Data In

How to edit my current props and transforms.conf to eliminate the first 10 lines of my sample log?

TheJagoff
Communicator

I have a log that I want to throw the first 9 lines to the bit bucket, but I can’t seem to get the transforms.conf to do it.
It’s doing the opposite of what I want it to do – it’s eliminating everything below the time\tthread… line

Here’s the the first 12 lines of the log… named trace.log

traceVersion=4.3.0
hostName=server1
clusterName=ADS
cpuType=INTEL
osType=LINUXRDH
buildType=gold
initCause=normal
initTime=Tue Aug 16 19:34:24 2016 (1471376064) schema=
time\tthreadID\tlevel\tunitName\tmessage
Aug 16 19:34:24:133     0xB76D06C0      3       srkDefault      SrkInitTime = 1471376064
Aug 16 19:34:24:143     0xB76D06C0      1       srkDefault      Watchdog Sucessfully Started
Aug 16 19:34:35:123     0xB76D06C0      0       srkDefault      Changing Trace Unit Launcher to Level 3

Here’s the props.conf

[trace-sourcetype]
TIME_PREFIX = ^
TIME_FORMAT = %b %d %H:%M:%S:%3N
MAX_TIMESTAMP_LOOKAHEAD = 19
SHOULD_LINEMERGE = false
LINE_BREAKER = ([\r\n]+)\w{3}\s\d{2}\s\d{2}:\d{2}:\d{2}:\d{3}
TRUNCATE = 999999
TRANSFORMS-t1=eliminate_headers
category = Custom
pulldown_type = true

Here’s the transforms.conf

[eliminate_headers]
REGEX=^(?:trace|hostName|clusterName|cpuType|osType|buildType|initCause|initTime|time)\s
DEST_KEY=queue
FORMAT=nullQueue

As always, any and all help is much appreciated.

Thank you

0 Karma

skoelpin
SplunkTrust
SplunkTrust

You can use SEDCMD command which is like sed..

You need to set SEDCMD in your props.conf since this needs to happen at index-time

[trace-sourcetype]
 TIME_PREFIX = ^
 TIME_FORMAT = %b %d %H:%M:%S:%3N
 MAX_TIMESTAMP_LOOKAHEAD = 19
 SHOULD_LINEMERGE = false
 LINE_BREAKER = ([\r\n]+)\w{3}\s\d{2}\s\d{2}:\d{2}:\d{2}:\d{3}
 TRUNCATE = 999999
 TRANSFORMS-t1=eliminate_headers
 category = Custom
 pulldown_type = true
 SEDCMD-strip-detail-msg = time\tthreadID\tlevel\tunitName\tmessage

http://docs.splunk.com/Documentation/Splunk/4.3.1/Data/Anonymizedatausingconfigurationfiles

0 Karma
Get Updates on the Splunk Community!

Observability Unlocked: Kubernetes Monitoring with Splunk Observability Cloud

 Ready to master Kubernetes and cloud monitoring like the pros? Join Splunk’s Growth Engineering team for an ...

Update Your SOAR Apps for Python 3.13: What Community Developers Need to Know

To Community SOAR App Developers - we're reaching out with an important update regarding Python 3.9's ...

October Community Champions: A Shoutout to Our Contributors!

As October comes to a close, we want to take a moment to celebrate the people who make the Splunk Community ...