Splunk Search
Highlighted

How to write the regex for the removal of a header from a log file?

Path Finder

I am trying to remove the header from a log file. I know that I need to put a stanza in props.conf on the forwarder and then create a transforms.conf like the following:

[skip_header_logfile]
REGEX = << 20-30 characters of your header line >>
DEST_KEY = queue
FORMAT = nullQueue

My problem is that my REGEX skills are horrible. My log file looks like this:

<Header>
  <Product>Microsoft SQL Server Reporting Services Version 2011.0110.2100.060 ((SQL11_RTM).120210-1917 )</Product>
  <Locale>English (United States)</Locale>
  <TimeZone>Central Standard Time</TimeZone>
  <Path>C:\Program Files\Microsoft SQL Server\MSRS11.MSSQLSERVER\Reporting Services\Logfiles\ReportServerService__01_06_2015_00_01_11.log</Path>
  <SystemName>SERVER</SystemName>
  <OSName>Microsoft Windows NT 6.1.7601 Service Pack 1</OSName>
  <OSVersion>6.1.7601</OSVersion>
  <ProcessID>10653</ProcessID>
  <Virtualization>None</Virtualization>
</Header>
library!WindowsService_112!195c!01/06/2015-00:01:11:: i INFO: Call to CleanBatch()

All that I need is REGEX that will select everything from to and of course include the header text, but I can not get it to work. Any help would be greatly appreciated.

0 Karma
Highlighted

Re: How to write the regex for the removal of a header from a log file?

Builder

AFAIK, having regex expression go to nullQueue will discard entire event, not just the matched pattern. You should be looking for a SEDCMD.

Try this:

props.conf
[your_sourcetype]
SEDCMD-null = s/(?s)<Header>.*(?=<\/Header>)<\/Header>//

Having said so, proceed with caution as this works directly at index time and there is no way to get back lost data. I suggest you try the command on sample data before putting it into production.

0 Karma
Highlighted

Re: How to write the regex for the removal of a header from a log file?

Path Finder

Thanks for the quick response, I will give it a shot and I am working with sample logs until I get it sorted.

0 Karma
Highlighted

Re: How to write the regex for the removal of a header from a log file?

SplunkTrust
SplunkTrust

Converted this to an answer. Bliide, if you are still around could you check if this works and accept this answer, or otherwise comment back on what's still needed?

0 Karma