Splunk Search

How to write the regex for the removal of a header from a log file?

Bliide
Path Finder

I am trying to remove the header from a log file. I know that I need to put a stanza in props.conf on the forwarder and then create a transforms.conf like the following:

[skip_header_logfile]
REGEX = << 20-30 characters of your header line >>
DEST_KEY = queue
FORMAT = nullQueue

My problem is that my REGEX skills are horrible. My log file looks like this:

<Header>
  <Product>Microsoft SQL Server Reporting Services Version 2011.0110.2100.060 ((SQL11_RTM).120210-1917 )</Product>
  <Locale>English (United States)</Locale>
  <TimeZone>Central Standard Time</TimeZone>
  <Path>C:\Program Files\Microsoft SQL Server\MSRS11.MSSQLSERVER\Reporting Services\Logfiles\ReportServerService__01_06_2015_00_01_11.log</Path>
  <SystemName>SERVER</SystemName>
  <OSName>Microsoft Windows NT 6.1.7601 Service Pack 1</OSName>
  <OSVersion>6.1.7601</OSVersion>
  <ProcessID>10653</ProcessID>
  <Virtualization>None</Virtualization>
</Header>
library!WindowsService_112!195c!01/06/2015-00:01:11:: i INFO: Call to CleanBatch()

All that I need is REGEX that will select everything from to and of course include the header text, but I can not get it to work. Any help would be greatly appreciated.

0 Karma

sk314
Builder

AFAIK, having regex expression go to nullQueue will discard entire event, not just the matched pattern. You should be looking for a SEDCMD.

Try this:

props.conf
[your_sourcetype]
SEDCMD-null = s/(?s)<Header>.*(?=<\/Header>)<\/Header>//

Having said so, proceed with caution as this works directly at index time and there is no way to get back lost data. I suggest you try the command on sample data before putting it into production.

0 Karma

Bliide
Path Finder

Thanks for the quick response, I will give it a shot and I am working with sample logs until I get it sorted.

0 Karma

Richfez
SplunkTrust
SplunkTrust

Converted this to an answer. Bliide, if you are still around could you check if this works and accept this answer, or otherwise comment back on what's still needed?

0 Karma
Get Updates on the Splunk Community!

Introducing the 2024 SplunkTrust!

Hello, Splunk Community! We are beyond thrilled to announce our newest group of SplunkTrust members!  The ...

Introducing the 2024 Splunk MVPs!

We are excited to announce the 2024 cohort of the Splunk MVP program. Splunk MVPs are passionate members of ...

Splunk Custom Visualizations App End of Life

The Splunk Custom Visualizations apps End of Life for SimpleXML will reach end of support on Dec 21, 2024, ...