I am trying to remove the header from a log file. I know that I need to put a stanza in props.conf on the forwarder and then create a transforms.conf like the following:
[skip_header_logfile] REGEX = << 20-30 characters of your header line >> DEST_KEY = queue FORMAT = nullQueue
My problem is that my REGEX skills are horrible. My log file looks like this:
<Header> <Product>Microsoft SQL Server Reporting Services Version 2011.0110.2100.060 ((SQL11_RTM).120210-1917 )</Product> <Locale>English (United States)</Locale> <TimeZone>Central Standard Time</TimeZone> <Path>C:\Program Files\Microsoft SQL Server\MSRS11.MSSQLSERVER\Reporting Services\Logfiles\ReportServerService__01_06_2015_00_01_11.log</Path> <SystemName>SERVER</SystemName> <OSName>Microsoft Windows NT 6.1.7601 Service Pack 1</OSName> <OSVersion>6.1.7601</OSVersion> <ProcessID>10653</ProcessID> <Virtualization>None</Virtualization> </Header> library!WindowsService_112!195c!01/06/2015-00:01:11:: i INFO: Call to CleanBatch()
All that I need is REGEX that will select everything from
to and of course include the header text, but I can not get it to work. Any help would be greatly appreciated.
AFAIK, having regex expression go to nullQueue will discard entire event, not just the matched pattern. You should be looking for a SEDCMD.
props.conf [your_sourcetype] SEDCMD-null = s/(?s)<Header>.*(?=<\/Header>)<\/Header>//
Having said so, proceed with caution as this works directly at index time and there is no way to get back lost data. I suggest you try the command on sample data before putting it into production.
Converted this to an answer. Bliide, if you are still around could you check if this works and accept this answer, or otherwise comment back on what's still needed?