props config for logs

[Strangertinz]

Hi Splunk Community, 

I am trying to create a props.conf for the sample log file below. 
My goal is to 
    * Delete the Header tag and remove the data from being ingested. 
    * Break the individual events by starting with ( "library!WindowsService_98!..." OR "processing!ReportServer_0-127!" )
    * Extracting time stamp such as ( "!11/26/2023-00:21:18::")

Heres the props.conf that I have so far but it is not working. 
---------

[sourcetype_name]

disabled = false
SHOULD_LINEMERGE = false
MAX_TIMESTAMP_LOOKAHEAD = 80
TIME_FORMAT = %m/%d/%Y-%H:%M:%S
LINE_BREAKER = ([\r\n]+)library!
SEDCMD-null = (<Header>([\s\S]*?)<\/Header>)

disabled



------------------- sample log file -------------------------
<Header>
<Product>Microsoft SQL Server Reporting Services Version 2007.0100.6000.029 ((Random_value).18802-2848 )</Product>
<Locale>English (United States)</Locale>
<TimeZone>Central Daylight Time</TimeZone>
<Path>C:\Program Files\Microsoft SQL Server\MSRS10.MSSQLSERVER\Reporting Services\Logfiles\ReportServerService__11_26_2023_00_00_01.log</Path>
<SystemName>hostName01</SystemName>
<OSName>Microsoft Windows NT 6.2.9200</OSName>
<OSVersion>6.2.9200</OSVersion>
<ProcessID>3088</ProcessID>
</Header>library!WindowsService_98!1234!11/26/2023-00:00:01:: i INFO: Call to CleanBatch()
library!WindowsService_98!1234!11/26/2023-00:00:01:: i INFO: Cleaned 0 batch records, 0 policies, 0 sessions, 0 cache entries, 0 snapshots, 0 chunks, 0 running jobs, 0 persisted streams, 0 segments, 0 segment mappings.
library!WindowsService_98!1234!11/26/2023-00:00:01:: i INFO: Call to CleanBatch() ends
library!WindowsService_98!1218!11/26/2023-00:10:01:: i INFO: Call to CleanBatch()
library!WindowsService_98!1218!11/26/2023-00:10:01:: i INFO: Cleaned 0 batch records, 0 policies, 1 sessions, 0 cache entries, 1 snapshots, 14 chunks, 0 running jobs, 0 persisted streams, 9 segments, 9 segment mappings.
library!WindowsService_98!1218!11/26/2023-00:10:01:: i INFO: Call to CleanBatch() ends
library!WindowsService_98!d00!11/26/2023-00:20:01:: i INFO: Call to CleanBatch()
library!WindowsService_98!d00!11/26/2023-00:20:01:: i INFO: Cleaned 0 batch records, 0 policies, 0 sessions, 0 cache entries, 0 snapshots, 0 chunks, 0 running jobs, 0 persisted streams, 0 segments, 0 segment mappings.
library!WindowsService_98!d00!11/26/2023-00:20:01:: i INFO: Call to CleanBatch() ends
library!ReportServer_0-127!2558!11/26/2023-00:21:18:: i INFO: RenderForNewSession('/Hampton.Common.Reports/BOL')
processing!ReportServer_0-127!2558!11/26/2023-00:21:18:: e ERROR: Throwing Microsoft.ReportingServices.ReportProcessing.ReportProcessingException: There is no data for the field at position 19., ;
Info: Microsoft.ReportingServices.ReportProcessing.ReportProcessingException: There is no data for the field at position 19.
processing!ReportServer_0-127!2558!11/26/2023-00:21:18:: e ERROR: Throwing Microsoft.ReportingServices.ReportProcessing.ReportProcessingException: There is no data for the field at position 54., ;
Info: Microsoft.ReportingServices.ReportProcessing.ReportProcessingException: There is no data for the field at position 54.
processing!ReportServer_0-127!2558!11/26/2023-00:21:18:: e ERROR: Throwing Microsoft.ReportingServices.ReportProcessing.ReportProcessingException: There is no data for the field at position 61., ;
Info: Microsoft.ReportingServices.ReportProcessing.ReportProcessingException: There is no data for the field at position 61.
processing!ReportServer_0-127!2558!11/26/2023-00:21:18:: e ERROR: Throwing Microsoft.ReportingServices.ReportProcessing.ReportProcessingException: There is no data for the field at position 62., ;
Info: Microsoft.ReportingServices.ReportProcessing.ReportProcessingException: There is no data for the field at position 62.
processing!ReportServer_0-127!2558!11/26/2023-00:21:19:: e ERROR: Throwing Microsoft.ReportingServices.ReportProcessing.ReportProcessingException: There is no data for the field at position 1., ;
Info: Microsoft.ReportingServices.ReportProcessing.ReportProcessingException: There is no data for the field at position 1.
processing!ReportServer_0-127!2558!11/26/2023-00:21:19:: e ERROR: Throwing Microsoft.ReportingServices.ReportProcessing.ReportProcessingException: There is no data for the field at position 2., ;
Info: Microsoft.ReportingServices.ReportProcessing.ReportProcessingException: There is no data for the field at position 2.
processing!ReportServer_0-127!2558!11/26/2023-00:21:19:: e ERROR: Throwing Microsoft.ReportingServices.ReportProcessing.ReportProcessingException: There is no data for the field at position 1., ;
Info: Microsoft.ReportingServices.ReportProcessing.ReportProcessingException: There is no data for the field at position 1.
processing!ReportServer_0-127!2558!11/26/2023-00:21:19:: e ERROR: Throwing Microsoft.ReportingServices.ReportProcessing.ReportProcessingException: There is no data for the field at position 2., ;
Info: Microsoft.ReportingServices.ReportProcessing.ReportProcessingException: There is no data for the field at position 2.
library!WindowsService_98!1234!11/26/2023-00:30:01:: i INFO: Call to CleanBatch()
------------------- sample log file end -------------------------

[richgalloway]

SEDCMD settings must contain either an s or y command not just a regex.

To properly extract a timestamp, the props stanza should contain TIME_PREFIX, TIME_FORMAT, and MAX_TIMESTAMP_LOOKAHEAD settings.

[sourcetype_name]
disabled = false
SHOULD_LINEMERGE = false
MAX_TIMESTAMP_LOOKAHEAD = 80
TIME_FORMAT = %m/%d/%Y-%H:%M:%S
TIME_PREFIX = \d!
LINE_BREAKER = ([\r\n]+)library!
SEDCMD-null = s/\<Header>[\s\S]*?\<\/Header>//g

You may have a problem with time zones, depending on the zones of the Splunk server and that in the data.  Ideally, the time zone should be specified as part of the timestamp rather than as a separate element.  The time zone should be a recognized abbreviation such as "CST" or "-0600".  BTW, Central Daylight Time is not in effect in November.

[Strangertinz]

Hi @richgalloway, 

Thanks for getting back to me!


I tried the props.conf you proposed and got an error. Can you please try to upload the sample data with the sourcetype config you provided and see if you have any luck. 

[richgalloway]

These settings may be cleaner, but I'm not sure what I'm trying to fix.

SHOULD_LINEMERGE=false
LINE_BREAKER=()library!
NO_BINARY_CHECK=true
TIME_FORMAT=%m/%d/%Y-%H:%M:%S
TIME_PREFIX=!\w{3,4}!
SEDCMD-null=s/\<Header>[\s\S]*?\<\/Header>//g
MAX_TIMESTAMP_LOOKAHEAD=80

[Strangertinz]

@richgalloway 
I am still getting the same error. Are you able to copy the sample data and ingest it into Splunk to see the errors I am getting? 

Thanks

[richgalloway]

I did copy the example text and ingest it successfully.  I did not see the encoded text you see.

[Strangertinz]

@richgalloway 

Can you share a picture of the sourcetype along with the Splunk web screenshot? I am still getting errors on my end.


Thanks

[richgalloway · Accepted solution]

[Strangertinz]

Thanks its fixed now

[richgalloway]

What was the error?

[Strangertinz]

@richgalloway 

I am getting the error below. I can't even get Splunk to interpret the data as regular text. 

[Sridevi1]

Please change the sourcetype and try