All Apps and Add-ons

How to configure the Splunk Add-on for Sophos to properly recognize the EventTime field for incoming data?

rhysjones
Path Finder

Hi,

I am attempting to set up the Sophos Add-On (App 1854) and have encountered a quandary.

I am setting it up using a forwarder on the Sophos Enterprise Console. The Reporting Interface is already there and working fine. LogWriter is putting logs out as expected. The logs closely match the ones included with the Add-On with the exception that mine do not have quotes (") around the data. I used all the default settings (but specified my own index to send data into) and found that while all the data was ingested, the EventTime field was not recognized as the time the event occurred so all the events were imported and stamped as happening "now". I reviewed the props.conf and modified these entries for the relevant types:

   TIME_PREFIX = EventTime="   (changed to remove the '=' and '"'
   TIME_FORMAT = %Y-%m-%d %H:%M:%S    (verified)
   MAX_TIMESTAMP_LOOKAHEAD = 25    (changed to 75 to match actual log files)

however that did not appear to help.

I did raise it with Sophos just in case it was a "quote" issue and I have found that the output from Reporting Interface/LogWriter does not have quotes and isn't easy to change to use quotes.

Any thoughts as to what I should look at ?

Thankyou

0 Karma
1 Solution

jkat54
SplunkTrust
SplunkTrust

Try this please:

TIME_PREFIX = EventTime=    #<- added equals back
TIME_FORMAT = "%Y-%m-%d %H:%M:%S"     #<- added quotes because of the space
MAX_TIMESTAMP_LOOKAHEAD = 78  #<- 78 as we discussed 

View solution in original post

0 Karma

jkat54
SplunkTrust
SplunkTrust

Try this please:

TIME_PREFIX = EventTime=    #<- added equals back
TIME_FORMAT = "%Y-%m-%d %H:%M:%S"     #<- added quotes because of the space
MAX_TIMESTAMP_LOOKAHEAD = 78  #<- 78 as we discussed 
0 Karma

rhysjones
Path Finder

Thankyou, sadly already had the "=" in there. I have this set in the local\props.conf
[sophos:firewall]
TIME_PREFIX = EventTime=
TIME_FORMAT = %Y-%m-%d %H:%M:%S
MAX_TIMESTAMP_LOOKAHEAD = 78

So, interestingly, I tried again this morning and I think I found the issue, or possibly it's a fluke. I wanted to try again today as the indexers/search head do a staged restart early in the morning. I am thinking that perhaps the changes on the forwarder were ok but the same props.conf changes on the indexer/search head had not been picked up. When I tried with a different set of data today the EventTime and _Time were correct.

I will try again later with another sourcetype just to make sure. If that is the case, then it was the MAX_TIMESTAMP_LOOKAHEAD = 78 that most likely resolved it. Il will let you know later today or tomorrow.

Thankyou !!!

0 Karma

rhysjones
Path Finder

Just added another sourcetype. Events are coming in correctly now.

Thankyou again. Much appreciated.

0 Karma

jkat54
SplunkTrust
SplunkTrust

You're very welcome. Thanks for staying tuned and reporting back!

0 Karma

jkat54
SplunkTrust
SplunkTrust

Can you please provide an example _raw event?

0 Karma

rhysjones
Path Finder

Raw records below (raw view in Splunk Search) with appropriate fields obsfucated. The _time entry for all 3 of these is :

1/6/16
11:56:20.000 AM .

InsertedAt=2016-01-06 00:52:05; EventID=1247; EventTime=2016-01-06 00:52:04; EventTypeID=; EventType=; Name=; ReportingName=; UserName=NT AUTHORITY\SYSTEM; ActionID=; Action=; SubTypeID=; SubType=; Role=; FileName=; FilePath=; FileVersion=; FileChecksum=; CommandLine=; Session=; Desktop=; Location=; ProtocolID=; Protocol=; DirectionID=; Direction=; LocalAddress=; RemoteAddress=; LocalPort=; RemotePort=; ComputerName=COMPNAM; ComputerDomain=THEDOM; ComputerIPAddress=99.99.99.99

InsertedAt=2016-01-04 02:17:47; EventID=1246; EventTime=2016-01-04 02:17:47; EventTypeID=; EventType=; Name=; ReportingName=; UserName=NT AUTHORITY\SYSTEM; ActionID=; Action=; SubTypeID=; SubType=; Role=; FileName=; FilePath=; FileVersion=; FileChecksum=; CommandLine=; Session=; Desktop=; Location=; ProtocolID=; Protocol=; DirectionID=; Direction=; LocalAddress=; RemoteAddress=; LocalPort=; RemotePort=; ComputerName=COMPNAM; ComputerDomain=THEDOM; ComputerIPAddress=99.99.99.99

InsertedAt=2016-01-04 02:17:41; EventID=1245; EventTime=2016-01-04 02:17:41; EventTypeID=; EventType=; Name=; ReportingName=; UserName=NT AUTHORITY\SYSTEM; ActionID=; Action=; SubTypeID=; SubType=; Role=; FileName=; FilePath=; FileVersion=; FileChecksum=; CommandLine=; Session=; Desktop=; Location=; ProtocolID=; Protocol=; DirectionID=; Direction=; LocalAddress=; RemoteAddress=; LocalPort=; RemotePort=; ComputerName=COMPNAM; ComputerDomain=THEDOM; ComputerIPAddress=99.99.99.99
0 Karma

jkat54
SplunkTrust
SplunkTrust

If you copy everything up to the semicolon ";" after EventTime, you'll get to the number of 76..

This means your MAX_TIMESTAMP_LOOKAHEAD should be 76 not 75. However, you have these EventID fields which equal a number in the thousands... it could just as easily report an eventID of 65535 (the max). Therefore you probably want to add another digit to MAX_TIMESTAMP_LOOKAHEAD. Bringing my final recommendation of 77 or 78.

Please try both MAX_TIMESTAMP_LOOKAHEAD = 77 and MAX_TIMESTAMP_LOOKAHEAD = 78 and let me know the results.

0 Karma

rhysjones
Path Finder

I may have another try tomorrow. I altered the lookahead to 78 and the following was the result (forwarder restarted to send data on 19th Jan at around 10:30am local time):

 1/18/16
8:17:11.000 AM  
InsertedAt=2016-01-17 21:12:44; EventID=1277; EventTime=2016-01-17 21:12:43; EventTypeID=; EventType=; Name=; ReportingName=; UserName=NT AUTHORITY\SYSTEM; ActionID=; Action=; SubTypeID=; SubType=; Role=; FileName=; FilePath=; FileVersion=; FileChecksum=; CommandLine=; Session=; Desktop=; Location=; ProtocolID=; Protocol=; DirectionID=; Direction=; LocalAddress=; RemoteAddress=; LocalPort=; RemotePort=; ComputerName=MYCOMP; ComputerDomain=MYDOM; ComputerIPAddress=99.99.99.99

    1/18/16
8:17:11.000 AM  
InsertedAt=2016-01-17 21:11:38; EventID=1276; EventTime=2016-01-17 21:11:38; EventTypeID=; EventType=; Name=; ReportingName=; UserName=NT AUTHORITY\SYSTEM; ActionID=; Action=; SubTypeID=; SubType=; Role=; FileName=; FilePath=; FileVersion=; FileChecksum=; CommandLine=; Session=; Desktop=; Location=; ProtocolID=; Protocol=; DirectionID=; Direction=; LocalAddress=; RemoteAddress=; LocalPort=; RemotePort=; ComputerName=MYCOMP; ComputerDomain=MYDOM; ComputerIPAddress=99.99.99.99

    1/18/16
8:17:11.000 AM  
InsertedAt=2016-01-17 21:11:34; EventID=1275; EventTime=2016-01-17 21:11:33; EventTypeID=; EventType=; Name=; ReportingName=; UserName=NT AUTHORITY\SYSTEM; ActionID=; Action=; SubTypeID=; SubType=; Role=; FileName=; FilePath=; FileVersion=; FileChecksum=; CommandLine=; Session=; Desktop=; Location=; ProtocolID=; Protocol=; DirectionID=; Direction=; LocalAddress=; RemoteAddress=; LocalPort=; RemotePort=; ComputerName=MYCOMP; ComputerDomain=MYDOM; ComputerIPAddress=99.99.99.99
0 Karma

jkat54
SplunkTrust
SplunkTrust

Your time prefix should have the equal sign too.

0 Karma

jkat54
SplunkTrust
SplunkTrust

Can you repost the props we have now?

0 Karma

rhysjones
Path Finder

Will do, thankyou !

0 Karma

rhysjones
Path Finder

Hi,
Thankyou. Will do next week when I have access again. I'm also going to have another go in my test environment.

0 Karma

jeffland
SplunkTrust
SplunkTrust

Just to make sure: you did verify your settings by restarting splunk and checking newly indexed data? Existing data will not be affected.

0 Karma

rhysjones
Path Finder

Hi. Absolutely, yes. Part of my paranoia process 🙂

Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...