All Apps and Add-ons

How to configure the Splunk Add-on for Sophos to properly recognize the EventTime field for incoming data?

Path Finder

Hi,

I am attempting to set up the Sophos Add-On (App 1854) and have encountered a quandary.

I am setting it up using a forwarder on the Sophos Enterprise Console. The Reporting Interface is already there and working fine. LogWriter is putting logs out as expected. The logs closely match the ones included with the Add-On with the exception that mine do not have quotes (") around the data. I used all the default settings (but specified my own index to send data into) and found that while all the data was ingested, the EventTime field was not recognized as the time the event occurred so all the events were imported and stamped as happening "now". I reviewed the props.conf and modified these entries for the relevant types:

   TIME_PREFIX = EventTime="   (changed to remove the '=' and '"'
   TIME_FORMAT = %Y-%m-%d %H:%M:%S    (verified)
   MAX_TIMESTAMP_LOOKAHEAD = 25    (changed to 75 to match actual log files)

however that did not appear to help.

I did raise it with Sophos just in case it was a "quote" issue and I have found that the output from Reporting Interface/LogWriter does not have quotes and isn't easy to change to use quotes.

Any thoughts as to what I should look at ?

Thankyou

0 Karma
1 Solution

SplunkTrust
SplunkTrust

Try this please:

TIME_PREFIX = EventTime=    #<- added equals back
TIME_FORMAT = "%Y-%m-%d %H:%M:%S"     #<- added quotes because of the space
MAX_TIMESTAMP_LOOKAHEAD = 78  #<- 78 as we discussed 

View solution in original post

0 Karma

SplunkTrust
SplunkTrust

Try this please:

TIME_PREFIX = EventTime=    #<- added equals back
TIME_FORMAT = "%Y-%m-%d %H:%M:%S"     #<- added quotes because of the space
MAX_TIMESTAMP_LOOKAHEAD = 78  #<- 78 as we discussed 

View solution in original post

0 Karma

Path Finder

Thankyou, sadly already had the "=" in there. I have this set in the local\props.conf
[sophos:firewall]
TIME_PREFIX = EventTime=
TIME_FORMAT = %Y-%m-%d %H:%M:%S
MAX_TIMESTAMP_LOOKAHEAD = 78

So, interestingly, I tried again this morning and I think I found the issue, or possibly it's a fluke. I wanted to try again today as the indexers/search head do a staged restart early in the morning. I am thinking that perhaps the changes on the forwarder were ok but the same props.conf changes on the indexer/search head had not been picked up. When I tried with a different set of data today the EventTime and _Time were correct.

I will try again later with another sourcetype just to make sure. If that is the case, then it was the MAX_TIMESTAMP_LOOKAHEAD = 78 that most likely resolved it. Il will let you know later today or tomorrow.

Thankyou !!!

0 Karma

Path Finder

Just added another sourcetype. Events are coming in correctly now.

Thankyou again. Much appreciated.

0 Karma

SplunkTrust
SplunkTrust

You're very welcome. Thanks for staying tuned and reporting back!

0 Karma

SplunkTrust
SplunkTrust

Can you please provide an example _raw event?

0 Karma

Path Finder

Raw records below (raw view in Splunk Search) with appropriate fields obsfucated. The _time entry for all 3 of these is :

1/6/16
11:56:20.000 AM .

InsertedAt=2016-01-06 00:52:05; EventID=1247; EventTime=2016-01-06 00:52:04; EventTypeID=; EventType=; Name=; ReportingName=; UserName=NT AUTHORITY\SYSTEM; ActionID=; Action=; SubTypeID=; SubType=; Role=; FileName=; FilePath=; FileVersion=; FileChecksum=; CommandLine=; Session=; Desktop=; Location=; ProtocolID=; Protocol=; DirectionID=; Direction=; LocalAddress=; RemoteAddress=; LocalPort=; RemotePort=; ComputerName=COMPNAM; ComputerDomain=THEDOM; ComputerIPAddress=99.99.99.99

InsertedAt=2016-01-04 02:17:47; EventID=1246; EventTime=2016-01-04 02:17:47; EventTypeID=; EventType=; Name=; ReportingName=; UserName=NT AUTHORITY\SYSTEM; ActionID=; Action=; SubTypeID=; SubType=; Role=; FileName=; FilePath=; FileVersion=; FileChecksum=; CommandLine=; Session=; Desktop=; Location=; ProtocolID=; Protocol=; DirectionID=; Direction=; LocalAddress=; RemoteAddress=; LocalPort=; RemotePort=; ComputerName=COMPNAM; ComputerDomain=THEDOM; ComputerIPAddress=99.99.99.99

InsertedAt=2016-01-04 02:17:41; EventID=1245; EventTime=2016-01-04 02:17:41; EventTypeID=; EventType=; Name=; ReportingName=; UserName=NT AUTHORITY\SYSTEM; ActionID=; Action=; SubTypeID=; SubType=; Role=; FileName=; FilePath=; FileVersion=; FileChecksum=; CommandLine=; Session=; Desktop=; Location=; ProtocolID=; Protocol=; DirectionID=; Direction=; LocalAddress=; RemoteAddress=; LocalPort=; RemotePort=; ComputerName=COMPNAM; ComputerDomain=THEDOM; ComputerIPAddress=99.99.99.99
0 Karma

SplunkTrust
SplunkTrust

If you copy everything up to the semicolon ";" after EventTime, you'll get to the number of 76..

This means your MAX_TIMESTAMP_LOOKAHEAD should be 76 not 75. However, you have these EventID fields which equal a number in the thousands... it could just as easily report an eventID of 65535 (the max). Therefore you probably want to add another digit to MAX_TIMESTAMP_LOOKAHEAD. Bringing my final recommendation of 77 or 78.

Please try both MAX_TIMESTAMP_LOOKAHEAD = 77 and MAX_TIMESTAMP_LOOKAHEAD = 78 and let me know the results.

0 Karma

Path Finder

I may have another try tomorrow. I altered the lookahead to 78 and the following was the result (forwarder restarted to send data on 19th Jan at around 10:30am local time):

 1/18/16
8:17:11.000 AM  
InsertedAt=2016-01-17 21:12:44; EventID=1277; EventTime=2016-01-17 21:12:43; EventTypeID=; EventType=; Name=; ReportingName=; UserName=NT AUTHORITY\SYSTEM; ActionID=; Action=; SubTypeID=; SubType=; Role=; FileName=; FilePath=; FileVersion=; FileChecksum=; CommandLine=; Session=; Desktop=; Location=; ProtocolID=; Protocol=; DirectionID=; Direction=; LocalAddress=; RemoteAddress=; LocalPort=; RemotePort=; ComputerName=MYCOMP; ComputerDomain=MYDOM; ComputerIPAddress=99.99.99.99

    1/18/16
8:17:11.000 AM  
InsertedAt=2016-01-17 21:11:38; EventID=1276; EventTime=2016-01-17 21:11:38; EventTypeID=; EventType=; Name=; ReportingName=; UserName=NT AUTHORITY\SYSTEM; ActionID=; Action=; SubTypeID=; SubType=; Role=; FileName=; FilePath=; FileVersion=; FileChecksum=; CommandLine=; Session=; Desktop=; Location=; ProtocolID=; Protocol=; DirectionID=; Direction=; LocalAddress=; RemoteAddress=; LocalPort=; RemotePort=; ComputerName=MYCOMP; ComputerDomain=MYDOM; ComputerIPAddress=99.99.99.99

    1/18/16
8:17:11.000 AM  
InsertedAt=2016-01-17 21:11:34; EventID=1275; EventTime=2016-01-17 21:11:33; EventTypeID=; EventType=; Name=; ReportingName=; UserName=NT AUTHORITY\SYSTEM; ActionID=; Action=; SubTypeID=; SubType=; Role=; FileName=; FilePath=; FileVersion=; FileChecksum=; CommandLine=; Session=; Desktop=; Location=; ProtocolID=; Protocol=; DirectionID=; Direction=; LocalAddress=; RemoteAddress=; LocalPort=; RemotePort=; ComputerName=MYCOMP; ComputerDomain=MYDOM; ComputerIPAddress=99.99.99.99
0 Karma

SplunkTrust
SplunkTrust

Your time prefix should have the equal sign too.

0 Karma

SplunkTrust
SplunkTrust

Can you repost the props we have now?

0 Karma

Path Finder

Will do, thankyou !

0 Karma

Path Finder

Hi,
Thankyou. Will do next week when I have access again. I'm also going to have another go in my test environment.

0 Karma

Champion

Just to make sure: you did verify your settings by restarting splunk and checking newly indexed data? Existing data will not be affected.

0 Karma

Path Finder

Hi. Absolutely, yes. Part of my paranoia process 🙂

State of Splunk Careers

Access the Splunk Careers Report to see real data that shows how Splunk mastery increases your value and job satisfaction.

Find out what your skills are worth!