timezone not parsed correctly in FireEye NX JSON logs

[awurster]

not sure why exactly the time stuff is commented out in the TA code, but i found that timestamps were not parsed correctly when sending our events from an NX appliance to Splunk via JSON / HTTPS and the FireEye TA v3.

event data:

...
"occurred": "2015-06-17 03:25:55+00", 
"id": "1", 
"action": "notified", 
"interface": {
"mode": "tap"
...

i found that i had to uncomment out and adjust the two TIME_ fields below to get things working.

###### FireEye JSON over HTTPS ######
# Universal Handler for FireEye JSON notifications (pre and post FireEye OS 7.1)
[fe_json]
TRUNCATE=0
SHOULD_LINEMERGE = false
LINE_BREAKER = ((?!))
KV_MODE = JSON
#TIME_PREFIX = \"occurred\"\:\s
#TIME_FORMAT = \"%Y-%m-%d %H:%M:%S+00\"

i think it should be the following, but it's not working for me. i've tried with and without quotes.

[fe_json]
...
TIME_PREFIX = "occurred"\:\s
TIME_FORMAT = "%Y-%m-%d %H:%M:%S%Z"

[TonyLeeVT]

Your observation is correct. I believe the latest version of the appliance added an "appliance-id" field which caused an error with time parsing. That bug fix will be in version 3.0.7.

The fix will be the following:

[fe_json]
TRUNCATE=0
SHOULD_LINEMERGE = false
LINE_BREAKER = ((?!))
KV_MODE = JSON
TIME_PREFIX = \"occurred\"\:\s
TIME_FORMAT = \"%Y-%m-%d %H:%M:%S+00\"
TZ = UTC

Thanks for reporting it.

[awurster]

thanks tony. this is cool, but it's sort of a hack, masking the original problem. i don't know if it has much to do with any appliance-id field.

i don't believe a two digit TZ code is standard. just tell the engineers to send a proper strftime variable, then splunk would automatically parse it. use either %z or %Z:
https://docs.python.org/2/library/datetime.html#strftime-and-strptime-behavior
http://docs.splunk.com/Documentation/Splunk/6.2.3/Data/Configuretimestamprecognition
http://docs.splunk.com/Documentation/Splunk/6.2.3/data/Applytimezoneoffsetstotimestamps