topic Radius Accounting Data Timestamping events in Splunk Search

Radius Accounting Data Timestamping events

anthonysomerset — Thu, 30 Jun 2016 10:33:19 GMT

I have some RADIUS accounting logs that i send into splunk and this is my current props.conf

[radius_acct_detail]
SHOULD_LINEMERGE = true
BREAK_ONLY_BEFORE = ^\w{3} \w{3} \d\d \d\d:\d\d:\d\d \d\d\d\d$
TIME_PREFIX = Event-Timestamp = "
TIME_FORMAT = %b %d %Y %H:%M:%S %z
MAX_TIMESTAMP_LOOKAHEAD = -1

Now most but not all radius accounting events have the Event-Timestamp field in them, for example:

Thu Jun 30 04:24:27 2016
    Acct-Status-Type = Accounting-Off
    Acct-Authentic = RADIUS
    User-Name = "XX-XX-XX-XX-XX-XX"
    NAS-IP-Address = 172.20.dd.ddd
    NAS-Identifier = "XX-XX-XX-XX-XX-XX"
    Called-Station-Id = "XX-XX-XX-XX-XX-XX:ZOLspot"
    Ruckus-SSID = "ZOLspot"
    Ruckus-Attr-14 = 0xXXXXXXXXXXXX
    Ruckus-WlanID = 362
    Ruckus-Attr-9 = 0xXXXXXXXX
    Ruckus-SCG-CBlade-IP = XXXXXXXXXX
    Ruckus-SCG-DBlade-IP = XXXXXXXXX
    Proxy-State = 0x31
    Acct-Unique-Session-Id = "6716f5d57c72963e"
    Realm = "NULL"
    Timestamp = 1467253467

as a result i get various regular parsing timestamp errors in the splunk log that i'd like to clean up. ideally what i'd want to happen in this case is for it to fall back to the Timestamp field (epoch time) which is present in every event or the first line of the event which is the time the packet is logged by the radius server (these are the same in the case of these packets but the Event-Timestamp field can and does in cases differ from this timestamp

reading the other answer it seems i could do some greedy regex to capture the first timestamp field if the Event-Timestamp field is not present but the format of the field is different:

    Event-Timestamp = "Jun 30 2016 08:29:11 CAT"

specifically day is omitted and TZ is added and Year is in a different spot. anyone have any thoughts on some regex foo that might work or should we somehow look down the custom datetime.xml route?

Re: Radius Accounting Data Timestamping events

anthonysomerset — Thu, 30 Jun 2016 13:53:03 GMT

So i ended up cheating on this with something thats probably not the best approach but works for me

Rather than try to solve the problem in splunk i looked at solving this at the source, and turns out with a little bit of freeradius logic i can solve this.

I defined the following function that i call within the accounting section of freeradius

append_timestamp_to_acct {
                if (Acct-Status-Type == "Accounting-On" || Acct-Status-Type == "Accounting-Off") {
                    if (!Event-Timestamp) {
                        update request {
                                Event-Timestamp = "%l"
                        }
                    }
                }
        }

what this function does is set the Event-Timestamp attribute if its not present and this is an Accounting-On or Accounting-Off request

it sets it to the current timestamp (which happens to be the correct format for the Event-Timestamp attribute to start with) which then gets matched in my event in splunk - %l is a freeradius runtime variable for timestamp - http://wiki.freeradius.org/config/run_time_variables

I probbaly don't need the acct-status-type as the check if Event-Timestamp is not present (or empty string) would probably suffice on its own, just being explicitly careful as these are the only packets i've found where the attribute is not set so far

Re: Radius Accounting Data Timestamping events

woodcock — Thu, 30 Jun 2016 14:25:55 GMT

When dealing with multiple time formats (not positions), you really have no choice but to use datetime.xml:

http://www.function1.com/2013/01/oh-no-splunking-log-files-with-multiple-formats-no-problem

Re: Radius Accounting Data Timestamping events

anthonysomerset — Thu, 30 Jun 2016 14:28:44 GMT

The question is, does custom datetime.xml accept first rule match? the default timestamp exists for every entry but if the Event-Timestamp exists i want to prefer it over the default timestamp

Re: Radius Accounting Data Timestamping events

woodcock — Thu, 30 Jun 2016 14:48:26 GMT

As far as preferential field selection for time, see this:

https://answers.splunk.com/answers/168995/how-to-write-regex-to-identify-and-use-time-field.html