Can anyone regex the time out in my search?

[moiezuddin]

In the search below, can anyone regex the time out instead of bucket span?

I need to figure out a way to filter time and data per user. (Need to use regex instead of bucket)

The expression which is there in the search is extracts the USERID field. ( | rex "(?i) Realm][][][][][][(?P[^]]+" )

Just want regex the time out instead of bucket span

index=casm_prod sourcetype=smtrace "Center realm" | rex "(?i) Realm][][][][][][(?P[^]]+" |bucket span=5m _time|stats values(user) as USER count(user) as eventcount by _time |fields USER eventcount

[esix_splunk · Accepted solution]

Expanding on that a bit, Im not sure if the start of the event is actually the "["...

... | rex field=_raw "\[(?P<date>\d{1,2}\/\d{1,2}\/\d{4})\]\[(?P<time>\d{1,2}:\d{1,2}:\d{1,2}.\d{1,3})\]" | table date time

[moiezuddin]

Now i can able to be found many times userid's each minute.

Can we fiter this so it only shows once per minute.

[esix_splunk]

... | rex field=_raw "\[(?P<date>\d{1,2}\/\d{1,2}\/\d{4})\]\[(?P<time>\d{1,2}:\d{1,2}:\d{1,2}.\d{1,3})\]" | bin span=1m _time | stats count by USER

But you need to extract your 'USER' if it isnt already.

[moiezuddin]

Yes your correct i extracted it , here is the query

index=casm_prod sourcetype=smtrace "Center realm"| rex "(?i) Realm\]\[\]\[\]\[\]\[\]\[\]\[(?P[^\]]+)"| rex field=_raw "[(?P\d{1,2}\/\d{1,2}\/\d{4})][(?P\d{1,2}:\d{1,2}:\d{1,2}.\d{1,3})]" | table date time user

Iam getting multiple events form one usersid
Without using bin span

can we get event per user per minute

[esix_splunk]

index=casm_prod sourcetype=smtrace "Center Realm" | rex field=_raw "\[(?P<date>\d{1,2}\/\d{1,2}\/\d{4})\]\[(?P<time>\d{1,2}:\d{1,2}:\d{1,2}.\d{1,3})\].*Center realm\](\[\]+){5}\[(?<user_id>[^\]]+)\]" | bin span=1m time | stats count by user_id

If you want to aggregate by time, you have to use stats or timechart for this. And since you say you want to aggregate over one minute, you have to user the bin span=1m. You can remove that, but it will group by all time in the events.

I also have to question if your time stamps are working correctly for these events...

[moiezuddin]

Error in the query

Time stamps is working correctly for your 1st query which you given''

[esix_splunk]

Timestamp should be working from index time, not search time. So currently your event time doesnt match the actual event time?

You should fix this, otherwise Splunk looses it real value.

Read this : http://docs.splunk.com/Documentation/Splunk/6.2.2/Data/HowSplunkextractstimestamps

You shouldnt have to extract date or timestamps at search time, these should be done at index time and available in search. This negates the whole purpose of Splunk and time based event monitoring.

Whats the query error?

[moiezuddin]

Thankyou very much for your quick response ..

its working fine now.

[aholzel]

This regex will give you the date and time in different fields:

rex field=_raw "^\[(?<date>[^\]]+)\]\[(?<time>[^\]]+)"

Edit: the naming did not work... please remove the space between the "<" and "date" and between "<" and "time"

[moiezuddin]

I need to figure out only userid which shows once per minute with regex,
Can you please provide me the complete query , i am unable to understand where to add in existing query.
Awaiting for your response

[aholzel]

I don't really understand what you mean. Are you looking for a specific username that only shows once a minute? or do you get multiple events from users and only what to work with one event per user per minute?

[moiezuddin]

i am getting multiple events form one usersid
I need one event per user per minute

[esix_splunk]

We need to know what your data looks like in order to extract your time.

[moiezuddin]

SAMPLE DATA FROM THE ABOVE QUERY

4/28/15
2:05:00.000 PM

[04/28/2015][11:06:09.634][347216816][s5324172/r23][Center realm][][][][][][206416426][][][][][][centerusushwswp222lprd][** Status: Authorized. ][]
index = casm_prod
4/28/15
2:05:00.000 PM

[04/28/2015][11:06:09.634][347216816][s5324172/r23][Center realm][][][][][][206416426][][][][][][centerusushwswp222lprd][Send response attribute 147, data size is 0][]
index = casm_prod
4/28/15
2:05:00.000 PM

[04/28/2015][11:06:09.634][347216816][s5324172/r23][Center realm][][][][][][206416426][][][][][][centerusushwswp222lprd][Send response attribute 146, data size is 0][]
index = casm_prod
4/28/15
2:05:00.000 PM

[04/28/2015][11:06:09.634][347216816][s5324172/r23][Center realm][][][][][][206416426][][][][][][centerusushwswp222lprd][Send response attribute 224, data size is 16][sso_id=206416426]
index = casm_prod sso_id = 206416426
4/28/15
2:05:00.000 PM

[04/28/2015][11:06:09.634][347216816][s5324172/r23][Center realm][][][][][][206416426][][][][][][centerusushwswp222lprd][Send response attribute 224, data size is 16][smuser=206416426]