topic Re: Can anyone regex the time out in my search? in Splunk Search

Can anyone regex the time out in my search?

moiezuddin — Tue, 28 Apr 2015 15:41:52 GMT

In the search below, can anyone regex the time out instead of bucket span?

I need to figure out a way to filter time and data per user. (Need to use regex instead of bucket)

The expression which is there in the search is extracts the USERID field. ( | rex "(?i) Realm][][][][][][(?P[^]]+" )

Just want regex the time out instead of bucket span

index=casm_prod sourcetype=smtrace "Center realm" | rex "(?i) Realm][][][][][][(?P[^]]+" |bucket span=5m _time|stats values(user) as USER count(user) as eventcount by _time |fields USER eventcount

Re: Can anyone regex the time out in my search?

esix_splunk — Wed, 29 Apr 2015 06:33:01 GMT

We need to know what your data looks like in order to extract your time.

Re: Can anyone regex the time out in my search?

moiezuddin — Mon, 28 Sep 2020 19:39:00 GMT

SAMPLE DATA FROM THE ABOVE QUERY

4/28/15
2:05:00.000 PM

[04/28/2015][11:06:09.634][347216816][s5324172/r23][Center realm][][][][][][206416426][][][][][][centerusushwswp222lprd][** Status: Authorized. ][]
index = casm_prod
4/28/15
2:05:00.000 PM

[04/28/2015][11:06:09.634][347216816][s5324172/r23][Center realm][][][][][][206416426][][][][][][centerusushwswp222lprd][Send response attribute 147, data size is 0][]
index = casm_prod
4/28/15
2:05:00.000 PM

[04/28/2015][11:06:09.634][347216816][s5324172/r23][Center realm][][][][][][206416426][][][][][][centerusushwswp222lprd][Send response attribute 146, data size is 0][]
index = casm_prod
4/28/15
2:05:00.000 PM

[04/28/2015][11:06:09.634][347216816][s5324172/r23][Center realm][][][][][][206416426][][][][][][centerusushwswp222lprd][Send response attribute 224, data size is 16][sso_id=206416426]
index = casm_prod sso_id = 206416426
4/28/15
2:05:00.000 PM

[04/28/2015][11:06:09.634][347216816][s5324172/r23][Center realm][][][][][][206416426][][][][][][centerusushwswp222lprd][Send response attribute 224, data size is 16][smuser=206416426]

Re: Can anyone regex the time out in my search?

aholzel — Wed, 29 Apr 2015 06:55:22 GMT

This regex will give you the date and time in different fields:

rex field=_raw "^\[(?<date>[^\]]+)\]\[(?<time>[^\]]+)"

Edit: the naming did not work... please remove the space between the "<" and "date" and between "<" and "time"

Re: Can anyone regex the time out in my search?

moiezuddin — Wed, 29 Apr 2015 06:58:26 GMT

I need to figure out only userid which shows once per minute with regex,
Can you please provide me the complete query , i am unable to understand where to add in existing query.
Awaiting for your response

Re: Can anyone regex the time out in my search?

esix_splunk — Wed, 29 Apr 2015 06:59:43 GMT

Expanding on that a bit, Im not sure if the start of the event is actually the "["...

... | rex field=_raw "\[(?P<date>\d{1,2}\/\d{1,2}\/\d{4})\]\[(?P<time>\d{1,2}:\d{1,2}:\d{1,2}.\d{1,3})\]" | table date time

Re: Can anyone regex the time out in my search?

aholzel — Wed, 29 Apr 2015 07:01:04 GMT

I don't really understand what you mean. Are you looking for a specific username that only shows once a minute? or do you get multiple events from users and only what to work with one event per user per minute?

Re: Can anyone regex the time out in my search?

moiezuddin — Wed, 29 Apr 2015 07:04:43 GMT

Now i can able to be found many times userid's each minute.

Can we fiter this so it only shows once per minute.

Re: Can anyone regex the time out in my search?

esix_splunk — Wed, 29 Apr 2015 07:06:38 GMT

... | rex field=_raw "\[(?P<date>\d{1,2}\/\d{1,2}\/\d{4})\]\[(?P<time>\d{1,2}:\d{1,2}:\d{1,2}.\d{1,3})\]" | bin span=1m _time | stats count by USER

But you need to extract your 'USER' if it isnt already.

Re: Can anyone regex the time out in my search?

moiezuddin — Wed, 29 Apr 2015 07:10:04 GMT

i am getting multiple events form one usersid
I need one event per user per minute

Re: Can anyone regex the time out in my search?

moiezuddin — Mon, 28 Sep 2020 19:42:39 GMT

Yes your correct i extracted it , here is the query

index=casm_prod sourcetype=smtrace "Center realm"| rex "(?i) Realm\]\[\]\[\]\[\]\[\]\[\]\[(?P[^\]]+)"| rex field=_raw "[(?P\d{1,2}\/\d{1,2}\/\d{4})][(?P\d{1,2}:\d{1,2}:\d{1,2}.\d{1,3})]" | table date time user

Iam getting multiple events form one usersid
Without using bin span

can we get event per user per minute

Re: Can anyone regex the time out in my search?

esix_splunk — Wed, 29 Apr 2015 07:20:36 GMT

index=casm_prod sourcetype=smtrace "Center Realm" | rex field=_raw "\[(?P<date>\d{1,2}\/\d{1,2}\/\d{4})\]\[(?P<time>\d{1,2}:\d{1,2}:\d{1,2}.\d{1,3})\].*Center realm\](\[\]+){5}\[(?<user_id>[^\]]+)\]" | bin span=1m time | stats count by user_id

If you want to aggregate by time, you have to use stats or timechart for this. And since you say you want to aggregate over one minute, you have to user the bin span=1m. You can remove that, but it will group by all time in the events.

I also have to question if your time stamps are working correctly for these events...

Re: Can anyone regex the time out in my search?

moiezuddin — Wed, 29 Apr 2015 07:25:04 GMT

Error in the query

Time stamps is working correctly for your 1st query which you given''

Re: Can anyone regex the time out in my search?

esix_splunk — Wed, 29 Apr 2015 07:30:20 GMT

Timestamp should be working from index time, not search time. So currently your event time doesnt match the actual event time?

You should fix this, otherwise Splunk looses it real value.

Read this : http://docs.splunk.com/Documentation/Splunk/6.2.2/Data/HowSplunkextractstimestamps

You shouldnt have to extract date or timestamps at search time, these should be done at index time and available in search. This negates the whole purpose of Splunk and time based event monitoring.

Whats the query error?

Re: Can anyone regex the time out in my search?

moiezuddin — Wed, 29 Apr 2015 07:37:20 GMT

Thankyou very much for your quick response ..

its working fine now.