Security

loop search?

crazyeva
Contributor

A field: a=1,2,3,4..... disordered
i need a search like:
a=1
| append [search a=2]
| append [search a=3]
| append [search a=4]
....
where i can not simpley use "sort" ...
is there any syntax like "if a<n a++ and search something...." in splunk?
by the way, is there better way than " ...| head 88 | tail 1" when i want the 88th event?
thank you

[EDIT]
OK! this is what i am after:
Puting events in the right order, for example a business procedur
acc=crazyeva (1a)
id=0001 (1b)
tim=20121009 (1c)
act=toopooltopurch (1d)

but "_raw" data is disordered by "_time":
# _raw
16 11:48 acc=1a
15 11:49 id=1b
14 11:50 acc=2a
13 11:51 tim=1c
12 11:52 acc=3a
11 11:53 act=1d
10 11:54 id=2b
9 11:55 id=3b
8 11:56 tim=2c
7 11:57 acc=4a
6 11:58 tim=3c
5 11:59 act=2d
4 11:60 id=4b
3 11:61 act=3d
2 11:62 tim=4c
1 11:63 act=4d
....
The only rule is in the time line:
1b never comes before 1a, the same manner, 1a >> 1b >> 1c >>1d, 2a >> 2b....;
2a never comes before 1a, the same manner, 1a >> 2a >> 3a >>4a, 1b >> 2b....

this is my solution:
acc | sort _time | head 1 | tail 1
| append [search id | sort _time | head 1 | tail 1]
| append [search tim | sort _time | head 1 | tail 1]
| append [search act | sort _time | head 1 | tail 1]
| append [search acc | sort _time | head 2 | tail 1]
| append [search id | sort _time | head 2 | tail 1]
| append [search tim | sort _time | head 2 | tail 1]
| append [search act | sort _time | head 2 | tail 1]
| append [search acc | sort _time | head 3 | tail 1]
| append [search id | sort _time | head 3 | tail 1]
| append [search tim | sort _time | head 3 | tail 1]
| append [search act | sort _time | head 3 | tail 1]
| append [search acc | sort _time | head 4 | tail 1]
| append [search id | sort _time | head 4 | tail 1]
| append [search tim | sort _time | head 4 | tail 1]
| append [search act | sort _time | head 4 | tail 1]
.........
| streamstats count | eval _time=count | sort _time
| transaction maxspan=4s
two problem:
1.I need to do a "loop search" if there are too many events
2."transaction" command does not work on written "_time"

a second way:
* | sort _time | stats list(acc)
| appendcols [search * | sort _time | stats list(id) ]
| appendcols [search * | sort _time | stats list(tim) ]
| appendcols [search * | sort _time | stats list(act) ]
| table list(acc) list(id) list(tim) list(act)
the result seems like a table, but its not a useful table at all
Could you help me to put them in order?

Tags (1)
0 Karma
1 Solution

lguinn2
Legend

This may work, but I am not sure that it will work in all cases. Try it with a few different time ranges...

<yoursearchhere> 
| rex "\s(?<fieldname>\S+?)="
| streamstats count as IdNum by fieldname
| transaction IdNum startswith=eval(fieldname="acc") endswith=eval(fieldname="act") keepevicted=true
| sort _time

You might want to remove keepevicted=true if you don't want partial transactions. And this may not work if you have a large number of transactions. But try it.

View solution in original post

lguinn2
Legend

This may work, but I am not sure that it will work in all cases. Try it with a few different time ranges...

<yoursearchhere> 
| rex "\s(?<fieldname>\S+?)="
| streamstats count as IdNum by fieldname
| transaction IdNum startswith=eval(fieldname="acc") endswith=eval(fieldname="act") keepevicted=true
| sort _time

You might want to remove keepevicted=true if you don't want partial transactions. And this may not work if you have a large number of transactions. But try it.

crazyeva
Contributor

Thank you
I think this is much better an answer

0 Karma

jonuwz
Influencer

You need to show what the raw data look like in splunk, not how it was logged.

Its a simple matter of extracting the fields, sorting on the 1a/1b/2a/2b field and including the time.

i.e. if your raw data looks like :

10 Oct 2012 host=wibble;tim=1a

You set up an extract like this :

<search> 
| rex ";(?<type>.*?)=(?<number>[0-9]+)(?<letter>[a-z]+)"
| sort number letter
| eval precedence=number.letter
| table _time type precedence
0 Karma

crazyeva
Contributor

Thank you
but i mean there is not any fixed form in the right part of "=". 1a,2b...are just examples. they could be any content.

0 Karma

crazyeva
Contributor

Sorry to make you baffled
I have posted below some raw data, and the results i expected

0 Karma

lguinn2
Legend

My suggestion is different - create two fields, one for each part of the "id" (1a, 2b, etc) and then sort them like this:

yoursearchere
| rex "=(?<sortField1>\d+)(?<sortField2>\S+)"
| sort sortField1 sortField2

and perhaps even

yoursearchere
| rex "=(?<sortField1>\d+)(?<sortField2>\S+)"
| sort sortField1 sortField2
| transaction sortField1

But if none of our answers are getting close, I suggest that you post a sample the actual raw data, and just change some names. If you are changing the format of the events in your examples, then the answers given are probably not going to be very useful.

0 Karma

crazyeva
Contributor

1a, 2b, such fields after "=" may be any content
do not try sort them by 1,2,3,4 or a,b,c,d...

11:48 acc=nice
11:49 id=to
11:50 acc=i
11:51 tim=meet
11:52 acc=thank
11:53 act=you
11:54 id=tried
11:55 id=you
11:56 tim=my
11:57 acc=what
11:58 tim=very
11:59 act=best
12:00 id=a
12:01 act=much
12:02 tim=nice
12:03 act=day
\

acc id tim act

nice to meet you
i tried my best
thank you very much
what a nice day

Thank you~

0 Karma

crazyeva
Contributor

Sorry I am not good at describing
There is same of the raw data:

11:48 acc=1a
11:49 id=1b
11:50 acc=2a
11:51 tim=1c
11:52 acc=3a
11:53 act=1d
11:54 id=2b
11:55 id=3b
11:56 tim=2c
11:57 acc=4a
11:58 tim=3c
11:59 act=2d
12:00 id=4b
12:01 act=3d
12:02 tim=4c
12:03 act=4d
.....

I want results like:
11:48 acc=1a
11:49 id=1b
11:51 tim=1c

11:53 act=1d

11:50 acc=2a
11:54 id=2b
11:56 tim=2c

11:59 act=2d

11:52 acc=3a
11:55 id=3b
11:58 tim=3c

12:01 act=3d

11:57 acc=4a
12:00 id=4b
12:02 tim=4c

10:03 act=4d

......

OR:
acc id tim act
1a 1b 1c 1d
2a 2b 2c 2d
3a 3b 3c 3d
4a 4b 4c 4d
......

0 Karma

sideview
SplunkTrust
SplunkTrust

I also am baffled as to what you're trying to do. I suspect that some combination of sort/rex/eval/stats can absolutely sort it the way that you need. Can you back up and describe the desired sort order more clearly?

0 Karma

crazyeva
Contributor

I have posted my real stanza, It is a "flow" data

0 Karma

dart
Splunk Employee
Splunk Employee

Does a=1 OR a=2 OR a=3 OR a=4 not return the results you need?

0 Karma

crazyeva
Contributor

Thank you, but my demands is not that simple

0 Karma

Ayn
Legend

I think you need to state your problem a bit more clearly. I tried to understand what you're after but failed. What do you want to sort, why, how?

Get Updates on the Splunk Community!

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...

What's new in Splunk Cloud Platform 9.1.2312?

Hi Splunky people! We are excited to share the newest updates in Splunk Cloud Platform 9.1.2312! Analysts can ...

What’s New in Splunk Security Essentials 3.8.0?

Splunk Security Essentials (SSE) is an app that can amplify the power of your existing Splunk Cloud Platform, ...