Splunk Search

want to use append instead of join

hrs2019
Path Finder

Hello everyone,

I just want to use append instead of a join.

My code is

index="yut" sourcetype="test"  cd IN(*)  level="rrr" severity="error" | eval evtdate = strftime(_time,"%Y-%m-%d")
| eval CD=cd | eval key=_time + url | table key ------(this query is giveing me 504 key)
| append ----------------( want to use here append)
    [ search index="yut" sourcetype="test" cd IN (*) severity="error" | eval  CD=cd 
    | eval evtdatein = strftime(_time,"%Y-%m-%d")
    | sort by _time desc
    | dedup url | eval newkey=_time + url
    | table newkey]  --------------(this query is giving me 6 newkeys)
| eventstats values(newkey) as UniqueKeys | search key IN UniqueKeys

want to add all 504 +6 = 510 key and remove the duplicate key from the 504

0 Karma

gcusello
Legend

Hi @hrs2019,
I don't understand your code because the main and secondary searches are the same and the only difference is the dedup in the secondary, so if you want to dedup the 504 keys you can do it in the main search

index="yut" sourcetype="test"  cd=*  level="rrr" severity="error"
| dedup url
| eval newkey=_time + url
| table key

Then there are some eval transformations that you don't use (e.g. CD=cd or evtdate).

Anyway, append (like join) has the problem that, using a subsearch, that's the limit of 50,000 results (in subsearch), so why don't you think to put the second search in the main search with OR operator and then tag the values of the first search using eval (e.g. if there's a value that identifies the results of the first search)?
You have only to adapt the other data elaboration.

Could you describe better your need?

Ciao.
Giuseppe

0 Karma

hrs2019
Path Finder

thanks @gcusello

Actually, my requirement is to get these 510 events where the key matches the new key.
Also to get 510 events where the key-value matches the new key generated
in subquery where only generates a list of latest url_time combination as a new key

index="yut" sourcetype="test"  cd IN(*)  level="rrr" severity="error" 
| eval evtdate = strftime(_time,"%Y-%m-%d")
| eval CD=cd 
| eval key=_time + url (this query is giveing me 504 key)
| append ----------------( want to use here append)
[ search index="yut" sourcetype="test" cd IN (*) severity="error" | eval  CD=cd 
| eval evtdatein = strftime(_time,"%Y-%m-%d")
| sort by _time desc
| dedup url | eval newkey=_time + url
| table newkey]  --------------(this query is giving me 6 newkeys)
0 Karma

gcusello
Legend

Hi @hrs2019,
let me understand: you want to find keys in the last period that were present also before?

I continue to see that the searches are the same, so results are the same, if you speak of latest url _time combination you have to insert a different time frame in the subsearch (something like earliest=... latest=...)

If this is you need, I answered to a similar question few days ago ( https://answers.splunk.com/answers/823555/how-to-search-data-which-is-not-present-in-the-las.html#an... ).

So you could try something like this (e.g. last period is the last 24 hours and the full period is 7 days):

 index="yut" sourcetype="test"  cd=*  level="rrr" severity="error"
 | eval new=if(now()-_time<86400,"yes","not")
 | eval key=_time + url
 | stats dc(new) AS count values(new) AS new values(key) AS key
 | where count>1 AND new="yes"
 | table key

Ciao.
Giuseppe

0 Karma
Get Updates on the Splunk Community!

Improve Your Security Posture

Watch NowImprove Your Security PostureCustomers are at the center of everything we do at Splunk and security ...

Maximize the Value from Microsoft Defender with Splunk

 Watch NowJoin Splunk and Sens Consulting for this Security Edition Tech TalkWho should attend:  Security ...

This Week's Community Digest - Splunk Community Happenings [6.27.22]

Get the latest news and updates from the Splunk Community here! News From Splunk Answers ✍️ Splunk Answers is ...