want to use append instead of join

[hrs2019]

Hello everyone,

I just want to use append instead of a join.

My code is

index="yut" sourcetype="test"  cd IN(*)  level="rrr" severity="error" | eval evtdate = strftime(_time,"%Y-%m-%d")
| eval CD=cd | eval key=_time + url | table key ------(this query is giveing me 504 key)
| append ----------------( want to use here append)
    [ search index="yut" sourcetype="test" cd IN (*) severity="error" | eval  CD=cd 
    | eval evtdatein = strftime(_time,"%Y-%m-%d")
    | sort by _time desc
    | dedup url | eval newkey=_time + url
    | table newkey]  --------------(this query is giving me 6 newkeys)
| eventstats values(newkey) as UniqueKeys | search key IN UniqueKeys

want to add all 504 +6 = 510 key and remove the duplicate key from the 504

[gcusello]

Hi @hrs2019,
I don't understand your code because the main and secondary searches are the same and the only difference is the dedup in the secondary, so if you want to dedup the 504 keys you can do it in the main search

index="yut" sourcetype="test"  cd=*  level="rrr" severity="error"
| dedup url
| eval newkey=_time + url
| table key

Then there are some eval transformations that you don't use (e.g. CD=cd or evtdate).

Anyway, append (like join) has the problem that, using a subsearch, that's the limit of 50,000 results (in subsearch), so why don't you think to put the second search in the main search with OR operator and then tag the values of the first search using eval (e.g. if there's a value that identifies the results of the first search)?
You have only to adapt the other data elaboration.

Could you describe better your need?

Ciao.
Giuseppe

[hrs2019]

thanks @gcusello

Actually, my requirement is to get these 510 events where the key matches the new key.
Also to get 510 events where the key-value matches the new key generated
in subquery where only generates a list of latest url_time combination as a new key

index="yut" sourcetype="test"  cd IN(*)  level="rrr" severity="error" 
| eval evtdate = strftime(_time,"%Y-%m-%d")
| eval CD=cd 
| eval key=_time + url (this query is giveing me 504 key)
| append ----------------( want to use here append)
[ search index="yut" sourcetype="test" cd IN (*) severity="error" | eval  CD=cd 
| eval evtdatein = strftime(_time,"%Y-%m-%d")
| sort by _time desc
| dedup url | eval newkey=_time + url
| table newkey]  --------------(this query is giving me 6 newkeys)

[gcusello]

Hi @hrs2019,
let me understand: you want to find keys in the last period that were present also before?

I continue to see that the searches are the same, so results are the same, if you speak of latest url _time combination you have to insert a different time frame in the subsearch (something like earliest=... latest=...)

If this is you need, I answered to a similar question few days ago ( https://answers.splunk.com/answers/823555/how-to-search-data-which-is-not-present-in-the-las.html#an... ).

So you could try something like this (e.g. last period is the last 24 hours and the full period is 7 days):

 index="yut" sourcetype="test"  cd=*  level="rrr" severity="error"
 | eval new=if(now()-_time<86400,"yes","not")
 | eval key=_time + url
 | stats dc(new) AS count values(new) AS new values(key) AS key
 | where count>1 AND new="yes"
 | table key

Ciao.
Giuseppe