Hello everyone,
I just want to use append instead of a join.
My code is
index="yut" sourcetype="test" cd IN(*) level="rrr" severity="error" | eval evtdate = strftime(_time,"%Y-%m-%d")
| eval CD=cd | eval key=_time + url | table key ------(this query is giveing me 504 key)
| append ----------------( want to use here append)
[ search index="yut" sourcetype="test" cd IN (*) severity="error" | eval CD=cd
| eval evtdatein = strftime(_time,"%Y-%m-%d")
| sort by _time desc
| dedup url | eval newkey=_time + url
| table newkey] --------------(this query is giving me 6 newkeys)
| eventstats values(newkey) as UniqueKeys | search key IN UniqueKeys
want to add all 504 +6 = 510 key and remove the duplicate key from the 504
Hi @hrs2019,
I don't understand your code because the main and secondary searches are the same and the only difference is the dedup in the secondary, so if you want to dedup the 504 keys you can do it in the main search
index="yut" sourcetype="test" cd=* level="rrr" severity="error"
| dedup url
| eval newkey=_time + url
| table key
Then there are some eval transformations that you don't use (e.g. CD=cd or evtdate).
Anyway, append (like join) has the problem that, using a subsearch, that's the limit of 50,000 results (in subsearch), so why don't you think to put the second search in the main search with OR operator and then tag the values of the first search using eval (e.g. if there's a value that identifies the results of the first search)?
You have only to adapt the other data elaboration.
Could you describe better your need?
Ciao.
Giuseppe
thanks @gcusello
Actually, my requirement is to get these 510 events where the key matches the new key.
Also to get 510 events where the key-value matches the new key generated
in subquery where only generates a list of latest url_time combination as a new key
index="yut" sourcetype="test" cd IN(*) level="rrr" severity="error"
| eval evtdate = strftime(_time,"%Y-%m-%d")
| eval CD=cd
| eval key=_time + url (this query is giveing me 504 key)
| append ----------------( want to use here append)
[ search index="yut" sourcetype="test" cd IN (*) severity="error" | eval CD=cd
| eval evtdatein = strftime(_time,"%Y-%m-%d")
| sort by _time desc
| dedup url | eval newkey=_time + url
| table newkey] --------------(this query is giving me 6 newkeys)
Hi @hrs2019,
let me understand: you want to find keys in the last period that were present also before?
I continue to see that the searches are the same, so results are the same, if you speak of latest url _time combination you have to insert a different time frame in the subsearch (something like earliest=... latest=...)
If this is you need, I answered to a similar question few days ago ( https://answers.splunk.com/answers/823555/how-to-search-data-which-is-not-present-in-the-las.html#an... ).
So you could try something like this (e.g. last period is the last 24 hours and the full period is 7 days):
index="yut" sourcetype="test" cd=* level="rrr" severity="error"
| eval new=if(now()-_time<86400,"yes","not")
| eval key=_time + url
| stats dc(new) AS count values(new) AS new values(key) AS key
| where count>1 AND new="yes"
| table key
Ciao.
Giuseppe