topic Re: Is it possible to dedup by span? in Splunk Search

Is it possible to dedup by span?

the_wolverine — Fri, 08 Jun 2012 18:44:17 GMT

I'm try to chart some data using span=1d and was wondering if it possible to dedup data across a timerange with span?

For example, I want to dedup duplicate users in a single day, but I also want those users to show up in previous days when I'm charting over a week.

I'm guessing * | dedup user | timechart span=7d .. would eliminate users from showing up in day 2-7.

I hope that makes sense.

Re: Is it possible to dedup by span?

kristian_kolb — Fri, 08 Jun 2012 19:00:30 GMT

Makes sense. Depending on what you ultimately want out of the logs, something like this could work;

...| stats values(user) by date_wday

or date_mday if that suits you better.

UPDATE:

or rather use;

... earliest=-X latest=-Y | timechart span=1d values(user)

Hope this helps,

Kristian

Re: Is it possible to dedup by span?

sideview — Fri, 08 Jun 2012 22:18:39 GMT

* | timechart dc(user) span=7d

where dc means "distinct count of".

This will make timechart count the distinct users per bucket, and since the span argument is setting the bucket size to 7 days, in the end you'll be counting the distinct users in every 7 day period.

Re: Is it possible to dedup by span?

the_wolverine — Sat, 09 Jun 2012 07:16:49 GMT

I'll test this, thank you. Is there a way to chart top(field) limit=X with using a span?

Re: Is it possible to dedup by span?

sideview — Sat, 09 Jun 2012 18:52:58 GMT

Probably, but you'll have to tell me more because that pseudo-search-syntax is pretty ambiguous. To take a wild guess and at least tell you something interesting -- you can use the bin command to bucket numeric quantities, and then use stats/chart/timechart to group by those bucketed values. ie "* | bin someNumericField span=100 | stats count over someNumericField" will yield a nice chart with "0-100", "100-200", "200-300" as the x-axis.