https://community.splunk.com/t5/Splunk-Search/How-to-avoid-displaying-duplicate-logs-in-search-results/m-p/257670#M77154 <P>I am trying to search /var/log/messages log with keywords like shutdown or Error and storing it in message.log</P> <P>and doing index on message.log</P> <P>I have a log around at 5 mins ago like 11:30 AM EST says system is shutdown and I displayed this error log in a table format.</P> <P>Around at 11:40 AM EST, when I ran search command in a table format, I see SYSTEM is Shutdown, but we already knew that system is down at 11:30 AM EST and now I don't want to see this message again.</P> <P>How can we achieve this?</P> Mon, 17 Oct 2016 17:01:38 GMT rajgowd1 2016-10-17T17:01:38Z https://community.splunk.com/t5/Splunk-Search/How-to-avoid-displaying-duplicate-logs-in-search-results/m-p/257670#M77154 <P>I am trying to search /var/log/messages log with keywords like shutdown or Error and storing it in message.log</P> <P>and doing index on message.log</P> <P>I have a log around at 5 mins ago like 11:30 AM EST says system is shutdown and I displayed this error log in a table format.</P> <P>Around at 11:40 AM EST, when I ran search command in a table format, I see SYSTEM is Shutdown, but we already knew that system is down at 11:30 AM EST and now I don't want to see this message again.</P> <P>How can we achieve this?</P> Mon, 17 Oct 2016 17:01:38 GMT https://community.splunk.com/t5/Splunk-Search/How-to-avoid-displaying-duplicate-logs-in-search-results/m-p/257670#M77154 rajgowd1 2016-10-17T17:01:38Z https://community.splunk.com/t5/Splunk-Search/How-to-avoid-displaying-duplicate-logs-in-search-results/m-p/257671#M77155 <P>rajgowd1,</P> <P>I'd like more information or perhaps a screenshot to help you to remove the duplicate logs. It sounds like you just need to change your search time window.</P> <P>Adam</P> Mon, 17 Oct 2016 17:11:02 GMT https://community.splunk.com/t5/Splunk-Search/How-to-avoid-displaying-duplicate-logs-in-search-results/m-p/257671#M77155 adamsaul 2016-10-17T17:11:02Z https://community.splunk.com/t5/Splunk-Search/How-to-avoid-displaying-duplicate-logs-in-search-results/m-p/257672#M77156 <P>You can use timemodifiers to limit your logs to a specific timeframe. So for example, if you run you search every 5 mins, you can limit your search to prev 5 mins data using <CODE>earliest=-5M@m</CODE> in your base search.</P> <P><A href="https://docs.splunk.com/Documentation/Splunk/6.5.0/SearchReference/SearchTimeModifiers">https://docs.splunk.com/Documentation/Splunk/6.5.0/SearchReference/SearchTimeModifiers</A></P> Mon, 17 Oct 2016 17:36:24 GMT https://community.splunk.com/t5/Splunk-Search/How-to-avoid-displaying-duplicate-logs-in-search-results/m-p/257672#M77156 sundareshr 2016-10-17T17:36:24Z https://community.splunk.com/t5/Splunk-Search/How-to-avoid-displaying-duplicate-logs-in-search-results/m-p/257673#M77157 <P>It sounds like you should be extracting these fields at search time and using dedup, but it is difficult to be sure without seeing your search. Can you post your search?</P> Mon, 17 Oct 2016 17:57:34 GMT https://community.splunk.com/t5/Splunk-Search/How-to-avoid-displaying-duplicate-logs-in-search-results/m-p/257673#M77157 lukejadamec 2016-10-17T17:57:34Z https://community.splunk.com/t5/Splunk-Search/How-to-avoid-displaying-duplicate-logs-in-search-results/m-p/257674#M77158 <P>I agree with the others that the best way is to limit your timerange.<BR /> Every way you could extract a field with error and dedup using this field and host.</P> <PRE><CODE>your search | dedup host error | table ... </CODE></PRE> <P>Bye,<BR /> Giuseppe</P> Tue, 18 Oct 2016 08:07:09 GMT https://community.splunk.com/t5/Splunk-Search/How-to-avoid-displaying-duplicate-logs-in-search-results/m-p/257674#M77158 gcusello 2016-10-18T08:07:09Z