Reporting

A scheduled job runs 2 times due to replication between Search Heads. User gets 2 emails. Want to run only once

damonmanni
Path Finder

Hello,

A user scheduled job to run & email him results. When fired off, He gets 2 emails on each run. This is not good.

I did locate his job scheduled on BOTH our Search Head-1 & Search Head-2 svrs via Settings > S&R.

Background: When we installed splunk- Our consultant setup replication between Search Heads for redundancy. This is good, except that the job gets mirrored onto SH2 also, thus creating a 2nd run, thus problem.

How do I manage jobs so that they run only once, while still maintaining the redundancy required. This must be a common problem of lots of duplication.

Thank you in advance for ideas to solve.

cheers,
Damon

Tags (1)
0 Karma

the_wolverine
Champion

You can disable the scheduler on one of the search heads:

http://answers.splunk.com/answers/57107/dedicated-splunk-job-scheduler

0 Karma

damonmanni
Path Finder

Thx for the quick response Wolverine - Grazie!

So, I put in your suggestion:
index=_internal source=*scheduler.log
in both SH-1 & SH-2 - but it just spins on both SH's...runs forever & no hits at all. So, that can't be good/right.

I also tried:
index=_internal host=mdc2brc0123 source="/opt/splunk/var/log/splunk/scheduler.log"

spins also.

So, why would it show Enable if its really disabled? Why can't it report what is tru? 😉 I've been knocking myself out here running in circles b/c I can't count on what i'm seeing. Frustrating for sure.

I'm stuck big time.

0 Karma

the_wolverine
Champion

You'll see jobs enabled but they won't run on both SHs. To verify check the index=_internal source=*scheduler.log. You should only see searches running from one SH, not both.

Also, you should be able to see all saved searches and reports from either SH since they are pooled right? If you do not have SH_POOL set up then you need to do this so you do not have redundant SHs.

0 Karma

damonmanni
Path Finder

I tried your suggestion - but I cannot tell if it worked or not.Here is what I did:
Note: SearchHead-1 & SearchHead-2 both run a job scheduler. I want to turn off job scheduler completely on SearchHead-2. Steps:
ssh SH-2
vi /opt/splunk/etc/system/default/default-mode.conf
...
[pipeline:scheduler]
disabled = true

/opt/splunk/bin/splunk restart (no issues)

Now, how do I gaurentee JS is Really not running on SH-2? I checked SH-2 Settings>S&R>any/any and I still see all jobs listed as ENABLED. Need help, I am getting slammed by customers complaining about still receiving 2x emails per job.

0 Karma
Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...