topic Re: Reroute data that is marked for an index? in Splunk Dev

Reroute data that is marked for an index?

the_wolverine — Tue, 28 Sep 2010 04:58:12 GMT

I have syslog-ng data coming from LWFs that have been earmarked for indexA. I want to intercept these events and reroute them to another index called indexB. It doesn't seem to be working. Am I missing something basic?

The sourcetype is syslog so in props I have:

[syslog]
TRANSFORMS-route = route2indexB

transforms.conf:

[route2indexB]
REGEX=(192.168.1.12)
DEST_KEY = _MetaData:Index
FORMAT = indexB

I've tried multiple iterations of this configuration including using source and host in props.conf. I can't seem to get the data to go to indexB.

Re: Reroute data that is marked for an index?

gkanapathy — Tue, 28 Sep 2010 10:14:11 GMT

please provide a complete splunk diag

Re: Reroute data that is marked for an index?

Simeon — Tue, 28 Sep 2010 22:40:03 GMT

You are doing IT wrong

Re: Reroute data that is marked for an index?

the_wolverine — Tue, 28 Sep 2010 23:02:41 GMT

Har Har Har, guys.

Re: Reroute data that is marked for an index?

the_wolverine — Tue, 28 Sep 2010 23:19:52 GMT

Simeon, how am I doing IT wrong?

Re: Reroute data that is marked for an index?

Genti — Wed, 29 Sep 2010 04:50:34 GMT

instead of doing it with props/transforms, why do you not tell the LWF to send to indexB? Rerouting with props/transforms even if possible should cause slowness in indexing...

Re: Reroute data that is marked for an index?

Genti — Wed, 29 Sep 2010 04:54:39 GMT

Continuing my comment:
Telling the LWF where to send the data should be cheaper (resourcewise) and quite easy. here's an answer with a similar idea: http://answers.splunk.com/questions/5134/can-i-forward-different-data-inputs-to-different-splunk-indexers-on-a-light-forwa

Re: Reroute data that is marked for an index?

the_wolverine — Thu, 30 Sep 2010 06:07:57 GMT

We have data from this host that is going to indexA. I really want to be able to keep my hands off the LWF configuration so I don't have to set those up.

Re: Reroute data that is marked for an index?

the_wolverine — Thu, 30 Sep 2010 06:09:48 GMT

Turns out the LWF was not a LWF. It was a heavyweight forwarder 🙂

Thanks to Raitz for figuring that out. He spotted the _linebreaker in the tcpdump output which is an indication of cooked data.

I had the system owner enable LWF from CLI and all is working as expected.

Re: Reroute data that is marked for an index?

the_wolverine — Thu, 30 Sep 2010 06:13:05 GMT

Here's the tcpdump command that was run at the indexer: /usr/sbin/tcpdump -A -s 1512 host and port 9997

Re: Reroute data that is marked for an index?

gkanapathy — Thu, 30 Sep 2010 10:50:37 GMT

Wow. Would have been easier if you'd sent a Splunk diag.

Re: Reroute data that is marked for an index?

gkanapathy — Thu, 30 Sep 2010 10:51:01 GMT

Ghetto. If you had control over the forwarder configs, maybe you could actually be sure it was a LWF.

Re: Reroute data that is marked for an index?

the_wolverine — Thu, 30 Sep 2010 22:40:36 GMT

I'm not here to babysit forwarders 😉 Not Ghetto.