topic Re: Override source (tcp:xxxx) of a tcp input using transforms in Getting Data In

Override source (tcp:xxxx) of a tcp input using transforms

parallaxed — Fri, 13 Aug 2010 18:56:16 GMT

Looks like MetaData:Source should be used, but despite many variations and | extract reload=t, I can't seem to get this to work, even by attempting to force it, as per below

transforms.conf

[net_type]
DEST_KEY = MetaData:Source
REGEX = .*
FORMAT = source::VMSTAT
WRITE_META = true

props.conf 

[net]
SHOULD_LINEMERGE=false
TRANSFORMS-net_type = net_type

^ Firstly, this "forcing" seems like it should be valid - it may not be, please correct me.

I'm looking to apply this depending on the raw text of the event, so my source type isn't fixed and can't be set in inputs.conf.

Is source override possible for only certain types of inputs?

I should add this is Splunk 4.1.x, and that this transformation works if I use MetaData:Sourcetype instead of MetaData:Source. Why would it work with one field but not the other?

Re: Override source (tcp:xxxx) of a tcp input using transforms

Lowell — Fri, 13 Aug 2010 21:56:44 GMT

I believe this should work. Please note that using | extract reload=T will not reload an index-time config like this, you will need to restart splunkd.

I would also suggest the following in transforms.conf:

[net_type]
DEST_KEY = MetaData:Source
REGEX = .
FORMAT = source::VMSTAT

Two changes: (1) You don't need WRITE_META in this case. (2) No need use the more-expensive .*, when a simple . will do the trick.

Re: Override source (tcp:xxxx) of a tcp input using transforms

parallaxed — Fri, 13 Aug 2010 22:09:54 GMT

Restart was definitively needed, that was clearly hampering the testing.

Re: Override source (tcp:xxxx) of a tcp input using transforms

gkanapathy — Wed, 18 Aug 2010 23:19:30 GMT

I am reasonably sure that on your input stanza, you can just specify:

[tcp:xxxx]
sourcetype = net_type
source = VMSTAT

This won't work on a splunktcp: input, as that comes from a forwarder and the source would have been set on the forwarder, but for scripted, udp, and tcp inputs, you can just override it like this.

Re: Override source (tcp:xxxx) of a tcp input using transforms

gkanapathy — Wed, 18 Aug 2010 23:23:17 GMT

A regex of . or .? seems better to me than .*, although I don't know if PCRE optimizes this out if there is no capture group. I personally use (?=) for a PCRE regex that always matches.