Getting Data In
Highlighted

per event sourcetype override

Path Finder

Hello,

I have events that by default are assigned to syslog sourcetype.

each of such event contains following sequence in it: Local7.Info 10.5.0.11 Feb 12 17:09:34 10.5.0.11 AlteonOS (and other info). I decided that describing this sequence via regex will identify those events that I would like to change a sourcetype for.

So I created stanza in TRANSFORMS

[sourcetypechange]
REGEX = \d{2}-\d{2}-\d{2}-\d{4}\s\d{2}:\d{2}:\d{2}\s\Local7.Info\s\d.\d.\d.\d\s\w{3}\s\d{2}\s\d{2}:\d{2}:\d{2}\s\d.\d.\d.\d\s\w\lteonOS\s\
DEST_KEY = MetaData:Sourcetype
FORMAT = sourcetype::Redsyslog

and also added following in PROPS

[syslog]
TRANSFORMS-sourcetype = sourcetypechange

As I understand, whenever splunk sees a syslog sourcetype - it has to check regex and if it matches it has to change sourcetype to Redsyslog, however I still have no result.

I was wondering, while writing a REGEX, shall it describe the whole event (from the beginning to the end) or just part (which is my case) is sufficient?

0 Karma
Highlighted

Re: per event sourcetype override

SplunkTrust
SplunkTrust

The REGEX string only needs to match part of the event.

---
If this reply helps you, an upvote would be appreciated.
0 Karma
Highlighted

Re: per event sourcetype override

Path Finder

thanks. This means there is something missing.

0 Karma
Highlighted

Re: per event sourcetype override

Influencer

I believe you have a mistake here:

SOURCE_KEY = _MetaData:Sourcetype

0 Karma
Highlighted

Re: per event sourcetype override

Path Finder

You mean instead of DESTKEY should be SOURCEKEY?

0 Karma
Highlighted

Re: per event sourcetype override

Influencer

Sorry it was my mistake about the syntax, nothing wrong with yours. Splunk applies the regex to the whole event. You should use a regex that uniquely identifies the kind of events you want to override Sourcertyoe. That is the criteria, a regex enoughly long and that let's your events be overridden.
Are you sure your regex is capturing the events?

0 Karma
Highlighted

Re: per event sourcetype override

Path Finder

Well I check it at https://regex101.com/ and it highlight my event. Any suggestions where else I can check my regex?

0 Karma
Highlighted

Re: per event sourcetype override

Influencer

Splunk uses Regex PCRE flavor of regular expressions, so anything that is PCRE-compliant is good to go. Did you make sure that was the flavor you were testing?

0 Karma
Highlighted

Re: per event sourcetype override

Path Finder

OK I think that was the issue, now sourcetype changes. thanks a lot. however, now it does not do time extraction.

0 Karma
Highlighted

Re: per event sourcetype override

Influencer

Ok good it is working.

The time may be missing for two reasons. Is the timestamp written after the 128th character in the event (which is the default of the parameter MAXTIMESTAMPLOOKAHEAD). Increase the MAXTIMESTAMPLOOKAHEAD if it is the case.

More, to make sure timestamp is recognised by Splunk, use the TIME_FORMAT parameter in the sourcetype definition (below the TRANSFORMS-sourcetype = sourcetypechange).

TIME_FORMAT=%b %d %H:%M:%S

For info on how the time variables work, http://docs.splunk.com/Documentation/Splunk/7.0.2/SearchReference/Commontimeformatvariables

TIME_FORMAT = <strptime-style format>
* Specifies a strptime format string to extract the date.
* strptime is an industry standard for designating time formats.
* For more information on strptime, see "Configure timestamp recognition" in
  the online documentation.
* TIME_FORMAT starts reading after the TIME_PREFIX. If both are specified,
  the TIME_PREFIX regex must match up to and including the character before
  the TIME_FORMAT date.
* For good results, the <strptime-style format> should describe the day of
  the year and the time of day.
* Defaults to empty.
0 Karma