topic Re: define custom fixed fields in Splunk Search

define custom fixed fields

acalvo — Thu, 17 Mar 2011 22:34:31 GMT

Is there any way to define custom fields which have a fixed size?

For instance: We can have field definition as: field 1: 3 characters field 2: 5 characters field 3: 120 characters total: 128 characters

And we'd like to have it separated and be able to search this fields.

Any suggestion?

Thanks

Re: define custom fixed fields

Ayn — Thu, 17 Mar 2011 22:50:31 GMT

Please explain in more detail what you want to achieve. My impression is that you want to create a field based on a match on a certain size of a word surrounded by whitespace in a log? I.e. in the following event

2011-03-15 03:18  foo fooba gazonk yiff

"field1" should match "foo", "field2" should match "fooba". Is this what you mean? In that case you can certainly do that, however could you please first acknowledge whether I'm correct in my assumption about what you want to achieve.

Re: define custom fixed fields

acalvo — Thu, 17 Mar 2011 23:36:51 GMT

yes, you are correct

so in your example, fooba could be field2 with 6 characters (note the starting whitespace)

Re: define custom fixed fields

David — Thu, 17 Mar 2011 23:46:08 GMT

I would do a field extraction as follows:

(?<Field1>.{3})(?<Field2>.{5})(?<Field3>.{120})$

The dollar sign will ensure that you're grabbing from the end of the line (making sure that you don't start Field1 in a timestamp, or anything like that).

You can either toss that in your props.conf, or do a search like:

YourSearch | rex field=_raw "(?<Field1>.{3})(?<Field2>.{5})(?<Field3>.{120})$" | table Field1 Field2 Field3

(Note that if your _raw doesn't have at least 128 characters, it won't extract any fields)

Added: Your props.conf might look like the below.

[yoursourcetype]
EXTRACT-FixedFields = (?<Field1>.{3})(?<Field2>.{5})(?<Field3>.{120})$

And you can replace yoursourcetype with (from the documentation):

1. <sourcetype>, the source type of an event.
2. host::<host>, where <host> is the host for an event.
3. source::<source>, where <source> is the source for an event.
4. rule::<rulename>, where <rulename> is a unique name of a source type classification rule.
5. delayedrule::<rulename>, where <rulename> is a unique name of a delayed source type classification rule.
These are only considered as a last resort before generating a new source type based on the source seen.

See the props.conf doc for more: http://www.splunk.com/base/Documentation/latest/admin/propsconf

Re: define custom fixed fields

acalvo — Fri, 18 Mar 2011 14:57:16 GMT

That's what I was looking for.

However, could you post an example for props.conf?

Will the fields by indexed or just used at search time?

Thanks

Re: define custom fixed fields

David — Fri, 18 Mar 2011 23:41:38 GMT

Sure. I just added it to the main response (so it wouldn't lose the formatting). The fields would be used at search time, as is the preferred way with Splunk. If you did want to go index-time, you can read that documentation (I've never done it myself): http://www.splunk.com/base/Documentation/4.2/Data/Configureindex-timefieldextraction