define custom fixed fields

[acalvo]

Is there any way to define custom fields which have a fixed size?

For instance: We can have field definition as: field 1: 3 characters field 2: 5 characters field 3: 120 characters total: 128 characters

And we'd like to have it separated and be able to search this fields.

Any suggestion?

Thanks

[David · Accepted solution]

I would do a field extraction as follows:

(?<Field1>.{3})(?<Field2>.{5})(?<Field3>.{120})$

The dollar sign will ensure that you're grabbing from the end of the line (making sure that you don't start Field1 in a timestamp, or anything like that).

You can either toss that in your props.conf, or do a search like:

YourSearch | rex field=_raw "(?<Field1>.{3})(?<Field2>.{5})(?<Field3>.{120})$" | table Field1 Field2 Field3

(Note that if your _raw doesn't have at least 128 characters, it won't extract any fields)

Added: Your props.conf might look like the below.

[yoursourcetype]
EXTRACT-FixedFields = (?<Field1>.{3})(?<Field2>.{5})(?<Field3>.{120})$

And you can replace yoursourcetype with (from the documentation):

1. <sourcetype>, the source type of an event.
2. host::<host>, where <host> is the host for an event.
3. source::<source>, where <source> is the source for an event.
4. rule::<rulename>, where <rulename> is a unique name of a source type classification rule.
5. delayedrule::<rulename>, where <rulename> is a unique name of a delayed source type classification rule.
These are only considered as a last resort before generating a new source type based on the source seen.

See the props.conf doc for more: http://www.splunk.com/base/Documentation/latest/admin/propsconf

[David]

Sure. I just added it to the main response (so it wouldn't lose the formatting). The fields would be used at search time, as is the preferred way with Splunk. If you did want to go index-time, you can read that documentation (I've never done it myself): http://www.splunk.com/base/Documentation/4.2/Data/Configureindex-timefieldextraction

[acalvo]

That's what I was looking for.

However, could you post an example for props.conf?

Will the fields by indexed or just used at search time?

Thanks

[Ayn]

Please explain in more detail what you want to achieve. My impression is that you want to create a field based on a match on a certain size of a word surrounded by whitespace in a log? I.e. in the following event

2011-03-15 03:18  foo fooba gazonk yiff

"field1" should match "foo", "field2" should match "fooba". Is this what you mean? In that case you can certainly do that, however could you please first acknowledge whether I'm correct in my assumption about what you want to achieve.

[acalvo]

yes, you are correct

so in your example, fooba could be field2 with 6 characters (note the starting whitespace)