Rex on an extracted field

[romantercero]

Hi,

I'm trying to extract the cluster name of my servers using the host name. So we have something like host=clusterx01.abc.com, host=clusterx02.abc.com, host=clusterx03.abc.com. I would like to extract the "cluster" part and leave out everything after that out. This would normally be easy if the hostname was in the _rawfield but it's not, and setting the rex field to host is not working. Also, erex is not helping. It keeps inclding the first zero in the hostname.

Any thoughts on how I can use rex on a field that is created on index time?

Thanks!!

[Lucas_K]

I don't have an install here right now to check the interface so I can't think what fields you get in the gui. Normally what I would have done would be to put it into my transforms.conf as a search time extraction.

example inside transforms.conf
[cluster_name_extraction]
SOURCE_KEY = host
REGEX = ^(D+).
FORMAT = cluster_name::$1

You'd then make a reference to it in your props.conf under your particular sourcetype definition

REPORT-cluster_name_extraction = cluster_name_extraction

More info here : http://docs.splunk.com/Documentation/Splunk/4.3.4/admin/Transformsconf

So perhaps some of those options above exist in the gui also (i'll check later if you hadn't seen this already and i'll update this answer).

[Lucas_K]

I think Splunk best practice is to make your extractions search time and not index time. I believe the reasoning behind this is that your understanding of the data can and most likely will change in the future.

By having your fields extracted at search time you are not stuck with bad decisions made when initially onboarding the data.

[romantercero]

That's interesting. I had though of index time extractions but stopped after realizing that other fields extracted at this time would most probably not be available for me to base an extraction on them. I think you found another way to tackle this issue. Thanks!

[Lucas_K · Accepted solution]

It should work with indexed fields fine just like any other field.
ie. rex field=host "^(?P\w+)." thats the first section of the name.

just modify the rex to remove the numbers off the end. ahla ^(?P\w+)\d\d.

[Lucas_K]

I don't have an install here right now to check the interface so I can't think what fields you get in the gui. Normally what I would have done would be to put it into my transforms.conf as a search time extraction (not enough characters available to put it here 😐 )

[romantercero]

Ok, followup question: How would I save my "field=host (?\D+)" extraction in the field extractions sections on the management page? It's not working. I have a few hunches about why but no workaround so far 😕 any toughs?

[romantercero]

I was actually trying rex field=host "(?\D+)" as that had worked in the past but it was not working when the data was not in the _raw field. I tried your suggestion and YES! It worked 🙂 I tried what I was doing before and that also works now : / maybe a I had a typo somewhere and retyping everything must have fixed it.

Thanks!