Implied field extraction in Splunk?

[Josh]

So I have used props and tranforms to extract a field in my application by using a particular regex expression.

For instance lets say there is a particular line of text in the logs as follows:

username@host logged in at 15:00

So I have used props and tranforms to extract the value username into a user field this works fine. So now what I want to do now is within this applicaton where ever the entry username occurs in other events I would like it to be taged also as the field user.

So lets say somewhere else in my logs there is another event which has the line:

username invoked method x

I would like this event to have the field user which stored the value username.

Is there anyway that we can match the field value accorss over events in the application and also have them extracted into the field, or do you have to write every single possible regex pattern where the username may appear in the logs?

[sideview]

You do have to create regexes to match the values in different sourcetypes in different ways.

A couple ideas and tools spring to mind though to make this easier. If you pick a typical username and search for say:

mildred OR gladys OR agnes | top sourcetype

That will give you an idea of the sourcetypes where the username values spread to. Then, for each of the sourcetypes where there isnt already a good extraction, find 5 or 10 usernames that seem representative and run the following:

sourcetype=someSourceType | head 1000  | erex user examples="mildred,gladys,agnes,nico,etc..."

erex will chew on the data for a while and then actually tell you the best regular expression it can find that matches all those values successfully.

http://www.splunk.com/base/Documentation/latest/SearchReference/Erex

And it'll give you this regex in the form of a suggested rex command, so you can try it for a while as a rex clause before baking the regex into the config as a search-time field extraction.

For example, you can take the rex command it gives you, and assuming that rex command is creating a field called username, an easy way to quickly check the events that did NOT match, is just to tack on:

| search username!=*