I don't know if I'm doing this correctly.
I have a text file that contains data I want to index. So with the universal forwarder I've setup a monitor statement like this:
The contents of the file look something like this:
[Fri Feb 22 11:54:51 2013] Serial Number: <333-333-222/12000000>
[Fri Feb 22 11:54:51 2013] Model Type:
[Fri Feb 22 11:54:51 2013] O/S:
I would like to pull the following data:
Serial Number as Field serialNumber
Model Type as Field modelType
And O/S as Field operatingSystem
I'm doing so with a combination of props.conf and transforms.conf
sourcetype = mystuff
REPORT-myst = mystuff_extractions
REGEX = my regex statement?????
FORMAT = serialNumber::$1 modelNumber::$2 operatingSystem::$3
Luckily all of the information is found within the brackets < >. But I am unsure of how to build a regular a regular expression that will do this and pull all three values at the same time. I've tested a few times but have not been successfull.
Aside from getting the regex right, I would like to perform all of this as efficiently as possible. I simple hacked to gether this stuff from different posts I've read, but if there is a better way to do this, please let me know.
This specific log file is very small. There is no larger example.
Before you can do that you're going to have to teach splunk what a record looks like. Your data is spread over serveral lines, and by default splunk sees every line as a record. Can you post a larger example of the log, or identify a consistend start or end to an individual record ?
The simplest in my mind is to use inline extractions, you can set these through Manager -> Fields -> Field Extractions. As for your expressions, something like this might work (one per extraction, whole line goes into the Inline field):
Serial Number: <(?<serialNumber>[^>]+)> Model Type: <(?<modelType>[^>]+)> O/S: <(?<operatingSystem>[^>]+)>
All three work in the same way - they first look for the prefix of your field, for example "O/S: <", then for the field content, at least one character that's not a closing angle bracket, and then for the suffix ">".
If you prefer editing the config files manually, add an EXTRACT-classname key for each to the appropriate props.conf stanza.
Concerning efficiency, these expressions are extremely efficient because they look for a very specific static prefix - in other words, they fail fast. There are no huge trees of options to walk before the automaton notices that it's in the wrong forest.
It was in my apps/search/local/. Still one more thing bugging me. I'd like to at least test this out by using the transforms.conf. Would you be able to provide me the right syntax so I can compare the results? Thanks!
Don't forget to check the user-specific directories if you didn't set the extraction permissions to global.
That's the thing. I created this through the GUI and looked for the configuration file change but I don't see it. I'm looking in system/local config files. I looked in both the props.conf and transforms.conf. Should I be looking at another props.conf file in one of the other directories? I was under the impression that all changes should be made in the system/local directory.
Do keep in mind, every setting in the manager eventually makes its way into a configuration file. In this case it's an EXTRACT-foo statement in props.conf - you can edit this manually of course.
I'm sure you can do the same with REPORT-foo and a stanza in transforms.conf, but it feels like more configuration effort for no functionality gain.
I tried this out and it worked. But I prefer to manage my configurations through text files. Is there a way to get this working through the transforms.conf?