How to parse and extract multiple hosts in one file so I can search based on hostname and per field?

[nirt]

I have a source file which looks like the following:

<MOBILE hostname="test1">
a = 2
b = 3
c = 4
...
</MOBILE>
<CENTRAL hostname="test2">
a = 2
b = 3
c = 4
...
</CENTRAL>
<MOBILE hostname="test3">
a = 2
b = 3
c = 4
...
</MOBILE>

What would be the best way to extract the data based on this file so I can search based on hostname and per field?

[gcato]

Hi nirt,

Using the example data provided you can set up your props.conf and transforms.conf in your apps local directory. For example:

props.conf
...
[my_sourcetype]
DATETIME_CONFIG = CURRENT
LINE_BREAKER = ([\n\r]+)\s*<[MC]+
SHOULD_LINEMERGE = false
NO_BINARY_CHECK = true
category = Custom
pulldown_type = true
disabled = false
TRANSFORMS-newhost = newhost

This will break the events by lines beginning with <(C|M) and the following will extract the hostname value and index it as host.

transforms.conf
...
[newhost]
REGEX = ^\s*<(?:MOBILE|CENTRAL)\s+hostname="(\w+)">
DEST_KEY = MetaData:Host
FORMAT = host::$1

The abc values will also be auto extracted due to the semantic logging, i.e. field=value.

Hope this helps.

[sundareshr]

Well, that depends. In the example you provide the data is not well-formed XML. It has some xml tags but not the a=2, b=3 etc are not valid xml. If all the data is valid xml, in your props.conf, you should be able to use KV_MODE = xml to extract xml data.

You could also use the xpath command in search to extract fields at search time and search based on hostname. http://docs.splunk.com/Documentation/Splunk/6.2.4/SearchReference/xpath

For example something like this might work

...| xpath outfield=hostname "//*/@hostname"