How to index XML files in a directory location?

[nir_sheep]

Hi,

I am new to Splunk. i was able to get data indexed for regular lg files., but we have some Ora audit XML files that we want to index to search data for truobleshooting some issues. to try this, we have put some of the XML files on the splunk server in a location /mnt/avtest

name of files = idprd1_ora_17472_1.xml, idprd1_ora_17482_2.xml ...

sample data :

11.2
1684097112014-11-25T18:09:48.792252Zuseruserhost17482unknown110000000000000000000Authenticated by: DATABASE; Client address: (ADDRESS=(PROTOCOL=tcp)(HOST=IP_address)(PORT=port))511111

1684097322014-11-25T18:09:48.795447ZUSERuserhost17482unknown1OwnerPLD_GROUP_PERSON103011111---------S------111111
 #1(6):147037
select pldgrouppe0_.id as id40_, pldgrouppe0_.CREATION_TIMESTAMP as CREATION2_40_, pldgrouppe0_.created_by as created3_40_, pldgrouppe0_.pld_group_id as pld6_40_, pldgrouppe0_.pld_person_id as pld7_40_, pldgrouppe0_.LAST_UPDATE_TIMESTAMP as LAST4_40_, pldgrouppe0_.LAST_UPDATED_BY as LAST5_40_ from pld_group_person pldgrouppe0_ where pldgrouppe0_.pld_group_id=:1

Can you please guide me how to add the XML data and parse it? I have tried doing it using the splunk web also followed examples to edit props.conf and inputs.conf to index data, but its still not indexing data.

Thanks
Navd

[cpetterborg]

I have XML files doing something like this in the props.conf file:

[<your_source_type>]
KV_MODE = xml
MAX_TIMESTAMP_LOOKAHEAD=150
SHOULD_LINEMERGE=true
BREAK_ONLY_BEFORE=^\<msg
NO_BINARY_CHECK=true

That should catch the fact that it is XML. You may have additional things you are needing to add to the configuration, and you may have to modify some of the parameters (like the value for MAX_TIMESTAMP_LOOKAHEAD and BREAK_ONLY_BEFORE,, but otherwise this should get you started.

You should follow the normal standards for specifying the inputs.conf file to have the same sourcetype that you specify in the props.conf file on your indexer(s).

[nir_sheep]

Thanks much.
It started parsing the data now., the actual issue looked with no data getting indexed at all after we downloaded splunk DB app, that was not configured. I deleted it and restarted splunk. its indexing data now, but its doing it for current time stamp. Needs to extract the time stamps :

props.conf
[aud_xml]
KV_MODE = xml
MAX_TIMESTAMP_LOOKAHEAD=165
SHOULD_LINEMERGE=true
BREAK_ONLY_BEFORE=<\s*\/AuditRecord\s*>
NO_BINARY_CHECK=true

inputs.conf

[monitor:///mnt/avtest]
disabled = 0
followTail = 0
host = host
sourcetype = aud_xml
whitelist = idprd1_ora_[\d]*_[\d].xml$
crcSalt =

sample file format :

11.2
1697608112014-11-25T22:07:27.695390ZUSERuserhost17498unknown110000000000000000000Authenticated by: DATABASE; Client address: (ADDRESS=(PROTOCOL=tcp)(HOST=ip)(PORT=port))5id

Thanks
Navd