Getting Data In

XML parsing is not working as expected, field values are truncating

imsidrai
Explorer

XML parsing is not working as expected, field values are truncating , tried changing truncate values in props but that doesnt help..

Tried with and without KV_MODE as well

props.conf on HF

[iiq_db]
TRUNCATE = 100000
NO_BINARY_CHECK = true
SHOULD_LINEMERGE = false
category = custom
pulldown_type = true
KV_MODE = xml

sample log

 

2021-09-24 14:28:29.011, id="XXXXXXX", created_dt="2021-09-24 08:18:01.87", created="1632489481870", source="RequestHandler", action="provision", target="XXXX", application="Enterprise Directory", account_name="XXXX,ou=XXXXX,ou=XXXX,ou=HO,o=XXX.com", attributes="<Attributes>
<Map>
<entry key="IIQDisabled">
<value>
<Boolean></Boolean>
</value>
</entry>
<entry key="accountFlag" value="ACTIVE"/>
<entry key="cn" value="XXXXX"/>
<entry key="dn" value="uid=XXXX,ou=XXXX,ou=XXXX,ou=HO,o=XXX.com"/>
<entry key="email" value="XX@XX.com"/>
<entry key="employeeNumber" value="XXXXX"/>
<entry key="employeeType" value="E"/>
<entry key="givenname" value="XXXXX"/>
<entry key="globaluid" value="XXX"/>
<entry key="mail" value="(XX0:XX1:XX3:XX0:XX0:XX0:XX0:XX0:XX0:XX0)XX@XX.com"/>
<entry key="mailAccessDomain" value="HO XXX"/>
<entry key="mailRoutingAddress" value="(XX0)EX"/>
<entry key="mailalternateaddress">
<value>
<List>
<String>(XX0:XX11)S=XXXX/G=XXX/OU=XXXX.com</String>
</List>
</value>
</entry>
<entry key="XXUniqueId" value="XYZ"/>
<entry key="XXaccountstatus" value="0"/>
<entry key="XXlcsp1" value="XYZ"/>
<entry key="XXlinteractivep1" value="XYZ"/>
<entry key="XXllcp1" value="XXXXYZ"/>
<entry key="XXlmaildisplayname" value="XY , AB"/>
<entry key="XXlmemberof">
<value>
<List>
<String>cn=Passphrase-Policy-Users,ou=groups,o=XXl.com</String>
</List>
</value>
</entry>
<entry key="XXlprofilechecksum" value="XXXX"/>
<entry key="XXorgmemberof">
<value>
<List>
<String>ou=XXX,ou=XXXX,ou=HO,o=XXl.com</String>
</List>
</value>
</entry>
<entry key="XXworkgroupmanager" value="XXXXXXX"/>
<entry key="nsAccountLock" value="FALSE"/>
<entry key="objectClass">
<value>
<List>
<String>top</String>
<String>person</String>
<String>organizationalperson</String>
<String>inetorgperson</String>
<String>mailrecipient</String>
<String>universaluniqueid</String>
<String>XXlorgperson</String>
</List>
</value>
</entry>
<entry key="op">
<value>
<ObjectOperation>Modify</ObjectOperation>
</value>
</entry>
<entry key="sn" value="XXXX"/>
<entry key="uid" value="XXXX"/>
<entry key="uidns" value="XXXX"/>
<entry key="uuid" value="XXXXX"/>
</Map>
</Attributes>
", string1="Enterprise Directory", string2="committed"

 

imsidrai_0-1632735163694.png

 

Labels (3)
0 Karma

ITWhisperer
SplunkTrust
SplunkTrust

It looks to me like the issue is not to do with the size but to do with the fact that the double quotes in the attributes field have not been escaped, so the attributes field is being terminated at the first double quote in the field.

0 Karma

imsidrai
Explorer

Hi @ITWhisperer 

I am a bit new to splunk , what should i do to avoid that ?

custom extractions?

Thank you 

0 Karma

ITWhisperer
SplunkTrust
SplunkTrust

Yes, I think you will need custom extractions

0 Karma

imsidrai
Explorer

@ITWhisperer it would be appreciable if you can point me to something to start with....

 

Thank you 

0 Karma
Get Updates on the Splunk Community!

Splunk Enterprise Security 8.0.2 Availability: On cloud and On-premise!

A few months ago, we released Splunk Enterprise Security 8.0 for our cloud customers. Today, we are excited to ...

Logs to Metrics

Logs and Metrics Logs are generally unstructured text or structured events emitted by applications and written ...

Developer Spotlight with Paul Stout

Welcome to our very first developer spotlight release series where we'll feature some awesome Splunk ...