Getting Data In

XML parsing is not working as expected, field values are truncating

imsidrai
Path Finder

XML parsing is not working as expected, field values are truncating , tried changing truncate values in props but that doesnt help..

Tried with and without KV_MODE as well

props.conf on HF

[iiq_db]
TRUNCATE = 100000
NO_BINARY_CHECK = true
SHOULD_LINEMERGE = false
category = custom
pulldown_type = true
KV_MODE = xml

sample log

 

2021-09-24 14:28:29.011, id="XXXXXXX", created_dt="2021-09-24 08:18:01.87", created="1632489481870", source="RequestHandler", action="provision", target="XXXX", application="Enterprise Directory", account_name="XXXX,ou=XXXXX,ou=XXXX,ou=HO,o=XXX.com", attributes="<Attributes>
<Map>
<entry key="IIQDisabled">
<value>
<Boolean></Boolean>
</value>
</entry>
<entry key="accountFlag" value="ACTIVE"/>
<entry key="cn" value="XXXXX"/>
<entry key="dn" value="uid=XXXX,ou=XXXX,ou=XXXX,ou=HO,o=XXX.com"/>
<entry key="email" value="[email protected]"/>
<entry key="employeeNumber" value="XXXXX"/>
<entry key="employeeType" value="E"/>
<entry key="givenname" value="XXXXX"/>
<entry key="globaluid" value="XXX"/>
<entry key="mail" value="(XX0:XX1:XX3:XX0:XX0:XX0:XX0:XX0:XX0:XX0)[email protected]"/>
<entry key="mailAccessDomain" value="HO XXX"/>
<entry key="mailRoutingAddress" value="(XX0)EX"/>
<entry key="mailalternateaddress">
<value>
<List>
<String>(XX0:XX11)S=XXXX/G=XXX/OU=XXXX.com</String>
</List>
</value>
</entry>
<entry key="XXUniqueId" value="XYZ"/>
<entry key="XXaccountstatus" value="0"/>
<entry key="XXlcsp1" value="XYZ"/>
<entry key="XXlinteractivep1" value="XYZ"/>
<entry key="XXllcp1" value="XXXXYZ"/>
<entry key="XXlmaildisplayname" value="XY , AB"/>
<entry key="XXlmemberof">
<value>
<List>
<String>cn=Passphrase-Policy-Users,ou=groups,o=XXl.com</String>
</List>
</value>
</entry>
<entry key="XXlprofilechecksum" value="XXXX"/>
<entry key="XXorgmemberof">
<value>
<List>
<String>ou=XXX,ou=XXXX,ou=HO,o=XXl.com</String>
</List>
</value>
</entry>
<entry key="XXworkgroupmanager" value="XXXXXXX"/>
<entry key="nsAccountLock" value="FALSE"/>
<entry key="objectClass">
<value>
<List>
<String>top</String>
<String>person</String>
<String>organizationalperson</String>
<String>inetorgperson</String>
<String>mailrecipient</String>
<String>universaluniqueid</String>
<String>XXlorgperson</String>
</List>
</value>
</entry>
<entry key="op">
<value>
<ObjectOperation>Modify</ObjectOperation>
</value>
</entry>
<entry key="sn" value="XXXX"/>
<entry key="uid" value="XXXX"/>
<entry key="uidns" value="XXXX"/>
<entry key="uuid" value="XXXXX"/>
</Map>
</Attributes>
", string1="Enterprise Directory", string2="committed"

 

imsidrai_0-1632735163694.png

 

Labels (3)
0 Karma

ITWhisperer
SplunkTrust
SplunkTrust

It looks to me like the issue is not to do with the size but to do with the fact that the double quotes in the attributes field have not been escaped, so the attributes field is being terminated at the first double quote in the field.

0 Karma

imsidrai
Path Finder

Hi @ITWhisperer 

I am a bit new to splunk , what should i do to avoid that ?

custom extractions?

Thank you 

0 Karma

ITWhisperer
SplunkTrust
SplunkTrust

Yes, I think you will need custom extractions

0 Karma

imsidrai
Path Finder

@ITWhisperer it would be appreciable if you can point me to something to start with....

 

Thank you 

0 Karma
Got questions? Get answers!

Join the Splunk Community Slack to learn, troubleshoot, and make connections with fellow Splunk practitioners in real time!

Meet up IRL or virtually!

Join Splunk User Groups to connect and learn in-person by region or remotely by topic or industry.

Get Updates on the Splunk Community!

A Four-Part Event Series: Full Stack Observability For the AI Era

As AI reshapes applications, infrastructure, and the way teams operate, the traditional boundaries of ...

SOC4Kafka - New Kafka Connector Powered by OpenTelemetry

The new SOC4Kafka connector, built on OpenTelemetry, enables the collection of Kafka messages and forwards ...

Event Series: Level up your SOC: Advancing with Splunk Enterprise Security

AI has fundamentally raised the stakes for security operations, and this three-part series is your guide to ...