Dashboards & Visualizations

modular input xml streaming doesn't accept time element

kundeng
Path Finder

splunk modular input xml streaming mode allows specifying timestamp explicitly for each event. But it is not working for me.
Instead, splunk tries to find the event in the raw data and use index time if it is unavailable.

For example, my XML looks like this, and it would take 20150324111100 as the timestamp instead of 1540488716

I have also configured props.conf according to a hint in the documentation and it doesn't work either.
http://docs.splunk.com/Documentation/Splunk/7.2.0/AdvancedDev/ModInputsStream#Streaming_example_.28X...

Modify $SPLUNK_HOME/etc/apps/myapp/default/props.conf

[my_config]
SHOULD_LINEMERGE = false

2018-10-25 13:31:56,858 INFO the xml file looks like

<stream><event stanza="hl7_modular_input://test"><index>main</index><time>1540488716</time><sourcetype>hl7</sourcetype><data>MSH|^~\&amp;|EPIC|GMC||GMC|20150324111100|RECRN|ADT^A10|725308|T|2.3||||||||| 
EVN|A10|20150324111100||ADT_EVENT|Model User^RECOVERY^NURSE^^^^^^OHSA^^^^^GMC1|20150320125007| 
PID|1||3100068670^^^EPI^MR||OPTIME^TESTAVS||19950320|M|||^^^^^US^P|||||||6100212816|636-99-9512|||||||||||N|| 
PD1|||GRANT MEDICAL CENTER^^1013100||||||||||||||| 
PV1|1|O|GSAG^OR ^OR^GMC1^^^^^^^DEPID|EL|||13020^LOMBARDI^ADOLPH^V^^^^^STARPROV^^^^STARPROV|||Surgery|PACG^^^GMC1^^^^^^^DEPID|||Phys/Clinic|||13020^LOMBARDI^ADOLPH^V^^^^^STARPROV^^^^STARPROV|OP SURGERY|6100212816|SELF||||||||||||||||||||||^^^GMC1^^^^^^^|^^^^^^^^^^|20150320125007||||||22150790003 
PV2||GENERAL||||||20150320130000|||||||||||||n|N||||||||||||||||||||||||||| 
ZPV||||||||||||20150320125007||||||||| 
AL1|1|Drug Class|45202^NO KNOWN ALLERGIES^|||||| 
GT1|1|100046657|OPTIME^TESTAVS^^||^^^^^US|||19950320|M|P/F|SLF|636-99-9512||||||||||||||||||||||||||||||||||||| 
</data></event></stream>
0 Karma
1 Solution

kundeng
Path Finder

I think I figured it out.
The splunk documentation is wrong.

http://docs.splunk.com/Documentation/Splunk/7.2.0/AdvancedDev/ModInputsStream#Streaming_example_.28X...

The setting to force using input-layer timestamp should be
DATETIME_CONFIG = NONE (or CURRENT)

View solution in original post

0 Karma

kundeng
Path Finder

I think I figured it out.
The splunk documentation is wrong.

http://docs.splunk.com/Documentation/Splunk/7.2.0/AdvancedDev/ModInputsStream#Streaming_example_.28X...

The setting to force using input-layer timestamp should be
DATETIME_CONFIG = NONE (or CURRENT)

0 Karma
Got questions? Get answers!

Join the Splunk Community Slack to learn, troubleshoot, and make connections with fellow Splunk practitioners in real time!

Meet up IRL or virtually!

Join Splunk User Groups to connect and learn in-person by region or remotely by topic or industry.

Get Updates on the Splunk Community!

Laser Bananas and Edge Hubs: Exploring Operational Technology (OT) Data Through a ...

  OT is a different environment to traditional IT and can have interesting challenges when interfacing the ...

Event Series: Mastering AI Tokenomics and Splunk Agent Observability

Beyond the Black Box: Correlating AI Performance and Tokenomics with Splunk Agent Observability   As ...

span_metrics: The OpenTelemetry-Idiomatic Way to See Inside Your Services

You open a trace in Splunk Observability Cloud and everything looks fine. One root span, order-pipeline, with ...