Getting Data In

log file parsing on IDX

secuc2r83
Path Finder

Hello,
I just want to parse a log file. I try every solution found on forum but never work.
(Splunk 7.3.3)

Log:

<event action=....>
  <start>2020-02-22 12:49:21:596</start>
  <client .../>
  <sent ...></sent>
</event>
<event action=...>
  <start>2020-02-22 12:49:20:435</start>
  <client .../>
  <sent ...></sent>
</event>

What i want on Splunk SH:

_time               _raw
2020-02-22 13:49:21 <event action=...>
                      <start>2020-02-22 12:49:21:596</start>
                      <client .../>
                      <sent ...></sent>
                    </event>
2020-02-22 13:45:20 <event action=...>
                      <start>2020-02-22 12:49:21:596</start>
                      <client .../>
                      <sent ...></sent>
                    </event>

What i have on Splunk SH:

_time               _raw
2020-02-22 13:49:21 <event action=...>
2020-02-22 13:45:20 <event action=...>

inputs.conf on UF:

index = test
sourcetype = my_sourcetype
disabled = 0

4 props.conf tried on Indexer (based on forum's solution):

[my_sourcetype]
LINE_BREAKER = ([\r\n]+)\<event
SHOULD_LINEMERGE = false
DATETIME_CONFIG = NONE

[my_sourcetype]
SHOULD_LINEMERGE = true
BREAK_ONLY_BEFORE = ^\s*\<event

[my_sourcetype]
SHOULD_LINEMERGE = true
MUST_BREAK_AFTER = \</event\>

[my_sourcetype]
DATETIME_CONFIG = CURRENT
KV_MODE = xml
LINE_BREAKER = ([\r\n]+)(?=\s*\<event.*?\>)   
BREAK_ONLY_BEFORE_DATE = False
MUST_BREAK_AFTER = \</event\>
NO_BINARY_CHECK = 1
SHOULD_LINEMERGE = false
TRUNCATE = 0

props.conf on Search Head:

[my_sourcetype]
KV_MODE = xml

After each change, i reboot splunk on Idx and SH
Thanks for help
Regards

0 Karma

richgalloway
SplunkTrust
SplunkTrust

Try these props on your indexers. Keep in mind changing props.conf will not affect data that is already indexed; only new data will be affected.

[my_sourcetype]
  KV_MODE = xml
  LINE_BREAKER = ([\r\n]+)\<event 
  TIME_PREFIX = start>
  TIME_FORMAT = %Y-%d-%n %H:%M:%S:%3N
---
If this reply helps you, Karma would be appreciated.
0 Karma

secuc2r83
Path Finder

Hi richgalloway,

Yes i generate new events for each test

<event action=...>
  <start>2020-02-28 18:04:14:706</start>
  <end>2020-02-28 18:04:32:629</end>
  <client .../>
  <sent ...></sent>
 </event>

With your conf, same result, juste one line in SH:

 _time                      _raw
2020-02-28 18:04:37     <event action=...>

Here are all settings on indexer with btool, perhaps could see the problem:

# ./splunk btool props list test --debug
.../etc/slave-apps/app_test/local/props.conf [test]
.../etc/system/default/props.conf             ADD_EXTRA_TIME_FIELDS = True
.../etc/system/default/props.conf             ANNOTATE_PUNCT = True
.../etc/system/default/props.conf             AUTO_KV_JSON = true
.../etc/system/default/props.conf             BREAK_ONLY_BEFORE =
.../etc/system/default/props.conf             BREAK_ONLY_BEFORE_DATE = True
.../etc/system/default/props.conf             CHARSET = UTF-8
.../etc/system/default/props.conf             DATETIME_CONFIG = /etc/datetime.xml
.../etc/system/default/props.conf             DEPTH_LIMIT = 1000
.../etc/system/default/props.conf             HEADER_MODE =
.../etc/slave-apps/app_test/local/props.conf  KV_MODE = xml
.../etc/system/default/props.conf             LEARN_MODEL = true
.../etc/system/default/props.conf             LEARN_SOURCETYPE = true
.../etc/slave-apps/app_test/local/props.conf  LINE_BREAKER = ([\r\n]+)\<event
.../etc/system/default/props.conf             LINE_BREAKER_LOOKBEHIND = 100
.../etc/system/default/props.conf             MATCH_LIMIT = 100000
.../etc/system/default/props.conf             MAX_DAYS_AGO = 2000
.../etc/system/default/props.conf             MAX_DAYS_HENCE = 2
.../etc/system/default/props.conf             MAX_DIFF_SECS_AGO = 3600
.../etc/system/default/props.conf             MAX_DIFF_SECS_HENCE = 604800
.../etc/system/default/props.conf             MAX_EVENTS = 256
.../etc/system/default/props.conf             MAX_TIMESTAMP_LOOKAHEAD = 128
.../etc/system/default/props.conf             MUST_BREAK_AFTER =
.../etc/system/default/props.conf             MUST_NOT_BREAK_AFTER =
.../etc/system/default/props.conf             MUST_NOT_BREAK_BEFORE =
.../etc/system/default/props.conf             SEGMENTATION = indexing
.../etc/system/default/props.conf             SEGMENTATION-all = full
.../etc/system/default/props.conf             SEGMENTATION-inner = inner
.../etc/system/default/props.conf             SEGMENTATION-outer = outer
.../etc/system/default/props.conf             SEGMENTATION-raw = none
.../etc/system/default/props.conf             SEGMENTATION-standard = standard
.../etc/system/default/props.conf             SHOULD_LINEMERGE = True
.../etc/slave-apps/app_test/local/props.conf  TIME_FORMAT = %Y-%d-%n %H:%M:%S:%3N
.../etc/slave-apps/app_test/local/props.conf  TIME_PREFIX = start>
.../etc/system/default/props.conf             TRANSFORMS =
.../etc/system/default/props.conf             TRUNCATE = 10000
.../etc/system/default/props.conf             detect_trailing_nulls = false
.../etc/system/default/props.conf             maxDist = 100
.../etc/system/default/props.conf             priority =
.../etc/system/default/props.conf             sourcetype =
0 Karma
Get Updates on the Splunk Community!

.conf24 | Registration Open!

Hello, hello! I come bearing good news: Registration for .conf24 is now open!   conf is Splunk’s rad annual ...

ICYMI - Check out the latest releases of Splunk Edge Processor

Splunk is pleased to announce the latest enhancements to Splunk Edge Processor.  HEC Receiver authorization ...

Introducing the 2024 SplunkTrust!

Hello, Splunk Community! We are beyond thrilled to announce our newest group of SplunkTrust members!  The ...