Hi,
In my data, I have two kinds of XML.
i.e. Request & Response
I want to break the log when my starts and ends with also when starts and ends with.
Can someone help to achieve this?
Below is my sample log data.. Out of it I only want the Request & Response XML information to be indexed and other should not indexed or shouldn't be visible in my search index result
11/28 07:20:20.31 do_ta(99)5733 syncWithNS: End syncing cache with NS
11/28 07:20:20.31 fgdf(99)556 collectIORs: HostNode:[hfeihf] LastTimeIORsCollected=1574925420, cur=1574925620. Diff = -200. Min = 240. So not collecting now.
11/28 07:20:20.31 fgdf(99)556 collectIORs: HostNode:[hfeihf] LastTimeIORsCollected=1574925420, cur=1574925620. Diff = -200. Min = 240. So not collecting now.
11/28 07:20:20.31 fgdf(99)556 collectIORs: HostNode:[hfeihf] LastTimeIORsCollected=1574925420, cur=1574925620. Diff = -200. Min = 240. So not collecting now.
11/28 07:20:20.31 fgdf(99)556 collectIORs: HostNode:[hfeihf] LastTimeIORsCollected=1574925420, cur=1574925620. Diff = -200. Min = 240. So not collecting now.
11/28 07:20:20.31 fgdf(99)556 collectIORs: HostNode:[feihf] LastTimeIORsCollected=1574925420, cur=1574925620. Diff = -200. Min = 240. So not collecting now.
11/28 07:16:59.10 oraconn(9979)873 executeStmt: EXEC Stmt: [SELECT * FROM abc WHERE active=1 AND root_id='xyz' AND root_kind='fgjh' AND domain_id='1213' AND domain_kind='dfsd' AND service_id='GG' AND service_kind='gfg' AND field1='Linux']
11/28 07:16:59.11 oraconn(9979)1575 fetchResultSet: Got 0 row(s) for the query. Time taken = 0.000 sec
11/28 07:16:59.08 oraconn(9979)873 executeStmt: EXEC Stmt: [SELECT * FROM abc WHERE active=1 AND root_id='xyz' AND root_kind='fgjh' AND domain_id='1213' AND domain_kind='dfsd' AND service_id='GWRNG' AND service_kind='gfg' AND field1='Linux']
11/28 07:16:59.08 oraconn(9979)1575 fetchResultSet: Got 0 row(s) for the query. Time taken = 0.000 sec
11/27 10:42:48.49 dfd(131)10151 comd::checktest KEY: aa:A78900007-, bb:76000009, cc:VS, REQID:0,
|123H
FT
TE
DS
10-118-224-197.
V
2019-11-27T10:42:15Z
Y
Y
Y
11/28 07:20:20.31 fgdf(9979)556 collectIORs: HostNode:[DNA4-aff] LastTimeIORsCollected=1574925420, cur=1574925620. Diff = -200. Min = 240. So not collecting now.
11/28 07:16:59.10 oraconn(9979)873 executeStmt: EXEC Stmt: [SELECT * FROM abc WHERE active=1 AND root_id='xyz' AND root_kind='fgjh' AND domain_id='121' AND service_id='GNG' AND service_kind='gfg' AND field1='Linux']
11/28 07:16:59.11 oraconn(9979)1575 fetchResultSet: Got 0 row(s) for the query. Time taken = 0.000 sec
TEST
TP
DS
|535EM
10-118-224-197
25
Y
Y
Y
2019-11-19T08:30:46Z
Demand
27
73
V
GC
CA
000
C
SUCCESS
<ATTACH_NAME>TS</ATTACH_NAME>
<ATTACH_DESC>lts</ATTACH_DESC>
<ATTACH_INFO>LTS</ATTACH_INFO>
</ATTACHMENT>
Well your data has neither the string request, nor the string response, nor any XML even remotely like that. It is impossible for anybody to help without a more clear explanation of what you need. Which line numbers should be thrown away? Which line numbers should be Event#1 and which Event#2?
Like this:
SHOULD_LINEMERGE = false
LINE_BREAKER = ((?:RegEx1Here)|(?:RegEx2Here))
Pick your sample and upload it in the Search-head UI as "add data".
Then you will have an editor to tweak your sourcetype props.conf and see the result live.
For linebreaking, read https://docs.splunk.com/Documentation/Splunk/8.0.0/Data/Configureeventlinebreaking
you probably need to put a proper regex in LINE_BREAKER for your xml format.
or if you already have a linebreaker, try to define a multiline grouping with BREAK_ONLY_BEFORE or MUST_BREAK_AFTER
It is easy to answer if you have a sample log.
How is it?