Hi,
In my data, I have two kinds of XML.
i.e. Request & Response
I want to break the log when my starts and ends with
also when starts and ends with
.
Can someone help to achieve this?
Below is my sample log data.. Out of it I only want the Request & Response XML information to be indexed and other should not indexed or shouldn't be visible in my search index result
11/28 07:20:20.31 do_ta(99)5733 syncWithNS: End syncing cache with NS
11/28 07:20:20.31 fgdf(99)556 collectIORs: HostNode:[hfeihf] LastTimeIORsCollected=1574925420, cur=1574925620. Diff = -200. Min = 240. So not collecting now.
11/28 07:20:20.31 fgdf(99)556 collectIORs: HostNode:[hfeihf] LastTimeIORsCollected=1574925420, cur=1574925620. Diff = -200. Min = 240. So not collecting now.
11/28 07:20:20.31 fgdf(99)556 collectIORs: HostNode:[hfeihf] LastTimeIORsCollected=1574925420, cur=1574925620. Diff = -200. Min = 240. So not collecting now.
11/28 07:20:20.31 fgdf(99)556 collectIORs: HostNode:[hfeihf] LastTimeIORsCollected=1574925420, cur=1574925620. Diff = -200. Min = 240. So not collecting now.
11/28 07:20:20.31 fgdf(99)556 collectIORs: HostNode:[feihf] LastTimeIORsCollected=1574925420, cur=1574925620. Diff = -200. Min = 240. So not collecting now.
11/28 07:16:59.10 oraconn(9979)873 executeStmt: EXEC Stmt: [SELECT * FROM abc WHERE active=1 AND root_id='xyz' AND root_kind='fgjh' AND domain_id='1213' AND domain_kind='dfsd' AND service_id='GG' AND service_kind='gfg' AND field1='Linux']
11/28 07:16:59.11 oraconn(9979)1575 fetchResultSet: Got 0 row(s) for the query. Time taken = 0.000 sec
11/28 07:16:59.08 oraconn(9979)873 executeStmt: EXEC Stmt: [SELECT * FROM abc WHERE active=1 AND root_id='xyz' AND root_kind='fgjh' AND domain_id='1213' AND domain_kind='dfsd' AND service_id='GWRNG' AND service_kind='gfg' AND field1='Linux']
11/28 07:16:59.08 oraconn(9979)1575 fetchResultSet: Got 0 row(s) for the query. Time taken = 0.000 sec
11/27 10:42:48.49 dfd(131)10151 comd::checktest KEY: aa:A78900007-, bb:76000009, cc:VS, REQID:0,
|123H
FT
TE
DS
10-118-224-197.
V
2019-11-27T10:42:15Z
Y
Y
Y
11/28 07:20:20.31 fgdf(9979)556 collectIORs: HostNode:[DNA4-aff] LastTimeIORsCollected=1574925420, cur=1574925620. Diff = -200. Min = 240. So not collecting now.
11/28 07:16:59.10 oraconn(9979)873 executeStmt: EXEC Stmt: [SELECT * FROM abc WHERE active=1 AND root_id='xyz' AND root_kind='fgjh' AND domain_id='121' AND service_id='GNG' AND service_kind='gfg' AND field1='Linux']
11/28 07:16:59.11 oraconn(9979)1575 fetchResultSet: Got 0 row(s) for the query. Time taken = 0.000 sec
TEST
TP
DS
|535EM
10-118-224-197
25
Y
Y
Y
2019-11-19T08:30:46Z
Demand
27
73
V
GC
CA
000
C
SUCCESS
<ATTACH_NAME>TS</ATTACH_NAME>
<ATTACH_DESC>lts</ATTACH_DESC>
<ATTACH_INFO>LTS</ATTACH_INFO>
</ATTACHMENT>
Well your data has neither the string request
, nor the string response
, nor any XML
even remotely like that. It is impossible for anybody to help without a more clear explanation of what you need. Which line numbers should be thrown away? Which line numbers should be Event#1 and which Event#2?
Like this:
SHOULD_LINEMERGE = false
LINE_BREAKER = ((?:RegEx1Here)|(?:RegEx2Here))
Pick your sample and upload it in the Search-head UI as "add data".
Then you will have an editor to tweak your sourcetype props.conf and see the result live.
For linebreaking, read https://docs.splunk.com/Documentation/Splunk/8.0.0/Data/Configureeventlinebreaking
you probably need to put a proper regex in LINE_BREAKER for your xml format.
or if you already have a linebreaker, try to define a multiline grouping with BREAK_ONLY_BEFORE or MUST_BREAK_AFTER
It is easy to answer if you have a sample log.
How is it?