Dashboards & Visualizations

parse xml that are more than 900K bytes

SplunkCSIT
Communicator

I got alot of more than 900K bytes xml file, i just want to index the first few tags of file within the xml. If my xml file is just less than 20K bytes, the below transforms able to work. But if the file is more than 50K bytes, the below transforms will not work. Any other alternative? The one alternative that i can think of is to write a batch script to remove the body and the content tag from the xml file before it go thru the splunk. I dont really know how to write a batch script. Any other easier suggestion? thks

props.conf

[xmlFilter1]

KV_MODE = xml
BREAK_ONLY_BEFORE = <xml>
SHOULD_LINEMERGE = false
NO_BINARY_CHECK = 1
pulldown_type = 1
TRUNCATE = 1000000000000
MAX_EVENTS = 1000000000000
TRANSFORMS-test1 = body, content

transforms.conf

[body]
LOOKAHEAD = 1000000000000
SOURCE_KEY=_raw
REGEX=(.*?)\<body\>.*?\</body\>(.*)
DEST_KEY=_raw
FORMAT=$1<body>####</body>$2

[content]

LOOKAHEAD = 1000000000000
SOURCE_KEY=_raw
REGEX=(.*?)\<content\>.*?\</content\>(.*)
DEST_KEY=_raw
FORMAT=$1<content>*******</content>$2
Tags (1)
0 Karma

SplunkCSIT
Communicator

Hi,
You can ignore the above xml, this is the sample xml. I wish to mask the content for and tag, what is the recommended regex? thks

123 456 Not to be forward to indexer 333 not to be forwardthis is to be validfsffggetewrwerwerwewff
1.tetette. 2rerere ererererer3.erererefr 23/4/2014 12:23:232 23/4/2014 12:24:232

0 Karma

lguinn2
Legend

One suggestion: in props.conf, set TRUNCATE = 0

0 Karma

SplunkCSIT
Communicator

Hi,
You can ignore the above xml, this is the sample xml. I wish to mask the content for and tag, what is the recommended regex? thks

123 456 Not to be forward to indexer 333 not to be forwardthis is to be validfsffggetewrwerwerwewff
1.tetette. 2rerere ererererer3.erererefr 23/4/2014 12:23:232 23/4/2014 12:24:232

0 Karma

lguinn2
Legend

It would help if we could see a bit of the file (you could remove the content and just show the tags with a little gibberish between them).

0 Karma
Get Updates on the Splunk Community!

Introduction to Splunk Observability Cloud - Building a Resilient Hybrid Cloud

Introduction to Splunk Observability Cloud - Building a Resilient Hybrid Cloud  In today’s fast-paced digital ...

Observability protocols to know about

Observability protocols define the specifications or formats for collecting, encoding, transporting, and ...

Take Your Breath Away with Splunk Risk-Based Alerting (RBA)

WATCH NOW!The Splunk Guide to Risk-Based Alerting is here to empower your SOC like never before. Join Haylee ...