how can i extract the below fields from my raw data through transforms and props.conf

[Sujithkumarkb]

I want to extract the below fields from my raw data and place it into a field .
How can i do it with transforms and props.conf and see the fields in interested fields.

Also how can i mask the values which are between these tags?

Can anyone please advice

Tags
"lt"AccountNum"gt" "lt"/AccountNum"gt"
Similarly, the below tags
CardSecVal /CardSecVal
BMLCustomerSSN /BMLCustomerSSN
CCAccountNum /CCAccountNum
ECPAccountDDA /ECPAccountDDA
OrbitalConnectionPassword /OrbitalConnectionPassword
OrbitalConnectionUsername /OrbitalConnectionUsername
CAVV /CAVV
AAV /AAV
EUDDIBAN /EUDDIBAN

[gcusello]

Hi Sujithkumarkb,
let me understand: when you say to use props.conf and transforms.conf, do you want to extract fields at index time?
This is good for searches, but it's an additional job for Indexers.
If instead you can extract fields at search time, you don't need transforms.conf and you can use the field extractor that helps you in extracting.

Anyway, to help you, I need an example of your raw data.
in addition isn't clear which are the fields to extract.

Bye.
Giuseppe

[Sujithkumarkb]

Yes , i want to extract fields at index time and also the existing values of the mentioned fields needs to be extracted as a field value pair and masked as it is sensitive information

Below raw data for reference .

D Thu Jul 18 01:35:22 2019 dalbrmap01xu dm:62936 /opt/app/BRM/workspace/HUM/HUM_APPLICATION/HUM_BRM/BRM_Build/7.5/verizon/source/sys/dm_orbital/dm_orbital_send.cpp:325 1:atlp0d:pin_collect:114:-316832:0:28113:0
?xml version="1.0" encoding="UTF-8"? Response
MessageType>AC/MessageType
MerchantID>2468/MerchantID
TerminalID>001/TerminalID
CardBrand>DI/CardBrand
AccountNum XXXXXXXXXXXX1855 /AccountNum
OrderID T1,3c27a,1 /OrderID

p.S i am able to fetch these fields with a search using rex , but not sure how to see this as a permanent field for index abc wrt below example
index=abc host="" source="/opt/app/7.5/var/dm_orbital/abcld_ral.inlog" | rex field=_raw "(?.+)<\/AccountNum>(.)"

I have removed the <> tags as i was not able to post the same here

[gcusello]

Hi Sujithkumarkb,
have you seen at https://docs.splunk.com/Documentation/Splunk/7.3.0/Data/Configureindex-timefieldextraction ?
Bye.
Giuseppe

[Sujithkumarkb]

Hi @gcusello ,
Thanks for the response .

i have went through the above link and implemented the below.
Props.conf
[source::/tmp/AccountNum-Check.csv]
REPORT-Hide_Account_num = Hide_Account_num
TRANSFORMS-= Hide_Account_num
Transforms.conf
[Hide_Account_num]
REGEX =_raw "(?.+)<\/AccountNum>(.*)"
FORMAT = AccountNum::$1
WRITE_META = true
fields.conf
[AccountNum]
INDEXED=false
INDEXED_VALUE=false

But the field AccountNum is poping up in sourcetype ,not sure why .
Can you please assist
Is the below regex for extracting account number Key-value fine?
AccounNum>value /AccountNum : | rex field=_raw "AccountNum(?AccountNum.+\/AccountNum(.*)"

[gcusello]

Hi Sujithkumarkb,
I usually prefer to use always sourcetype instead source, in this way I am sure to associate the fields to a sourcetype and always have results, using source, sometimes there are errors.

About the regex, it should be different, something like this:

REGEX = AccountNum\s+(?<AccountNum>[^ ]*)\s+\/AccountNum

other parameters are OK.

Bye.
Giuseppe