How to remove the Auto extraction fields to create a new field extraction

[Dharani]

Hey folks, 

I need help on field extraction.

I have index=abs, source =123. When I search this in the Splunk, I can see some fields are auto extracted.

for example , 

session=12345, 

Status=Success,NA,NA.

In this case , I tried to create a new field extraction on top of this, but it was not working. I don't want that "Status=Success,NA,NA." I need to separate it as "Status=Success" ", Exception=NA", "SubAPITime=NA".

when I tried to create a new field in search level (UI) and Index level(Index Cluster) , it is not working.

I hope, we have to first remove the existing fields and then create our new field extraction.

Please help me on this one !

props.conf for new field extraction which is not working :

[source::123]
TRANSFORMS-extract-app_rewards = rewards_qual

transforms.conf

[rewards_qual]
SOURCE_KEY = MetaData:Source
REGEX = ^(?P<SessionId>[^,]+),(?P<User>[^,]+),(?P<DateTime>[^,]+),(?P<View>[^,]+),(?P<AppliedFilters>[^,]+),(?P<Status>[^,]+),(?P<Exception>[^,]+),(?P<SubAPITime>[^,]+),(?P<SubAPIName>[^,]+),(?P<TransactionId>[^,]+),(?P<HANATime>[^,]+),(?P<TotalTime>.+)
FORMAT = SessionId::$1 User::$2 DateTime::$3 View::$4 AppliedFilters::$5 Status::$6 Exception::$7 SubAPITime::$8 SubAPIName::$1 TransactionId::$1 HANATime::$1 TotalTime::$1
WRITE_META = true

I tried with source and source type as well.

Thanks,

Dharani.

[scelikok]

Hi @Dharani,

Can you please post a sample event?

[Dharani]

Sure.

Sample event:

3BAA7FA81A4486A656489C,aksaware,2021-01-22T05:35:51,Curate create(SFDC),opportunityType::TS#savmGroupName::THERMO  SCIENTIFIC  PRIVATE LTD#opportunityName::RNW_SY Video TeleCraft_TS#opptyOwner::aksaware#lookupBy::true#expectedProspectBookingDate::2021-01-22#sfdcAccountId::0013400001LFlomAAD#partyId::264252765#transactionID::aksaware_20210121213430#,Success,N/A,2021-01-22T05:36:09.285Z,deleteLookUpCurate,aksaware_20210121213430,19,18659

[scelikok · Accepted solution]

Hi @Dharani,

Your extraction regex in transform is set to work on source value. It should be _raw, also since you are using captured groups in your regex, you don't need FORMAT. Please try below;

[rewards_qual]
REGEX = ^(?P<SessionId>[^,]+),(?P<User>[^,]+),(?P<DateTime>[^,]+),(?P<View>[^,]+),(?P<AppliedFilters>[^,]+),(?P<Status>[^,]+),(?P<Exception>[^,]+),(?P<SubAPITime>[^,]+),(?P<SubAPIName>[^,]+),(?P<TransactionId>[^,]+),(?P<HANATime>[^,]+),(?P<TotalTime>.+)
WRITE_META = true

If this reply helps you an upvote is appreciated.

[Dharani]

Hey , 

I gave it a time. Now I am getting "rewards_qual = the first set of string in _raw log"

(EX: rewards_qual=3BAA7FA6DD578008A81A4486A656489C) 

Not getting all the fields in regex. But the good thing is the old fields are gone now 🙂 

Thanks!

[Dharani]

Hi @scelikok ,

Thank you for your response, 

I tried this, but not working. Still I am having that old fields, the new ones which we pushed now are not showing off.