Props.conf Extractions Not Applied to Saved Search (collect) Output in New Index

[zksvc]

Hi All, 

i do create new index but the source data is from savedsearch let say i create savedsearch from index=ABC then the output i sent to index=BCD 

then in index=BCD have result _raw field from index=ABC, like this.
all the fields from index=ABC all is gone

Then when i try to create regex using SPL query all fields was created

index=zake-alert sourcetype=alert:access 
| rex field=_raw "ComputerName:\s+\"?(?<ComputerName>[^\",\r\n]+)" 
| rex field=_raw "Locked Out User\s*ID:\s+\"?(?<LockedOutUserID>[^\",\r\n]+)" 
| rex field=_raw "User Full Name:\s+\"?(?<UserFullName>[^\"\r\n]*)" 
| rex field=_raw "Source of Lockout:\s+\"?(?<SourceOfLockout>[^\",\r\n]+)" 
| rex field=_raw "RecordNumber:\s+\"?(?<RecordNumber>\d+)" 
| rex field=_raw "Raw_Message:\s+\"?(?<Raw_Message>[^\r\n\"]+)" 
| eval mitre_tactic_id="TA0006", mitre_technique_id="T1110" 
| table _time ComputerName LockedOutUserID UserFullName SourceOfLockout RecordNumber Raw_Message mitre_tactic_id mitre_technique_id

But when i set up using props.conf nothing happened... 

Can someone explain me this ? 

[livehybrid]

Hi @zksvc 

Please can you share your props.conf ? Have you configured the props to extract the fields from both the original sourcetype and collected sourcetype name?

🌟 Did this answer help you? If so, please consider:

• Adding karma to show it was useful
• Marking it as the solution if it resolved your issue
• Commenting if you need any clarification

Your feedback encourages the volunteers in this community to continue contributing

[gcusello]

Hi @zksvc ,

let me understand: you copy a subset of data from index=ABC using the collect command, is this correct?

if this is your approach, check the sourcetype that you have in DEF index because, by default using collect command, Splunk assign a sourcetype called stash and not the original one.

If you want to use the original sourcetype also in the DEF index you have to pay twice the license.

Then about regexes, the collect command saves the fields you have in the scheduled search (can you share it?) and anyway it modifies the original raw events, so probably you should create new field extractions.

Maybe a better approach is to extract all the fields in the savedsearch and in the secondary index use the sourcetype=stash (to avoid to pay twice the license) and you already have the fields you need.

Ciao.

Giuseppe

[zksvc]

Hi @gcusello 

I use adaptive response, and choose "Log Event" to send data from ABC to DEF

but i use this new sourcetype to regex it using props.conf 

[gcusello]

Hi @zksvc ,

for my knowledge, using a new:sourcetype you pay license even if in an Adaptive Response.

Anyway, see the output of the Adaptive Response to see how the log is build, so you can check your regexes, probably they are different than the original logs.

Ciao.

Giuseppe

[zksvc]

Hi @gcusello 

What do you mean license? that is different license like ingest size license ? 

Yes the log is different from the original one, but when i try to call it using regex SPL all is good. The trouble is when i try to parsing it using props.conf that is no impact

[gcusello]

Hi @zksvc ,

about license, for my knowledge, if you save search results in an index using collect and using a sourcetypoe different than stash, you have to pay the license.

About regexes, I suppose that you created them using the events in the DEF index, what do you mean that you want to parse events using props.conf? all the field extraction, also the ones done by interface, are stored in props.conf.

Ciao.

Giuseppe