Getting Data In

Second index with fewer fields

tnkoehn
Path Finder

I have large CDR records that are going to one index (let's call it index1). I need a second index (let's call it index2) that only uses a subset of the fields in the CDR records without messing up index1. For example, a CDR record looks like:

field1|field2|field3|...............|field200

I want index2 to only have these fields:

field20,field30,field33,field101,field150,field172

I know I can get these fields using regex in transforms.conf but just need to know (if possible) what to do in props and transforms to get these fields in index2.

The current props.conf looks like:

[index1]
REPORT-parsefields = index1_parse

The current transforms.conf looks like:

[index1_parse]
DELIMS = "|"
FIELDS = "field1", "field2", "field3",......."field200"
Tags (1)
0 Karma

martin_mueller
SplunkTrust
SplunkTrust

You could set up indexer1 to forward this data to indexer2 on top of indexing it itself. Then tell indexer2 to apply a huge SEDCMD to the data that strips out the first nineteen fields, keeps the 20th, strips out nine, keeps the 30th, and so on.

0 Karma

tnkoehn
Path Finder

Can you give me quick example of what my conf files would look like?

0 Karma
Got questions? Get answers!

Join the Splunk Community Slack to learn, troubleshoot, and make connections with fellow Splunk practitioners in real time!

Meet up IRL or virtually!

Join Splunk User Groups to connect and learn in-person by region or remotely by topic or industry.

Get Updates on the Splunk Community!

Your Path to AgenticOps: AI Experiences for Every Splunk Practitioner

Your Path to AgenticOps: AI Experiences for Every Splunk Practitioner   Watch on demand us for a demo-driven ...

Splunk Enterprise Security: An Automated Phishing Response Workflow

Originally posted on Splunk.com.   Key takeaways  The volume of phishing emails can overwhelm even the most ...

Sharper ES Detections, Agentic AI Integrations, Smarter Workload Management, and More ...

Splunk Lantern is Splunk’s customer success center that provides practical guidance from Splunk experts on key ...