Getting Data In

Why in my ingest_time lookup - field does not show up?

PaulPanther
Motivator

Hello colleagues,

we've implemented the ingest_time lookups but unfortunately the expected field from the configured csv-lookup does not show up in our searches.

Following implementation steps were executed:

1. props.conf & transforms.conf prepared and stored under $SPLUNK_HOME/etc/system/local on all indexer nodes within the cluster.
2. index_lookup.csv prepared and stored under $SPLUNK_HOME/etc/system/lookups on all indexer nodes within the cluster.
3. Rolling restart of the nodes
4. fields.conf prepared and deployed via SHD to our SHs

props.conf:

[aws:cloudwatch]
TRANSFORMS-define_index = define_rds_index

transforms.conf:

[define_rds_index]
INGEST_EVAL = test_index=json_extract(lookup("index_lookup.csv", json_object("account_id", account_id), json_array(index_tag)),"index_tag")

index_lookup.csv:

account_id index_tag
886089063862 index_platform-sandbox-dev

 

fields.conf:

[test_index]
INDEXED = True

 

Has anyone an idea if we missed a step or something is misconfigured? Thank you very much!

 

 

 

Labels (1)
0 Karma

bapun18
Communicator

Check the permissions of the lookup, if you have read permissions to the lookup, also check if this lookup has permission to view across apps.

0 Karma

PaulPanther
Motivator

The lookup is set up on the indexer nodes.  Read and write permissions are  set.

0 Karma

somesoni2
Revered Legend

See if this makes any difference:

transforms.conf (adding double quotes around output field):

[define_rds_index]
INGEST_EVAL = test_index=json_extract(lookup("index_lookup.csv", json_object("account_id", account_id), json_array("index_tag")),"index_tag")
0 Karma

PaulPanther
Motivator

@somesoni2 Thanks for your answer! Unfortunately it doesn't solve it.

Here is an example event:

{"Average": 4.975, "Maximum": 5.0, "Minimum": 4.95, "SampleCount": 2.0, "Sum": 9.95, "Unit": "Percent", "account_id": "886089063862", "metric_dimensions": "DBInstanceIdentifier=[aric664tyxxx]", "metric_name": "CPUUtilization", "period": 120, "timestamp": "2022-03-31T15:48:00Z"}
0 Karma

somesoni2
Revered Legend

Can you confirm (via running btool on your sourcetype) if you've INDEXED_EXTRACTION=json?

 

0 Karma

PaulPanther
Motivator

Yes, correct. The parameter comes with the Splunk_TA_aws.

Tags (2)
0 Karma

PaulPanther
Motivator

@somesoni2 Do we must disable the INDEXED_EXTRACTIONS and apply manually all parsing & formatting parameter?

0 Karma
Got questions? Get answers!

Join the Splunk Community Slack to learn, troubleshoot, and make connections with fellow Splunk practitioners in real time!

Meet up IRL or virtually!

Join Splunk User Groups to connect and learn in-person by region or remotely by topic or industry.

Get Updates on the Splunk Community!

Announcing Modern Navigation: A New Era of Splunk User Experience

We are excited to introduce the Modern Navigation feature in the Splunk Platform, available to both cloud and ...

Modernize your Splunk Apps – Introducing Python 3.13 in Splunk

We are excited to announce that the upcoming releases of Splunk Enterprise 10.2.x and Splunk Cloud Platform ...

Step into “Hunt the Insider: An Splunk ES Premier Mystery” to catch a cybercriminal ...

After a whole week of being on call, you fell asleep on your keyboard, and you hit a sequence of buttons that ...