https://community.splunk.com/t5/Getting-Data-In/How-to-efficiently-push-data-to-Splunk/m-p/538889#M90291 <P>You probably could associate some INGEST_EVAL settings with the appropriate sourcetype or source so Splunk will automatically add fields to the events, but it's far easier to have the Python app continue to do it.</P> Sun, 07 Feb 2021 19:54:06 GMT richgalloway 2021-02-07T19:54:06Z https://community.splunk.com/t5/Getting-Data-In/How-to-efficiently-push-data-to-Splunk/m-p/538886#M90290 <P>Dear Splunk community,</P><P>I have a Python application that pushes data to Splunk every time is executed. Multiple events are pushed using JSON format. Only a subset of the data being sent, namely two fields are changing during job execution, the rest are constant per job execution (think of them as some sort of job metadata). I would like to have that metadata in splunk so I can filter it, but I do not like also pushing lots of identical data for each event. &nbsp;I guess what I am looking for is some sort of bulk tagging after each import where each job metadata field would be a label. &nbsp;<BR /><BR /></P><P>I appreciate any thoughts/suggestions how to do this usinng splunk BKMs.</P> Sun, 07 Feb 2021 15:10:45 GMT https://community.splunk.com/t5/Getting-Data-In/How-to-efficiently-push-data-to-Splunk/m-p/538886#M90290 Valentin 2021-02-07T15:10:45Z https://community.splunk.com/t5/Getting-Data-In/How-to-efficiently-push-data-to-Splunk/m-p/538889#M90291 <P>You probably could associate some INGEST_EVAL settings with the appropriate sourcetype or source so Splunk will automatically add fields to the events, but it's far easier to have the Python app continue to do it.</P> Sun, 07 Feb 2021 19:54:06 GMT https://community.splunk.com/t5/Getting-Data-In/How-to-efficiently-push-data-to-Splunk/m-p/538889#M90291 richgalloway 2021-02-07T19:54:06Z https://community.splunk.com/t5/Getting-Data-In/How-to-efficiently-push-data-to-Splunk/m-p/539422#M90365 <P>If I use INGEST_EVAL, is Splunk going to literally add those fields to each event, or is it going to do some internal JOIN? What I want to avoid is the fact that some metadata fields like `a-quite-long-string-that-really-does-not-change` is actually copied to each event.&nbsp;</P> Thu, 11 Feb 2021 08:29:02 GMT https://community.splunk.com/t5/Getting-Data-In/How-to-efficiently-push-data-to-Splunk/m-p/539422#M90365 Valentin 2021-02-11T08:29:02Z https://community.splunk.com/t5/Getting-Data-In/How-to-efficiently-push-data-to-Splunk/m-p/539463#M90372 <P>Yes, INGEST_EVAL adds the result as a field to each event.</P><P>Bear in mind that any field that is NOT in an event cannot be used as a filter on that event.</P> Thu, 11 Feb 2021 13:45:45 GMT https://community.splunk.com/t5/Getting-Data-In/How-to-efficiently-push-data-to-Splunk/m-p/539463#M90372 richgalloway 2021-02-11T13:45:45Z https://community.splunk.com/t5/Getting-Data-In/How-to-efficiently-push-data-to-Splunk/m-p/539537#M90386 <P>thank you. it seems I have no other choice than adding the metadata in the events.</P> Thu, 11 Feb 2021 17:28:59 GMT https://community.splunk.com/t5/Getting-Data-In/How-to-efficiently-push-data-to-Splunk/m-p/539537#M90386 Valentin 2021-02-11T17:28:59Z