All Apps and Add-ons

Response filter pattern doesn't work for this case

rsokolova
Path Finder

Thanks in advance... I have an issues where the response filter is not working which is creating duplicates. I have a json response where there is need to check multiple values against the indexed data. Already checked and the regex is matching the raw data from the indexer and from the API call.

We need to check all the markets and the count by market.

Here is the regex passed in the Response Filter Pattern :

\"([A-Z][A-Z][A-Z]|[0-9]+)\"

Raw data:

{  "Orders" : [ {    "SourceApplication" : "CC",    "Market" : "GRP",    "OrderDate" : "2019-05-01T00:00:00.000-04:00",    "Count" : "25"  }, {    "SourceApplication" : "CC",    "Market" : "HOU",    "OrderDate" : "2019-05-01T00:00:00.000-04:00",    "Count" : "8"  }, {    "SourceApplication" : "CC",    "Market" : "SAA",    "OrderDate" : "2019-05-01T00:00:00.000-04:00",    "Count" : "14"  }, {    "SourceApplication" : "EDI",    "Market" : "LOU",    "OrderDate" : "2019-05-01T00:00:00.000-04:00",    "Count" : "1"  }, {    "SourceApplication" : "PA",    "Market" : "LOU",    "OrderDate" : "2019-05-01T00:00:00.000-04:00",    "Count" : "312"  } ]}

Data from API:

{
    "Orders": [
        {
            "SourceApplication": "CC",
            "Market": "GRP",
            "OrderDate": "2019-05-01T00:00:00.000-04:00",
            "Count": "25"
        },
        {
            "SourceApplication": "CC",
            "Market": "HOU",
            "OrderDate": "2019-05-01T00:00:00.000-04:00",
            "Count": "8"
        },
        {
            "SourceApplication": "CC",
            "Market": "SAA",
            "OrderDate": "2019-05-01T00:00:00.000-04:00",
            "Count": "14"
        },
        {
            "SourceApplication": "EDI",
            "Market": "LOU",
            "OrderDate": "2019-05-01T00:00:00.000-04:00",
            "Count": "1"
        },
        {
            "SourceApplication": "PA",
            "Market": "LOU",
            "OrderDate": "2019-05-01T00:00:00.000-04:00",
            "Count": "312"
        }
    ]
}
0 Karma

mhoogcarspel_sp
Splunk Employee
Splunk Employee

Response filter just filters out responses that don't match the regex, and you regex is fine for that.

Deduping at indextime against data already indexed isn't really possible, unless you can use some sort of checkpointing mechanism (a-la DBX rising column for example).

You might be able to use the token function of this addon for some sort of checkpointing.

0 Karma
Get Updates on the Splunk Community!

Introducing Ingest Actions: Filter, Mask, Route, Repeat

WATCH NOW Ingest Actions (IA) is the best new way to easily filter, mask and route your data in Splunk® ...

Splunk Forwarders and Forced Time Based Load Balancing

Splunk customers use universal forwarders to collect and send data to Splunk. A universal forwarder can send ...

NEW! Log Views in Splunk Observability Dashboards Gives Context From a Single Page

Today, Splunk Observability releases log views, a new feature for users to add their logs data from Splunk Log ...