topic Re: how to index a json file ? in Getting Data In

how to index a json file ?

abilis — Thu, 17 May 2018 02:18:31 GMT

HI,

i am trying to index a local json file, but when going trough the sourcetype the predefined json source type is not reading the file properly..splunk put everything in one line...no detecting time format or something (see attached file)

this is an exemple inside the file

{
    "records": 
    [

        {
            "time": "2018-05-11T13:29:03Z",     
             "GatewayId": "4r566-5678-4753-968f-34568",
             "Region": "unknown",
              "operationName": "ApplicationGatewayAccess",
              "category": "ApplicationGatewayAccessLog",
            }
        ,
{
            "time": "2018-05-11T13:29:05Z",         
             "GatewayId": "4r566-ae57-dfg543-968f-xxx45t67",
             "Region": "unknown",
             "operationName": "ApplicationGatewayAccess",
             "category": "ApplicationGatewayAccessLog",
            }

can someone help me to figure this out ?

thanks for your support

Re: how to index a json file ?

MuS — Thu, 17 May 2018 04:19:49 GMT

If Splunk does not pick up the JSON event straight away, it is most likely not pure JSON.
Put your JSON events into any JSON validator to see if it is pure JSON.

cheers, MuS

Re: how to index a json file ?

MuS — Thu, 17 May 2018 04:21:13 GMT

But, looking at the screenshot this looks not too bad. What or where do you think it breaks or behaves badly?

Re: how to index a json file ?

xpac — Thu, 17 May 2018 04:55:12 GMT

I guess he/she wants it to be separate events, but the whole JSON is indexed as a single event. Right?

Re: how to index a json file ?

ansif — Thu, 17 May 2018 06:07:29 GMT

If the JSON response is from REST API call then I can help you with rest_ta response handler script.

Re: how to index a json file ?

abilis — Thu, 17 May 2018 13:30:45 GMT

i verified, the validator says json is valid, splunk is showing all records in one line with only one timestamp...i am expecting 4 lines

is this a time format error ?

Re: how to index a json file ?

abilis — Thu, 17 May 2018 13:31:39 GMT

the jason file is stored locally in splunk server to index once

Re: how to index a json file ?

abilis — Thu, 17 May 2018 13:42:57 GMT

yes, you are correct...i want separate events since they are at different times

Re: how to index a json file ?

abilis — Thu, 17 May 2018 19:37:22 GMT

i found that splunk is not indexing separate events because the json file starts with { and ends with } if i removed those character splunk will give me a line per event.

now the question is: how can i remove the { at the beginning and the } at the end with splunk before indexing?

thanks

Re: how to index a json file ?

poete — Fri, 18 May 2018 08:49:02 GMT

Hello. The pb is i the json file. Please remove the last comma of each record, and try again. For instance, based on your example :

{
    "records": [

        {
            "time": "2018-05-11T13:29:03Z",
            "GatewayId": "4r566-5678-4753-968f-34568",
            "Region": "unknown",
            "operationName": "ApplicationGatewayAccess",
            "category": "ApplicationGatewayAccessLog"
        },
        {
            "time": "2018-05-11T13:29:05Z",
            "GatewayId": "4r566-ae57-dfg543-968f-xxx45t67",
            "Region": "unknown",
            "operationName": "ApplicationGatewayAccess",
            "category": "ApplicationGatewayAccessLog"
        }

Re: how to index a json file ?

krishnapriya — Thu, 27 Jul 2023 18:41:47 GMT

Hi Have you found the answers to it. Even I am facing the same problem.