KVStore Troubleshooting

[samhodgson]

Hi,

Seeing some very strange behaviour when writing to the KVStore, the following works:

            var record = {"projectId": projID,"projName": projName};
            console.log(tokens);
            console.log(keyid);
            console.log(record);
            service.del("storage/collections/data/topas4/")
            service.request(
            "storage/collections/data/topas4/",
            "POST",
            null,
            null,
            JSON.stringify(record),
            {"Content-Type": "application/json"},
            null);

When I change:

var record = {"projID": projID,"projName": projName};

to be:

var record = {"projectId": projID,"projName": projName};

The write appears to work however an inputlookup shows only the projName field. If I do an inputlookup | fields projName,projectId the projectId field is blank.

Started splunk in debug mode and the mongo log doesnt show anything, the splunkd.log shows that it appears to write the correct data:

07-27-2017 11:01:55.412 +0100 DEBUG CollectionHandler - DISPATCH::DATA method='POST' collection='topas4' key=''
07-27-2017 11:01:55.413 +0100 DEBUG AuthenticationManagerSplunk - Getting info for user: admin
07-27-2017 11:01:55.413 +0100 DEBUG PropertiesMapConfig - Pattern 'topas4' matches with priority 100
07-27-2017 11:01:55.413 +0100 DEBUG KVStorageProvider - Insert data: {"projectId":"262","projName":"Account Management"} -- { "projectId" : "262", "projName" : "Account Management" }
07-27-2017 11:01:55.413 +0100 DEBUG MongoClient - Pop new connection with type 8
07-27-2017 11:01:55.413 +0100 DEBUG MongoClient - Auto -> ReplicaSet
07-27-2017 11:01:55.414 +0100 ERROR mongodlog - 2017-07-27T10:01:55.414Z I NETWORK [initandlisten] connection accepted from 127.0.0.1:36426 #22 (14 connections now open)
07-27-2017 11:01:55.439 +0100 ERROR mongodlog - 2017-07-27T10:01:55.439Z I ACCESS [conn22] Successfully authenticated as principal __system on local
07-27-2017 11:01:55.440 +0100 DEBUG MongoClient - Pushing back connection with type 4

Makes me think my inputlookup is incorrect however I thought that:

| inputlookup topas4_lookup

Would show the entire contents?

Even if i do:

| inputlookup topas4_lookup | eval KeyID = _key | fields KeyID,projName,projectId

I see the key but the the projectId field is still blank.

Any ideas on what might be causing this or how to further troubleshoot it would do wonders for my sanity.

Cheers

Sam

[samhodgson · Accepted solution]

fields_list in transforms.conf dammit! totally my bad...

[samhodgson]

So it appears that using (lowercase d in projId):

var record = {"projId": projID,"projName": projName};

also fails.

Changing back to projID works!

Is someone going to tell me I wasted my entire morning on this because for some bizarre reason the key name MUST be identical to the variable name being passed and any failures surrounding this will be logged absolutely nowhere?

What madness is this? I must be doing something wrong here surely...