Why are all of our JSON fields alphanumeric strings?

cdstealer — Tue, 26 Sep 2017 08:54:41 GMT

Hi,
I'm ingesting data in pure json and all fields are being extracted. However, all fields are strings regardless of whether the field contains a float or an integer. Trying to convert the required fields in search has no effect, so we are unable to execute any arithmetic or numeric comparisons. Is there anything that can be done to have the correct type at indextime?

Cheers
Steve

Re: Why are all of our JSON fields alphanumeric strings?

niketn — Tue, 26 Sep 2017 08:59:53 GMT

@cdstealer, can you add a mocked up sample of the JSON data being idexed? Ideally numeric fields should be treated as numbers. Try trim() function in case numbers in JSON have spaces before or after i.e. " 30 "

Re: Why are all of our JSON fields alphanumeric strings?

cdstealer — Tue, 26 Sep 2017 09:23:57 GMT

Hi @niketnilay, I've attached an image and sample data to the thread. Thanks for your help.

Re: Why are all of our JSON fields alphanumeric strings?

DalJeanis — Wed, 27 Sep 2017 19:43:37 GMT

Please post your .conf stanzas for that sourcetype. Your JSON works just fine in spath. Run anywhere code follows.

| makeresults | eval _raw="[{\"total\": 40848933, \"name\": \"DR_SP_OBOFF_IFX_VGINFADMIN\", \"system\": null, \"snapshots\": 0, \"volumes\": 40848933, \"data_reduction\": 5.5917148342819205, \"size\": 75161927680, \"shared_space\": null, \"thin_provisioning\": 0.9629603181566511, \"total_reduction\": 150.9655201124792}, {\"total\": 78303476810, \"name\": \"DR_SP_OBOFF_IFX_SYS01-1\", \"system\": null, \"snapshots\": 0, \"volumes\": 78303476810, \"data_reduction\": 3.0048772949588285, \"size\": 279172874240, \"shared_space\": null, \"thin_provisioning\": 0.13935706065251277, \"total_reduction\": 3.4914331572127146}, {\"total\": 1151894, \"name\": \"DR_SP_OBOFF_IFX_TMP01-1\", \"system\": null, \"snapshots\": 0, \"volumes\": 1151894, \"data_reduction\": 5.826071620538336, \"size\": 59055800320, \"shared_space\": null, \"thin_provisioning\": 0.12769851684570313, \"total_reduction\": 6.67896562490172}, {\"total\": 4194101563, \"name\": \"DR_SP_OBOFF_IFX_APP01-1\", \"system\": null, \"snapshots\": 0, \"volumes\": 4194101563, \"data_reduction\": 5.762749779233561, \"size\": 751619276800, \"shared_space\": null, \"thin_provisioning\": 0.3507057993752616, \"total_reduction\": 8.87540620829318}, {\"total\": 30325162691, \"name\": \"DR_SP_OBOFF_IFX_AUD01-1\", \"system\": null, \"snapshots\": 0, \"volumes\": 30325162691, \"data_reduction\": 5.7856689710564675, \"size\": 16492674416640, \"shared_space\": null, \"thin_provisioning\": 0.4147625645622611, \"total_reduction\": 9.886019965091556}, {\"total\": 26692006101, \"name\": \"DR_SP_OBOFF_IFX_FUN01-1\", \"system\": null, \"snapshots\": 0, \"volumes\": 26692006101, \"data_reduction\": 5.713406580985547, \"size\": 2684354560000, \"shared_space\": null, \"thin_provisioning\": 0.37600740242004393, \"total_reduction\": 9.156208908797916}, {\"total\": 74914950529, \"name\": \"DR_SP_OBOFF_IFX_VLAT01-1\", \"system\": null, \"snapshots\": 0, \"volumes\": 74914950529, \"data_reduction\": 5.810545839529534, \"size\": 21990232555520, \"shared_space\": null, \"thin_provisioning\": 0.44200734570622446, \"total_reduction\": 10.413301671298274}, {\"total\": 1150454538, \"name\": \"DR_SP_OBOFF_IFX_MGS-APP1-1\", \"system\": null, \"snapshots\": 0, \"volumes\": 1150454538, \"data_reduction\": 2.8930656374658144, \"size\": 36507222016, \"shared_space\": null, \"thin_provisioning\": 0.9088305866017061, \"total_reduction\": 31.732853482634717}, {\"total\": 635496861, \"name\": \"DR_SP_OBOFF_IFX_MGS-AUD1-1\", \"system\": null, \"snapshots\": 0, \"volumes\": 635496861, \"data_reduction\": 6.0887140231403905, \"size\": 74088185856, \"shared_space\": null, \"thin_provisioning\": 0.9477627519248188, \"total_reduction\": 116.55885881234325}, {\"total\": 3419466396, \"name\": \"DR_SP_OBOFF_IFX_MGS-FUN1-1\", \"system\": null, \"snapshots\": 0, \"volumes\": 3419466396, \"data_reduction\": 2.808043834085122, \"size\": 92341796864, \"shared_space\": null, \"thin_provisioning\": 0.8959998640903207, \"total_reduction\": 27.00038619683935}, {\"total\": 36900508384, \"name\": \"DR_SP_OBOFF_IFX_MGS-SYS1-1\", \"system\": null, \"snapshots\": 0, \"volumes\": 36900508384, \"data_reduction\": 3.252184366603066, \"size\": 148176371712, \"shared_space\": null, \"thin_provisioning\": 0.15868666552115177, \"total_reduction\": 3.8656042087073685}, {\"total\": 560836420, \"name\": \"DR_SP_OBOFF_IFX_MGS-TMP1-1\", \"system\": null, \"snapshots\": 0, \"volumes\": 560836420, \"data_reduction\": 5.952361806622659, \"size\": 74088185856, \"shared_space\": null, \"thin_provisioning\": 0.9549372023430424, \"total_reduction\": 132.09037423586636}, {\"total\": 1128669697, \"name\": \"DR_SP_OBOFF_IFX_MGS-VGINFADMIN\", \"system\": null, \"snapshots\": 0, \"volumes\": 1128669697, \"data_reduction\": 5.370497559256691, \"size\": 55834574848, \"shared_space\": null, \"thin_provisioning\": 0.24247801303863525, \"total_reduction\": 7.089559975413095}, {\"total\": 384092766229, \"name\": \"DR_SP_OBOFF_IFX_MGS-VLAT1-1\", \"system\": null, \"snapshots\": 0, \"volumes\": 384092766229, \"data_reduction\": 2.7056223050113544, \"size\": 1884416901120, \"shared_space\": null, \"thin_provisioning\": 0.44581108854027557, \"total_reduction\": 4.882129990448185}, {\"total\": 14591737, \"name\": \"DR_SP_OBOFF_IFX_MUDDY_VLAT01-1\", \"system\": null, \"snapshots\": 0, \"volumes\": 14591737, \"data_reduction\": 5.825555352683882, \"size\": 21990232555520, \"shared_space\": null, \"thin_provisioning\": 0.4441834176890552, \"total_reduction\": 10.481075120973713}, {\"total\": 12940424, \"name\": \"DR_SP_OBOFF_IFX_MUDDY_VGINFADMIN\", \"system\": null, \"snapshots\": 0, \"volumes\": 12940424, \"data_reduction\": 5.769282773109169, \"size\": 75161927680, \"shared_space\": null, \"thin_provisioning\": 0.9629402092524937, \"total_reduction\": 155.6749959117721}, {\"total\": 41327630331, \"name\": \"DR_SP_OBOFF_IFX_MUDDY_SYS01-1\", \"system\": null, \"snapshots\": 0, \"volumes\": 41327630331, \"data_reduction\": 4.196256297529051, \"size\": 279172874240, \"shared_space\": null, \"thin_provisioning\": 0.1393570459806002, \"total_reduction\": 4.875722595452125}, {\"total\": 2093944, \"name\": \"DR_SP_OBOFF_IFX_MUDDY_TMP01-1\", \"system\": null, \"snapshots\": 0, \"volumes\": 2093944, \"data_reduction\": 5.82486857314305, \"size\": 59055800320, \"shared_space\": null, \"thin_provisioning\": 0.12769844748757103, \"total_reduction\": 6.67758592927651}, {\"total\": 110567079, \"name\": \"DR_SP_OBOFF_IFX_MUDDY_APP01-1\", \"system\": null, \"snapshots\": 0, \"volumes\": 110567079, \"data_reduction\": 5.8216771539896355, \"size\": 751619276800, \"shared_space\": null, \"thin_provisioning\": 0.35301270621163505, \"total_reduction\": 8.998132126987265}, {\"total\": 11450502587, \"name\": \"DR_SP_OBOFF_IFX_MUDDY_FUN01-1\", \"system\": null, \"snapshots\": 0, \"volumes\": 11450502587, \"data_reduction\": 5.717953722895729, \"size\": 2684354560000, \"shared_space\": null, \"thin_provisioning\": 0.37651851806640624, \"total_reduction\": 9.171008103019716}, {\"total\": 8724590790, \"name\": \"DR_SP_OBOFF_IFX_MUDDY_AUD01-1\", \"system\": null, \"snapshots\": 0, \"volumes\": 8724590790, \"data_reduction\": 5.815905885628052, \"size\": 16492674416640, \"shared_space\": null, \"thin_provisioning\": 0.41740459154049553, \"total_reduction\": 9.982752697976865}, {\"total\": 187692578634, \"name\": \"PRC_CORP_UIM_MYSQL_DB1-1\", \"system\": null, \"snapshots\": 0, \"volumes\": 187692578634, \"data_reduction\": 2.8722118919761837, \"size\": 6597069766656, \"shared_space\": null, \"thin_provisioning\": 0.9182717454774926, \"total_reduction\": 35.14343856671008}, {\"total\": 55128374940, \"name\": \"PRN_MS_MARS_MYSQL_DATA-SLAVE1-1\", \"system\": null, \"snapshots\": 0, \"volumes\": 55128374940, \"data_reduction\": 2.887891129235254, \"size\": 187904819200, \"shared_space\": null, \"thin_provisioning\": 0.1473721149989537, \"total_reduction\": 3.387047479958634}, {\"total\": 0, \"name\": \"PRN_MS_MARS_MYSQL_SLAVE-BACKUP1-1\", \"system\": null, \"snapshots\": 0, \"volumes\": 0, \"data_reduction\": 1.0, \"size\": 279172874240, \"shared_space\": null, \"thin_provisioning\": 1.0, \"total_reduction\": 1.0}, {\"total\": 0, \"name\": \"PRN_MS_MARS_MYSQL_SLAVE-DATA1-1\", \"system\": null, \"snapshots\": 0, \"volumes\": 0, \"data_reduction\": 1.0, \"size\": 92341796864, \"shared_space\": null, \"thin_provisioning\": 1.0, \"total_reduction\": 1.0}, {\"total\": 0, \"name\": \"PRN_MS_MARS_MYSQL_SLAVE-LOGS01-1\", \"system\": null, \"snapshots\": 0, \"volumes\": 0, \"data_reduction\": 1.0, \"size\": 59055800320, \"shared_space\": null, \"thin_provisioning\": 1.0, \"total_reduction\": 1.0}, {\"total\": 0, \"name\": \"PRN_MS_MARS_MYSQL_SLAVE-QUORUM\", \"system\": null, \"snapshots\": 0, \"volumes\": 0, \"data_reduction\": 1.0, \"size\": 1073741824, \"shared_space\": null, \"thin_provisioning\": 1.0, \"total_reduction\": 1.0}, {\"total\": 0, \"name\": \"PRN_MS_MARS_MYSQL_SLAVE-TMP1-1\", \"system\": null, \"snapshots\": 0, \"volumes\": 0, \"data_reduction\": 1.0, \"size\": 37580963840, \"shared_space\": null, \"thin_provisioning\": 1.0, \"total_reduction\": 1.0}]"
| spath
| rename {}.data_reduction as DR
| table DR
| mvexpand DR 
| eval DR1 = DR + 1

topic Re: Why are all of our JSON fields alphanumeric strings? in Getting Data In

Why are all of our JSON fields alphanumeric strings?

Re: Why are all of our JSON fields alphanumeric strings?

Re: Why are all of our JSON fields alphanumeric strings?

Re: Why are all of our JSON fields alphanumeric strings?