Getting Data In

Why are all of our JSON fields alphanumeric strings?

cdstealer
Contributor

Hi,
I'm ingesting data in pure json and all fields are being extracted. However, all fields are strings regardless of whether the field contains a float or an integer. Trying to convert the required fields in search has no effect, so we are unable to execute any arithmetic or numeric comparisons. Is there anything that can be done to have the correct type at indextime?

Cheers
Steve

0 Karma

DalJeanis
SplunkTrust
SplunkTrust

Please post your .conf stanzas for that sourcetype. Your JSON works just fine in spath. Run anywhere code follows.

| makeresults | eval _raw="[{\"total\": 40848933, \"name\": \"DR_SP_OBOFF_IFX_VGINFADMIN\", \"system\": null, \"snapshots\": 0, \"volumes\": 40848933, \"data_reduction\": 5.5917148342819205, \"size\": 75161927680, \"shared_space\": null, \"thin_provisioning\": 0.9629603181566511, \"total_reduction\": 150.9655201124792}, {\"total\": 78303476810, \"name\": \"DR_SP_OBOFF_IFX_SYS01-1\", \"system\": null, \"snapshots\": 0, \"volumes\": 78303476810, \"data_reduction\": 3.0048772949588285, \"size\": 279172874240, \"shared_space\": null, \"thin_provisioning\": 0.13935706065251277, \"total_reduction\": 3.4914331572127146}, {\"total\": 1151894, \"name\": \"DR_SP_OBOFF_IFX_TMP01-1\", \"system\": null, \"snapshots\": 0, \"volumes\": 1151894, \"data_reduction\": 5.826071620538336, \"size\": 59055800320, \"shared_space\": null, \"thin_provisioning\": 0.12769851684570313, \"total_reduction\": 6.67896562490172}, {\"total\": 4194101563, \"name\": \"DR_SP_OBOFF_IFX_APP01-1\", \"system\": null, \"snapshots\": 0, \"volumes\": 4194101563, \"data_reduction\": 5.762749779233561, \"size\": 751619276800, \"shared_space\": null, \"thin_provisioning\": 0.3507057993752616, \"total_reduction\": 8.87540620829318}, {\"total\": 30325162691, \"name\": \"DR_SP_OBOFF_IFX_AUD01-1\", \"system\": null, \"snapshots\": 0, \"volumes\": 30325162691, \"data_reduction\": 5.7856689710564675, \"size\": 16492674416640, \"shared_space\": null, \"thin_provisioning\": 0.4147625645622611, \"total_reduction\": 9.886019965091556}, {\"total\": 26692006101, \"name\": \"DR_SP_OBOFF_IFX_FUN01-1\", \"system\": null, \"snapshots\": 0, \"volumes\": 26692006101, \"data_reduction\": 5.713406580985547, \"size\": 2684354560000, \"shared_space\": null, \"thin_provisioning\": 0.37600740242004393, \"total_reduction\": 9.156208908797916}, {\"total\": 74914950529, \"name\": \"DR_SP_OBOFF_IFX_VLAT01-1\", \"system\": null, \"snapshots\": 0, \"volumes\": 74914950529, \"data_reduction\": 5.810545839529534, \"size\": 21990232555520, \"shared_space\": null, \"thin_provisioning\": 0.44200734570622446, \"total_reduction\": 10.413301671298274}, {\"total\": 1150454538, \"name\": \"DR_SP_OBOFF_IFX_MGS-APP1-1\", \"system\": null, \"snapshots\": 0, \"volumes\": 1150454538, \"data_reduction\": 2.8930656374658144, \"size\": 36507222016, \"shared_space\": null, \"thin_provisioning\": 0.9088305866017061, \"total_reduction\": 31.732853482634717}, {\"total\": 635496861, \"name\": \"DR_SP_OBOFF_IFX_MGS-AUD1-1\", \"system\": null, \"snapshots\": 0, \"volumes\": 635496861, \"data_reduction\": 6.0887140231403905, \"size\": 74088185856, \"shared_space\": null, \"thin_provisioning\": 0.9477627519248188, \"total_reduction\": 116.55885881234325}, {\"total\": 3419466396, \"name\": \"DR_SP_OBOFF_IFX_MGS-FUN1-1\", \"system\": null, \"snapshots\": 0, \"volumes\": 3419466396, \"data_reduction\": 2.808043834085122, \"size\": 92341796864, \"shared_space\": null, \"thin_provisioning\": 0.8959998640903207, \"total_reduction\": 27.00038619683935}, {\"total\": 36900508384, \"name\": \"DR_SP_OBOFF_IFX_MGS-SYS1-1\", \"system\": null, \"snapshots\": 0, \"volumes\": 36900508384, \"data_reduction\": 3.252184366603066, \"size\": 148176371712, \"shared_space\": null, \"thin_provisioning\": 0.15868666552115177, \"total_reduction\": 3.8656042087073685}, {\"total\": 560836420, \"name\": \"DR_SP_OBOFF_IFX_MGS-TMP1-1\", \"system\": null, \"snapshots\": 0, \"volumes\": 560836420, \"data_reduction\": 5.952361806622659, \"size\": 74088185856, \"shared_space\": null, \"thin_provisioning\": 0.9549372023430424, \"total_reduction\": 132.09037423586636}, {\"total\": 1128669697, \"name\": \"DR_SP_OBOFF_IFX_MGS-VGINFADMIN\", \"system\": null, \"snapshots\": 0, \"volumes\": 1128669697, \"data_reduction\": 5.370497559256691, \"size\": 55834574848, \"shared_space\": null, \"thin_provisioning\": 0.24247801303863525, \"total_reduction\": 7.089559975413095}, {\"total\": 384092766229, \"name\": \"DR_SP_OBOFF_IFX_MGS-VLAT1-1\", \"system\": null, \"snapshots\": 0, \"volumes\": 384092766229, \"data_reduction\": 2.7056223050113544, \"size\": 1884416901120, \"shared_space\": null, \"thin_provisioning\": 0.44581108854027557, \"total_reduction\": 4.882129990448185}, {\"total\": 14591737, \"name\": \"DR_SP_OBOFF_IFX_MUDDY_VLAT01-1\", \"system\": null, \"snapshots\": 0, \"volumes\": 14591737, \"data_reduction\": 5.825555352683882, \"size\": 21990232555520, \"shared_space\": null, \"thin_provisioning\": 0.4441834176890552, \"total_reduction\": 10.481075120973713}, {\"total\": 12940424, \"name\": \"DR_SP_OBOFF_IFX_MUDDY_VGINFADMIN\", \"system\": null, \"snapshots\": 0, \"volumes\": 12940424, \"data_reduction\": 5.769282773109169, \"size\": 75161927680, \"shared_space\": null, \"thin_provisioning\": 0.9629402092524937, \"total_reduction\": 155.6749959117721}, {\"total\": 41327630331, \"name\": \"DR_SP_OBOFF_IFX_MUDDY_SYS01-1\", \"system\": null, \"snapshots\": 0, \"volumes\": 41327630331, \"data_reduction\": 4.196256297529051, \"size\": 279172874240, \"shared_space\": null, \"thin_provisioning\": 0.1393570459806002, \"total_reduction\": 4.875722595452125}, {\"total\": 2093944, \"name\": \"DR_SP_OBOFF_IFX_MUDDY_TMP01-1\", \"system\": null, \"snapshots\": 0, \"volumes\": 2093944, \"data_reduction\": 5.82486857314305, \"size\": 59055800320, \"shared_space\": null, \"thin_provisioning\": 0.12769844748757103, \"total_reduction\": 6.67758592927651}, {\"total\": 110567079, \"name\": \"DR_SP_OBOFF_IFX_MUDDY_APP01-1\", \"system\": null, \"snapshots\": 0, \"volumes\": 110567079, \"data_reduction\": 5.8216771539896355, \"size\": 751619276800, \"shared_space\": null, \"thin_provisioning\": 0.35301270621163505, \"total_reduction\": 8.998132126987265}, {\"total\": 11450502587, \"name\": \"DR_SP_OBOFF_IFX_MUDDY_FUN01-1\", \"system\": null, \"snapshots\": 0, \"volumes\": 11450502587, \"data_reduction\": 5.717953722895729, \"size\": 2684354560000, \"shared_space\": null, \"thin_provisioning\": 0.37651851806640624, \"total_reduction\": 9.171008103019716}, {\"total\": 8724590790, \"name\": \"DR_SP_OBOFF_IFX_MUDDY_AUD01-1\", \"system\": null, \"snapshots\": 0, \"volumes\": 8724590790, \"data_reduction\": 5.815905885628052, \"size\": 16492674416640, \"shared_space\": null, \"thin_provisioning\": 0.41740459154049553, \"total_reduction\": 9.982752697976865}, {\"total\": 187692578634, \"name\": \"PRC_CORP_UIM_MYSQL_DB1-1\", \"system\": null, \"snapshots\": 0, \"volumes\": 187692578634, \"data_reduction\": 2.8722118919761837, \"size\": 6597069766656, \"shared_space\": null, \"thin_provisioning\": 0.9182717454774926, \"total_reduction\": 35.14343856671008}, {\"total\": 55128374940, \"name\": \"PRN_MS_MARS_MYSQL_DATA-SLAVE1-1\", \"system\": null, \"snapshots\": 0, \"volumes\": 55128374940, \"data_reduction\": 2.887891129235254, \"size\": 187904819200, \"shared_space\": null, \"thin_provisioning\": 0.1473721149989537, \"total_reduction\": 3.387047479958634}, {\"total\": 0, \"name\": \"PRN_MS_MARS_MYSQL_SLAVE-BACKUP1-1\", \"system\": null, \"snapshots\": 0, \"volumes\": 0, \"data_reduction\": 1.0, \"size\": 279172874240, \"shared_space\": null, \"thin_provisioning\": 1.0, \"total_reduction\": 1.0}, {\"total\": 0, \"name\": \"PRN_MS_MARS_MYSQL_SLAVE-DATA1-1\", \"system\": null, \"snapshots\": 0, \"volumes\": 0, \"data_reduction\": 1.0, \"size\": 92341796864, \"shared_space\": null, \"thin_provisioning\": 1.0, \"total_reduction\": 1.0}, {\"total\": 0, \"name\": \"PRN_MS_MARS_MYSQL_SLAVE-LOGS01-1\", \"system\": null, \"snapshots\": 0, \"volumes\": 0, \"data_reduction\": 1.0, \"size\": 59055800320, \"shared_space\": null, \"thin_provisioning\": 1.0, \"total_reduction\": 1.0}, {\"total\": 0, \"name\": \"PRN_MS_MARS_MYSQL_SLAVE-QUORUM\", \"system\": null, \"snapshots\": 0, \"volumes\": 0, \"data_reduction\": 1.0, \"size\": 1073741824, \"shared_space\": null, \"thin_provisioning\": 1.0, \"total_reduction\": 1.0}, {\"total\": 0, \"name\": \"PRN_MS_MARS_MYSQL_SLAVE-TMP1-1\", \"system\": null, \"snapshots\": 0, \"volumes\": 0, \"data_reduction\": 1.0, \"size\": 37580963840, \"shared_space\": null, \"thin_provisioning\": 1.0, \"total_reduction\": 1.0}]"
| spath
| rename {}.data_reduction as DR
| table DR
| mvexpand DR 
| eval DR1 = DR + 1
0 Karma

niketn
Legend

@cdstealer, can you add a mocked up sample of the JSON data being idexed? Ideally numeric fields should be treated as numbers. Try trim() function in case numbers in JSON have spaces before or after i.e. " 30 "

____________________________________________
| makeresults | eval message= "Happy Splunking!!!"
0 Karma

cdstealer
Contributor

Hi @niketnilay, I've attached an image and sample data to the thread. Thanks for your help.

0 Karma
Get Updates on the Splunk Community!

Splunk APM & RUM | Upcoming Planned Maintenance

There will be planned maintenance of the streaming infrastructure for Splunk APM and Splunk RUM in the coming ...

Part 2: Diving Deeper With AIOps

Getting the Most Out of Event Correlation and Alert Storm Detection in Splunk IT Service Intelligence   Watch ...

User Groups | Upcoming Events!

If by chance you weren't already aware, the Splunk Community is host to numerous User Groups, organized ...