What is the best way to get field from raw data?

[Marttin123]

how can i get date data in fields to use the splunk features.

What is the best way for it. The raw data are coming from a kafka stream:

e.g.:

• MAX_CURRENT
• BLOCK_POSITION_NUMBER
• OPERATING_DISTANCE_KM
  
  row data:

{"body":"{\"namespace\":\"xxx.messages.sensor.sensortelemetrymessage\",\"payload\":{\"dataSource\":{\"endpointUrl\":\"opc.tcp://123.164.72.6:4840/\",\"id\":\"\",\"name\":\"\",\"route\":\"\"},\"data\":{\"key\":\"ns=3;s=\\\"TEST_CARRIER_DB\\\".\\\"CARRIER\\\"[105]\",\"value\":[{\"key\":\"TIMESTAMP\",\"value\":\"0001-01-01T00:00:00Z\",\"dataType\":\"DateTime\"},{\"key\":\"PLC_CARRIER\",\"value\":[{\"key\":\"OPERATION_MODE\",\"value\":\"0\",\"dataType\":\"Int16\"},{\"key\":\"TEMP_CABINET\",\"value\":\"0\",\"dataType\":\"Int16\"},{\"key\":\"F_PROG_SIG\",\"value\":[{\"key\":\"0\",\"value\":\"32\",\"dataType\":\"Byte\"},{\"key\":\"1\",\"value\":\"32\",\"dataType\":\"Byte\"},{\"key\":\"2\",\"value\":\"32\",\"dataType\":\"Byte\"},{\"key\":\"3\",\"value\":\"32\",\"dataType\":\"Byte\"},{\"key\":\"4\",\"value\":\"32\",\"dataType\":\"Byte\"},{\"key\":\"5\",\"value\":\"32\",\"dataType\":\"Byte\"},{\"key\":\"6\",\"value\":\"32\",\"dataType\":\"Byte\"},{\"key\":\"7\",\"value\":\"32\",\"dataType\":\"Byte\"},{\"key\":\"8\",\"value\":\"32\",\"dataType\":\"Byte\"},{\"key\":\"9\",\"value\":\"32\",\"dataType\":\"Byte\"},{\"key\":\"10\",\"value\":\"32\",\"dataType\":\"Byte\"},{\"key\":\"11\",\"value\":\"32\",\"dataType\":\"Byte\"},{\"key\":\"12\",\"value\":\"32\",\"dataType\":\"Byte\"},{\"key\":\"13\",\"value\":\"32\",\"dataType\":\"Byte\"},{\"key\":\"14\",\"value\":\"32\",\"dataType\":\"Byte\"},{\"key\":\"15\",\"value\":\"32\",\"dataType\":\"Byte\"},{\"key\":\"16\",\"value\":\"32\",\"dataType\":\"Byte\"},{\"key\":\"17\",\"value\":\"32\",\"dataType\":\"Byte\"},{\"key\":\"18\",\"value\":\"32\",\"dataType\":\"Byte\"},{\"key\":\"19\",\"value\":\"32\",\"dataType\":\"Byte\"},{\"key\":\"20\",\"value\":\"32\",\"dataType\":\"Byte\"},{\"key\":\"21\",\"value\":\"32\",\"dataType\":\"Byte\"},{\"key\":\"22\",\"value\":\"32\",\"dataType\":\"Byte\"},{\"key\":\"23\",\"value\":\"32\",\"dataType\":\"Byte\"},{\"key\":\"24\",\"value\":\"32\",\"dataType\":\"Byte\"},{\"key\":\"25\",\"value\":\"32\",\"dataType\":\"Byte\"},{\"key\":\"26\",\"value\":\"32\",\"dataType\":\"Byte\"},{\"key\":\"27\",\"value\":\"32\",\"dataType\":\"Byte\"},{\"key\":\"28\",\"value\":\"32\",\"dataType\":\"Byte\"},{\"key\":\"29\",\"value\":\"32\",\"dataType\":\"Byte\"},{\"key\":\"30\",\"value\":\"32\",\"dataType\":\"Byte\"},{\"key\":\"31\",\"value\":\"32\",\"dataType\":\"Byte\"},{\"key\":\"32\",\"value\":\"32\",\"dataType\":\"Byte\"},{\"key\":\"33\",\"value\":\"32\",\"dataType\":\"Byte\"},{\"key\":\"34\",\"value\":\"32\",\"dataType\":\"Byte\"},{\"key\":\"35\",\"value\":\"32\",\"dataType\":\"Byte\"},{\"key\":\"36\",\"value\":\"32\",\"dataType\":\"Byte\"},{\"key\":\"37\",\"value\":\"32\",\"dataType\":\"Byte\"},{\"key\":\"38\",\"value\":\"32\",\"dataType\":\"Byte\"},{\"key\":\"39\",\"value\":\"32\",\"dataType\":\"Byte\"}],\"dataType\":\"ByteCollection\"},{\"key\":\"PROG_DAT\",\"value\":[{\"key\":\"0\",\"value\":\"32\",\"dataType\":\"Byte\"},{\"key\":\"1\",\"value\":\"32\",\"dataType\":\"Byte\"},{\"key\":\"2\",\"value\":\"32\",\"dataType\":\"Byte\"},{\"key\":\"3\",\"value\":\"32\",\"dataType\":\"Byte\"},{\"key\":\"4\",\"value\":\"32\",\"dataType\":\"Byte\"},{\"key\":\"5\",\"value\":\"32\",\"dataType\":\"Byte\"},{\"key\":\"6\",\"value\":\"32\",\"dataType\":\"Byte\"},{\"key\":\"7\",\"value\":\"32\",\"dataType\":\"Byte\"},{\"key\":\"8\",\"value\":\"32\",\"dataType\":\"Byte\"},{\"key\":\"9\",\"value\":\"32\",\"dataType\":\"Byte\"},{\"key\":\"10\",\"value\":\"32\",\"dataType\":\"Byte\"},{\"key\":\"11\",\"value\":\"32\",\"dataType\":\"Byte\"},{\"key\":\"12\",\"value\":\"32\",\"dataType\":\"Byte\"},{\"key\":\"13\",\"value\":\"32\",\"dataType\":\"Byte\"},{\"key\":\"14\",\"value\":\"32\",\"dataType\":\"Byte\"},{\"key\":\"15\",\"value\":\"32\",\"dataType\":\"Byte\"},{\"key\":\"16\",\"value\":\"32\",\"dataType\":\"Byte\"},{\"key\":\"17\",\"value\":\"32\",\"dataType\":\"Byte\"},{\"key\":\"18\",\"value\":\"32\",\"dataType\":\"Byte\"},{\"key\":\"19\",\"value\":\"32\",\"dataType\":\"Byte\"},{\"key\":\"20\",\"value\":\"32\",\"dataType\":\"Byte\"},{\"key\":\"21\",\"value\":\"32\",\"dataType\":\"Byte\"},{\"key\":\"22\",\"value\":\"32\",\"dataType\":\"Byte\"},{\"key\":\"23\",\"value\":\"32\",\"dataType\":\"Byte\"},{\"key\":\"24\",\"value\":\"32\",\"dataType\":\"Byte\"},{\"key\":\"25\",\"value\":\"32\",\"dataType\":\"Byte\"},{\"key\":\"26\",\"value\":\"32\",\"dataType\":\"Byte\"},{\"key\":\"27\",\"value\":\"32\",\"dataType\":\"Byte\"},{\"key\":\"28\",\"value\":\"32\",\"dataType\":\"Byte\"},{\"key\":\"29\",\"value\":\"32\",\"dataType\":\"Byte\"},{\"key\":\"30\",\"value\":\"32\",\"dataType\":\"Byte\"},{\"key\":\"31\",\"value\":\"32\",\"dataType\":\"Byte\"},{\"key\":\"32\",\"value\":\"32\",\"dataType\":\"Byte\"},{\"key\":\"33\",\"value\":\"32\",\"dataType\":\"Byte\"},{\"key\":\"34\",\"value\":\"32\",\"dataType\":\"Byte\"},{\"key\":\"35\",\"value\":\"32\",\"dataType\":\"Byte\"},{\"key\":\"36\",\"value\":\"32\",\"dataType\":\"Byte\"},{\"key\":\"37\",\"value\":\"32\",\"dataType\":\"Byte\"},{\"key\":\"38\",\"value\":\"32\",\"dataType\":\"Byte\"},{\"key\":\"39\",\"value\":\"32\",\"dataType\":\"Byte\"},{\"key\":\"40\",\"value\":\"32\",\"dataType\":\"Byte\"},{\"key\":\"41\",\"value\":\"32\",\"dataType\":\"Byte\"},{\"key\":\"42\",\"value\":\"32\",\"dataType\":\"Byte\"},{\"key\":\"43\",\"value\":\"32\",\"dataType\":\"Byte\"},{\"key\":\"44\",\"value\":\"32\",\"dataType\":\"Byte\"},{\"key\":\"45\",\"value\":\"32\",\"dataType\":\"Byte\"},{\"key\":\"46\",\"value\":\"32\",\"dataType\":\"Byte\"},{\"key\":\"47\",\"value\":\"32\",\"dataType\":\"Byte\"},{\"key\":\"48\",\"value\":\"32\",\"dataType\":\"Byte\"},{\"key\":\"49\",\"value\":\"32\",\"dataType\":\"Byte\"},{\"key\":\"50\",\"value\":\"32\",\"dataType\":\"Byte\"},{\"key\":\"51\",\"value\":\"32\",\"dataType\":\"Byte\"},{\"key\":\"52\",\"value\":\"32\",\"dataType\":\"Byte\"},{\"key\":\"53\",\"value\":\"32\",\"dataType\":\"Byte\"},{\"key\":\"54\",\"value\":\"32\",\"dataType\":\"Byte\"},{\"key\":\"55\",\"value\":\"32\",\"dataType\":\"Byte\"},{\"key\":\"56\",\"value\":\"32\",\"dataType\":\"Byte\"},{\"key\":\"57\",\"value\":\"32\",\"dataType\":\"Byte\"},{\"key\":\"58\",\"value\":\"32\",\"dataType\":\"Byte\"},{\"key\":\"59\",\"value\":\"32\",\"dataType\":\"Byte\"},{\"key\":\"60\",\"value\":\"32\",\"dataType\":\"Byte\"},{\"key\":\"61\",\"value\":\"32\",\"dataType\":\"Byte\"},{\"key\":\"62\",\"value\":\"32\",\"dataType\":\"Byte\"},{\"key\":\"63\",\"value\":\"32\",\"dataType\":\"Byte\"},{\"key\":\"64\",\"value\":\"32\",\"dataType\":\"Byte\"},{\"key\":\"65\",\"value\":\"32\",\"dataType\":\"Byte\"},{\"key\":\"66\",\"value\":\"32\",\"dataType\":\"Byte\"},{\"key\":\"67\",\"value\":\"32\",\"dataType\":\"Byte\"},{\"key\":\"68\",\"value\":\"32\",\"dataType\":\"Byte\"},{\"key\":\"69\",\"value\":\"32\",\"dataType\":\"Byte\"},{\"key\":\"70\",\"value\":\"32\",\"dataType\":\"Byte\"},{\"key\":\"71\",\"value\":\"32\",\"dataType\":\"Byte\"},{\"key\":\"72\",\"value\":\"32\",\"dataType\":\"Byte\"},{\"key\":\"73\",\"value\":\"32\",\"dataType\":\"Byte\"},{\"key\":\"74\",\"value\":\"32\",\"dataType\":\"Byte\"},{\"key\":\"75\",\"value\":\"32\",\"dataType\":\"Byte\"},{\"key\":\"76\",\"value\":\"32\",\"dataType\":\"Byte\"},{\"key\":\"77\",\"value\":\"32\",\"dataType\":\"Byte\"},{\"key\":\"78\",\"value\":\"32\",\"dataType\":\"Byte\"},{\"key\":\"79\",\"value\":\"32\",\"dataType\":\"Byte\"}],\"dataType\":\"ByteCollection\"},{\"key\":\"CYCLETIME_AVERAGE\",\"value\":\"0\",\"dataType\":\"Int16\"}],\"dataType\":\"Struct<TEST_PLC_CARRIER_UDT>\"},{\"key\":\"CARRIER\",\"value\":[{\"key\":\"BLOCK_POSITION_NUMBER\",\"value\":\"0\",\"dataType\":\"Int16\"},{\"key\":\"OPERATING_HOURS\",\"value\":\"0\",\"dataType\":\"Int32\"}],\"dataType\":\"Struct<TEST_CARRIER_UDT>\"},{\"key\":\"DRIVE\",\"value\":[{\"key\":\"BMK\",\"value\":\"0\",\"dataType\":\"Int16\"},{\"key\":\"POSITION\",\"value\":\"0\",\"dataType\":\"Int32\"},{\"key\":\"SPEED\",\"value\":\"0\",\"dataType\":\"Int16\"},{\"key\":\"ACT_CURRENT\",\"value\":\"0\",\"dataType\":\"Int16\"},{\"key\":\"MAX_CURRENT\",\"value\":\"0\",\"dataType\":\"Int16\"},{\"key\":\"MAX_CURRENT_AVERAGE\",\"value\":\"0\",\"dataType\":\"Int16\"},{\"key\":\"ERRORCODE\",\"value\":\"0\",\"dataType\":\"Int16\"},{\"key\":\"OPERATING_DISTANCE_MM\",\"value\":\"0\",\"dataType\":\"Int32\"},{\"key\":\"OPERATING_DISTANCE_KM\",\"value\":\"0\",\"dataType\":\"Int32\"}],\"dataType\":\"Struct<TEST_FREQU_CONV_UDT>\"},{\"key\":\"USER\",\"value\":[{\"key\":\"BARCODE_DRIVE\",\"value\":\"0\",\"dataType\":\"Int16\"},{\"key\":\"BARCODE_DRIVE_MIN\",\"value\":\"0\",\"dataType\":\"Int16\"},{\"key\":\"WIFI_SIGNAL_STRENGTH\",\"value\":\"0\",\"dataType\":\"Int16\"},{\"key\":\"WIFI_CHANNEL\",\"value\":\"0\",\"dataType\":\"UInt16\"},{\"key\":\"TX_RATE\",\"value\":\"0\",\"dataType\":\"UInt16\"},{\"key\":\"SC_RESPONSE_TIME\",\"value\":\"0\",\"dataType\":\"UInt32\"}],\"dataType\":\"Struct<TEST_CARRIER_MAX_UDT_USER>\"}],\"status\":\"Good\",\"lastChangeTimestamp\":\"2022-08-14T06:04:02.9607079Z\",\"measurementTimestamp\":\"2022-08-14T06:04:02.9607079Z\",\"dataType\":\"Struct<TEST_CARRIER_MAX_UDT>\"}},\"id\":\"c9e9009b-f79b-44c8-9a44-ebd5a59b0814\",\"$schema\":\"https://xxxxx.blob.core.windows.net/schemas/message_schemas/2021-05-05/xxx.sensor.sensortelemetrymes...",\"metadata\":{\"timestamp\":\"2022-08-14T06:04:22.2257279Z\",\"correlationIds\":[],\"senderIdentifier\":{\"id\":\"aaaaa-1493-4f61-993b-e3fb046908aa\",\"name\":\"OPC UA Connector\",\"type\":\"gateway\",\"route\":\"\"},\"destinationIdentifiers\":[]}}","SCHEMA_MAPPER":"xxxx.TELEMETRY","enqueuedTime":"2022-08-14T06:04:22.225Z","YEAR":2022,"MONTH":8,"DAY":14,"HOUR":6}

[richgalloway]

What fields do you want to get out of that data?  What have you tried so far?

[Marttin123]

This is what we get at the moment:

DAY

HOUR

MONTH

SCHEMA_MAPPER

YEAR

body

enqueredTime

[richgalloway]

Thank you, but that response answers neither of my questions.