topic Re: How to configure props/transforms.conf for this data in Getting Data In

How to configure props/transforms.conf for this data

dbcase — Tue, 14 Feb 2017 16:00:00 GMT

Hi,

I have this data and need to know what I need to configure for props/transforms.conf to parse the data correctly. Correctly= KV pair - field=Manufacturer value=Kwikset

Thank you!

002446fffd003274.2:
    Label: Back Door Lock
    Manufacturer: Kwikset
    Model: SMARTCODE_DEADBOLT_5
    Firmware version: 0x3071cb06
    Hardware version: 3
    User Properties:
        NearEndRssi: -41
        NearEndLqi: 243
        label: Back Door Lock
        deadboltJammed: false
    Battery Operated: True
    Voltage: 5.8V
    FE radio: -67/254
    NE radio: -41/243
    Date added: Thu Oct 27 08:02:42 CDT 2016
    Date of last communication: Mon Feb 13 14:15:14 CST 2017
    In Communication Failure: false
    In firmware upgrade failure: false
    Firmware upgrade available: false
    Is Locked: true
    Max Users: 30
    Operation Mode: normal

002446fffd00bd27.2:
    Label: Front Door Lock
    Manufacturer: Kwikset
    Model: SMARTCODE_DEADBOLT_10
    Firmware version: 0x3071c405
    Hardware version: 3
    User Properties:
        NearEndRssi: -51
        NearEndLqi: 255
        label: Front Door Lock
    Battery Operated: True
    Voltage: 5.8V
    FE radio: -50/255
    NE radio: -51/255
    Date added: Wed Oct 12 19:11:33 CDT 2016
    Date of last communication: Mon Feb 13 14:15:23 CST 2017
    In Communication Failure: false
    In firmware upgrade failure: false
    Firmware upgrade available: false
    Is Locked: true
    Max Users: 30
    Operation Mode: normal

Re: How to configure props/transforms.conf for this data

gcusello — Tue, 14 Feb 2017 16:28:14 GMT

Hi dbcase,
the regex to extract your Manifacturer field is (see https://regex101.com/r/QMxJpT/1):

Manufacturer:\s(?<Manufacturer>.*)

But what is you timestamp "Date added" or "Date of last communication"? This is the first information.

Every way, if your timestamp is "Date added"
your props.conf is

SHOULD_LINEMERGE=true
NO_BINARY_CHECK=true
BREAK_ONLY_BEFORE=\w*\.\d+:
TIME_PREFIX=Date added:
TIME_FORMAT=%b %d %H:%M:%S %Z %Y
EXTRACT-Comp_Name = Manufacturer:\s(?<Manufacturer>.*)

if your timestamp is "Date of last communication"
your props.conf is

SHOULD_LINEMERGE=true
NO_BINARY_CHECK=true
BREAK_ONLY_BEFORE=\w*\.\d+:
TIME_PREFIX=Date of last communication:
TIME_FORMAT=%b %d %H:%M:%S %Z %Y
EXTRACT-Comp_Name = Manufacturer:\s(?<Manufacturer>.*)

Bye.
Giuseppe

Re: How to configure props/transforms.conf for this data

dbcase — Tue, 14 Feb 2017 16:30:33 GMT

Hi Giuseppe!

Thanks! What I'm trying to do is extract ALL the fields I just used Manufacturer as an example.

Re: How to configure props/transforms.conf for this data

gcusello — Tue, 14 Feb 2017 16:34:21 GMT

Extraction is the same for the other fields.
Test your extraction in regex101.com (see https://regex101.com/r/QMxJpT/2)
Bye.
Giuseppe

Re: How to configure props/transforms.conf for this data

somesoni2 — Tue, 14 Feb 2017 16:35:26 GMT

Give this a try

props.conf (on Search Head)

[YourSourceType]
REPORT-extractkv = extract_key_value_pair

transforms.conf(on Search Head)

[extract_key_value_pair]
REGEX = (.+?): (.+?)$
FORMAT = $1::$2
CLEAN_KEYS = true
MV_ADD = true

Re: How to configure props/transforms.conf for this data

dbcase — Tue, 14 Feb 2017 16:43:46 GMT

Perfect! Thank you Somesoni2!!