I want to extract as below using universal forwarder props.conf
Whatever data I have before: should be the field name and after : would be the value
eg- for Class field value is Catalyst 9500
"class": "Catalyst 9500",
"var_actionname": "Logstash - Chain",
"var_alertid": "4000",
"var_app_sys_id": "",
"var_assetfloor": "0",
"var_assetlocation": "",
"var_assetmake": "mycompany Systems",
"var_assetmodel": "Catalyst 9500",
"var_assetpanel": "",
"var_assetplate": "",
"var_assetpunch": "",
"var_assetrack": "",
"var_assetroom": "",
"var_assetserial": "",
"var_assetshelf": "",
"var_assettag": "",
"var_assetzone": "",
"var_autopolicyname": "Chain Active Events",
"var_autopolicynote": "",
"var_categoryid": "8",
"var_categoryname": "Network.Switches",
"var_classid": "6659",
"var_classname": "Catalyst 9500",
"var_classtype": "mycompany Systems",
"var_clearuser": "",
"var_collector": "csit2apacdca06",
"var_composite_criticality": 3,
"var_composite_id": "0",
"var_device_back_link": "https://123.121.12.13//index.?exec=registry&act=registry_device_management#devmgt_search.did=4526",
"var_deviceid": "4526",
"var_duty_pager": "",
"var_esp_class_name": "",
"var_event_back_link": "https://123.121.12.13//index.?exec=device_events&did=4526&etype=12708",
"var_event_guid": "EEBC704A15AFBB55FA19EF7D50A93993",
"var_eventcategory": "",
"var_eventcounter": "1",
"var_evententityid": "4526",
"var_evententityname": "ccntrx4-cn-bb-gw2.mycompany.com",
"var_evententitytype": "1",
"var_eventfirstoccurtime": "2022-09-22 22:32:05",
"var_eventid": "10784243",
"var_eventindexid": ".1199",
"var_eventlastoccurtime": "2022-09-22 22:32:05",
"var_eventmessage": "mycompany: Temperature problem. Currently, Temperature (TenGigabitEthernet1/0/40 Module Temperature Sensor) status: unavailable",
"var_eventpolicy": "mycompany: Temperature Unavailable",
"var_eventpolicycause": "<strong><!--StartFragment-->Description</strong><br>mycompany network device is reporting an "unavailable" status on temperature. Meaning that the agent presently can not report the temperature's sensor value.<br><br><strong>Probable Cause</strong><br><ul class=\"fr-tag\"><li class=\"fr-tag\">The sensor could have a hard failure (disconnected wire).</li><li class=\"fr-tag\">The sensor could have a soft failure such as out-of-range, jitter, or wildly fluctuating readings.</li></ul><br><strong>Resolution</strong><br>Manually check functioning of fan and replace if necessary.<!--EndFragment-->",
"var_eventpolicyexternalid": "",
"var_eventpolicyid": "12708",
"var_eventseverity_deprecated": "2",
"var_eventseveritylevel": "3",
"var_eventseveritytext": "MAJOR",
"var_eventsourceid": "4",
"var_eventsourcename": "Dynamic",
"var_eventstate": "Active",
"var_eventstateful": "1",
"var_eventsubentityid": "0",
"var_eventsubentityname": ".1199",
"var_eventsubentitytype": "0",
"var_eventticketid": "",
"var_eventtimeactive": "2022-09-22 22:32:05",
"var_eventtimedeleted": "None",
"var_eventurllink": "https://123.121.12.13//index.?exec=events&q_type=aid&q_arg=10784243&q_sev=1&q_sort=0&q_oper=0",
"var_eventusercleared": "",
"var_eventusernote": "",
"var_ipaddress": "10.79.194.32",
"var_orgbillingid": "",
"var_orgcrmid": "ff7ac89f1b5f8d94d73aec22b24bcbe9",
"var_orgid": "2",
"var_orgimpacted": "",
"var_orgname": "mycompany IT",
"var_parentid": "",
"var_parentname": "",
"var_priority": "",
"var_resultvalue": "unavailable",
"var_rootid": "",
"var_rootname": "",
"var_slsystemname": "",
"var_super_organization": "unknown",
"var_support_group": "",
"var_sysid": "fd19769ddb00c3ccdaeaf9551d961908",
"var_threshold": "",
"var_ticketemailsubject": "2",
"var_ticketid": "0",
"var_username": "",
"external_id": "ScienceLogic_",
"manager": "SCIENCELOGIC__ASSURED",
"signature": "ccntrx4-cn-bb-gw2.mycompany.com::Catalyst 9500::.1199",
"source": "ccntrx4-cn-bb-gw2.mycompany.com",
"source_id": "1234"
I will attach the example of the log file that needs to be pushed with extracted fields, in the comment section
I want the values of the below fields
⦁ Date/time of the chain message
⦁ var_classname
⦁ var_entityname
⦁ var_deviceid
⦁ var_ipaddress
⦁ var_evententityid
⦁ var_eventpolicy
⦁ var_eventstate
⦁var_sys_id
⦁ var_composite_id
⦁ var_composite_criticality
Hi
You cannot do field extractions on UF. That needs HF or indexer if you want to do it on indexing time. Another option is do it on search time if you are ingesting all data on that event.
r. Ismo
I can do it on search time but, don't want to ingest unformatted data, can you help me with indexer-level filters.
I add here couple of example, as I'm sure that you can do rest by yourself with these instructions. If this is not enough, then please ask help from some local splunk partners or others who will do this as their business.
regex101.com is your friend when you want to test regex.
The next examples will store the whole event and indexed the fields which you have defined on props+transforms.conf.
props.conf and transforms.conf will go to HF / indexers which one is first on path from source (UF) to indexers.
[your-sourcetype]
TRANSFORMS-get-var_classname = get-var_classname
TRANSFORMS-get-var_entityname = get-var_entityname
.... rest fields
transforms.conf
[get-var_classname]
REGEX = "var_classname": "([^"]+)"
FORMAT = var_classname::$1
WRITE_META = true
[get-var_entityname]
REGEX = "var_entityname": "([^"]+)"
FORMAT = var_entityname::$1
WRITE_META = true
.... rest extractions
fields.conf, this must be on all search heads
[var_classname]
INDEXED = true
[var_entityname]
INDEXED = true
... rest fields