topic Re: I want to extract as below using universal forwarder props.conf in Splunk Search

How to extract as below using universal forwarder props.conf?

bapun18 — Thu, 22 Sep 2022 23:38:49 GMT

I want to extract as below using universal forwarder props.conf

Whatever data I have before: should be the field name and after : would be the value eg- for Class field value is Catalyst 9500 "class": "Catalyst 9500", "var_actionname": "Logstash - Chain", "var_alertid": "4000", "var_app_sys_id": "", "var_assetfloor": "0", "var_assetlocation": "", "var_assetmake": "mycompany Systems", "var_assetmodel": "Catalyst 9500", "var_assetpanel": "", "var_assetplate": "", "var_assetpunch": "", "var_assetrack": "", "var_assetroom": "", "var_assetserial": "", "var_assetshelf": "", "var_assettag": "", "var_assetzone": "", "var_autopolicyname": "Chain Active Events", "var_autopolicynote": "", "var_categoryid": "8", "var_categoryname": "Network.Switches", "var_classid": "6659", "var_classname": "Catalyst 9500", "var_classtype": "mycompany Systems", "var_clearuser": "", "var_collector": "csit2apacdca06", "var_composite_criticality": 3, "var_composite_id": "0", "var_device_back_link": "https://123.121.12.13//index.?exec=registry&act=registry_device_management#devmgt_search.did=4526", "var_deviceid": "4526", "var_duty_pager": "", "var_esp_class_name": "", "var_event_back_link": "https://123.121.12.13//index.?exec=device_events&did=4526&etype=12708", "var_event_guid": "EEBC704A15AFBB55FA19EF7D50A93993", "var_eventcategory": "", "var_eventcounter": "1", "var_evententityid": "4526", "var_evententityname": "ccntrx4-cn-bb-gw2.mycompany.com", "var_evententitytype": "1", "var_eventfirstoccurtime": "2022-09-22 22:32:05", "var_eventid": "10784243", "var_eventindexid": ".1199", "var_eventlastoccurtime": "2022-09-22 22:32:05", "var_eventmessage": "mycompany: Temperature problem. Currently, Temperature (TenGigabitEthernet1/0/40 Module Temperature Sensor) status: unavailable", "var_eventpolicy": "mycompany: Temperature Unavailable", "var_eventpolicycause": "Description mycompany network device is reporting an "unavailable" status on temperature. Meaning that the agent presently can not report the temperature's sensor value. Probable Cause <ul class=\"fr-tag\"><li class=\"fr-tag\">The sensor could have a hard failure (disconnected wire).</li><li class=\"fr-tag\">The sensor could have a soft failure such as out-of-range, jitter, or wildly fluctuating readings.</li></ul> Resolution Manually check functioning of fan and replace if necessary.", "var_eventpolicyexternalid": "", "var_eventpolicyid": "12708", "var_eventseverity_deprecated": "2", "var_eventseveritylevel": "3", "var_eventseveritytext": "MAJOR", "var_eventsourceid": "4", "var_eventsourcename": "Dynamic", "var_eventstate": "Active", "var_eventstateful": "1", "var_eventsubentityid": "0", "var_eventsubentityname": ".1199", "var_eventsubentitytype": "0", "var_eventticketid": "", "var_eventtimeactive": "2022-09-22 22:32:05", "var_eventtimedeleted": "None", "var_eventurllink": "https://123.121.12.13//index.?exec=events&q_type=aid&q_arg=10784243&q_sev=1&q_sort=0&q_oper=0", "var_eventusercleared": "", "var_eventusernote": "", "var_ipaddress": "10.79.194.32", "var_orgbillingid": "", "var_orgcrmid": "ff7ac89f1b5f8d94d73aec22b24bcbe9", "var_orgid": "2", "var_orgimpacted": "", "var_orgname": "mycompany IT", "var_parentid": "", "var_parentname": "", "var_priority": "", "var_resultvalue": "unavailable", "var_rootid": "", "var_rootname": "", "var_slsystemname": "", "var_super_organization": "unknown", "var_support_group": "", "var_sysid": "fd19769ddb00c3ccdaeaf9551d961908", "var_threshold": "", "var_ticketemailsubject": "2", "var_ticketid": "0", "var_username": "", "external_id": "ScienceLogic_", "manager": "SCIENCELOGIC__ASSURED", "signature": "ccntrx4-cn-bb-gw2.mycompany.com::Catalyst 9500::.1199", "source": "ccntrx4-cn-bb-gw2.mycompany.com", "source_id": "1234"

I will attach the example of the log file that needs to be pushed with extracted fields, in the comment section

Re: I want to extract as below using universal forwarder props.conf

bapun18 — Thu, 22 Sep 2022 22:58:49 GMT

https://drive.google.com/file/d/1wkjoejCrAyeTX5v6bfKTDPUXgmuHONJ6/view?usp=sharing

Re: I want to extract as below using universal forwarder props.conf

bapun18 — Thu, 22 Sep 2022 23:01:56 GMT

I want the values of the below fields
⦁ Date/time of the chain message
⦁ var_classname
⦁ var_entityname
⦁ var_deviceid
⦁ var_ipaddress
⦁ var_evententityid
⦁ var_eventpolicy
⦁ var_eventstate
⦁var_sys_id
⦁ var_composite_id
⦁ var_composite_criticality

Re: I want to extract as below using universal forwarder props.conf

isoutamo — Fri, 23 Sep 2022 07:10:48 GMT

You cannot do field extractions on UF. That needs HF or indexer if you want to do it on indexing time. Another option is do it on search time if you are ingesting all data on that event.

r. Ismo

Re: I want to extract as below using universal forwarder props.conf

bapun18 — Fri, 23 Sep 2022 12:37:11 GMT

I can do it on search time but, don't want to ingest unformatted data, can you help me with indexer-level filters.

Re: I want to extract as below using universal forwarder props.conf

isoutamo — Fri, 23 Sep 2022 19:08:11 GMT

I add here couple of example, as I'm sure that you can do rest by yourself with these instructions. If this is not enough, then please ask help from some local splunk partners or others who will do this as their business.

regex101.com is your friend when you want to test regex.

The next examples will store the whole event and indexed the fields which you have defined on props+transforms.conf.

props.conf and transforms.conf will go to HF / indexers which one is first on path from source (UF) to indexers.

[your-sourcetype] TRANSFORMS-get-var_classname = get-var_classname TRANSFORMS-get-var_entityname = get-var_entityname .... rest fields

transforms.conf

[get-var_classname] REGEX = "var_classname": "([^"]+)" FORMAT = var_classname::$1 WRITE_META = true [get-var_entityname] REGEX = "var_entityname": "([^"]+)" FORMAT = var_entityname::$1 WRITE_META = true .... rest extractions

fields.conf, this must be on all search heads

[var_classname] INDEXED = true [var_entityname] INDEXED = true ... rest fields