Splunk Search

How to extract as below using universal forwarder props.conf?

bapun18
Communicator

I want to extract as below using universal forwarder props.conf

 

 

 

 

 

Whatever data I have before: should be the field name and after : would be the value
eg- for Class field value is Catalyst 9500
  "class": "Catalyst 9500", 
  "var_actionname": "Logstash - Chain", 
  "var_alertid": "4000", 
  "var_app_sys_id": "", 
  "var_assetfloor": "0", 
  "var_assetlocation": "", 
  "var_assetmake": "mycompany Systems", 
  "var_assetmodel": "Catalyst 9500", 
  "var_assetpanel": "", 
  "var_assetplate": "", 
  "var_assetpunch": "", 
  "var_assetrack": "", 
  "var_assetroom": "", 
  "var_assetserial": "", 
  "var_assetshelf": "", 
  "var_assettag": "", 
  "var_assetzone": "", 
  "var_autopolicyname": "Chain Active Events", 
  "var_autopolicynote": "", 
  "var_categoryid": "8", 
  "var_categoryname": "Network.Switches", 
  "var_classid": "6659", 
  "var_classname": "Catalyst 9500", 
  "var_classtype": "mycompany Systems", 
  "var_clearuser": "", 
  "var_collector": "csit2apacdca06", 
  "var_composite_criticality": 3, 
  "var_composite_id": "0", 
  "var_device_back_link": "https://123.121.12.13//index.?exec=registry&act=registry_device_management#devmgt_search.did=4526", 
  "var_deviceid": "4526", 
  "var_duty_pager": "", 
  "var_esp_class_name": "", 
  "var_event_back_link": "https://123.121.12.13//index.?exec=device_events&did=4526&etype=12708", 
  "var_event_guid": "EEBC704A15AFBB55FA19EF7D50A93993", 
  "var_eventcategory": "", 
  "var_eventcounter": "1", 
  "var_evententityid": "4526", 
  "var_evententityname": "ccntrx4-cn-bb-gw2.mycompany.com", 
  "var_evententitytype": "1", 
  "var_eventfirstoccurtime": "2022-09-22 22:32:05", 
  "var_eventid": "10784243", 
  "var_eventindexid": ".1199", 
  "var_eventlastoccurtime": "2022-09-22 22:32:05", 
  "var_eventmessage": "mycompany: Temperature problem. Currently, Temperature (TenGigabitEthernet1/0/40 Module Temperature Sensor) status: unavailable", 
  "var_eventpolicy": "mycompany: Temperature Unavailable", 
  "var_eventpolicycause": "<strong><!--StartFragment-->Description</strong><br>mycompany network device is reporting an &quot;unavailable&quot; status on temperature. Meaning that the agent presently can not report the temperature&apos;s sensor value.<br><br><strong>Probable Cause</strong><br><ul class=\"fr-tag\"><li class=\"fr-tag\">The sensor could have a hard failure (disconnected wire).</li><li class=\"fr-tag\">The sensor could have a soft failure such as out-of-range, jitter, or wildly fluctuating readings.</li></ul><br><strong>Resolution</strong><br>Manually check functioning of fan and replace if necessary.<!--EndFragment-->", 
  "var_eventpolicyexternalid": "", 
  "var_eventpolicyid": "12708", 
  "var_eventseverity_deprecated": "2", 
  "var_eventseveritylevel": "3", 
  "var_eventseveritytext": "MAJOR", 
  "var_eventsourceid": "4", 
  "var_eventsourcename": "Dynamic", 
  "var_eventstate": "Active", 
  "var_eventstateful": "1", 
  "var_eventsubentityid": "0", 
  "var_eventsubentityname": ".1199", 
  "var_eventsubentitytype": "0", 
  "var_eventticketid": "", 
  "var_eventtimeactive": "2022-09-22 22:32:05", 
  "var_eventtimedeleted": "None", 
  "var_eventurllink": "https://123.121.12.13//index.?exec=events&q_type=aid&q_arg=10784243&q_sev=1&q_sort=0&q_oper=0", 
  "var_eventusercleared": "", 
  "var_eventusernote": "", 
  "var_ipaddress": "10.79.194.32", 
  "var_orgbillingid": "", 
  "var_orgcrmid": "ff7ac89f1b5f8d94d73aec22b24bcbe9", 
  "var_orgid": "2", 
  "var_orgimpacted": "", 
  "var_orgname": "mycompany IT", 
  "var_parentid": "", 
  "var_parentname": "", 
  "var_priority": "", 
  "var_resultvalue": "unavailable", 
  "var_rootid": "", 
  "var_rootname": "", 
  "var_slsystemname": "", 
  "var_super_organization": "unknown", 
  "var_support_group": "", 
  "var_sysid": "fd19769ddb00c3ccdaeaf9551d961908", 
  "var_threshold": "", 
  "var_ticketemailsubject": "2", 
  "var_ticketid": "0", 
  "var_username": "", 
  "external_id": "ScienceLogic_", 
  "manager": "SCIENCELOGIC__ASSURED", 
  "signature": "ccntrx4-cn-bb-gw2.mycompany.com::Catalyst 9500::.1199", 
  "source": "ccntrx4-cn-bb-gw2.mycompany.com", 
  "source_id": "1234"

 

 

 

 

 

 I will attach the example of the log file that needs to be pushed with extracted fields, in the comment section

 

 

 

 

Labels (1)
Tags (1)
0 Karma

bapun18
Communicator

I want the values of the below fields
⦁ Date/time of the chain message
⦁ var_classname
⦁ var_entityname
⦁ var_deviceid
⦁ var_ipaddress
⦁ var_evententityid
⦁ var_eventpolicy
⦁ var_eventstate
⦁var_sys_id
⦁ var_composite_id
⦁ var_composite_criticality

0 Karma

isoutamo
SplunkTrust
SplunkTrust

Hi

You cannot do field extractions on UF. That needs HF or indexer if you want to do it on indexing time. Another option is do it on search time if you are ingesting all data on that event.

r. Ismo

0 Karma

bapun18
Communicator

I can do it on search time but, don't want to ingest unformatted data, can you help me with indexer-level filters.

0 Karma

isoutamo
SplunkTrust
SplunkTrust

I add here couple of example, as I'm sure that you can do rest by yourself with these instructions. If this is not enough, then please ask help from some local splunk partners or others who will do this as their business. 

regex101.com is your friend when you want to test regex.

The next examples will store the whole event and indexed the fields which you have defined on props+transforms.conf. 

props.conf and transforms.conf will go to HF / indexers which one is first on path from source (UF) to indexers.

[your-sourcetype]
TRANSFORMS-get-var_classname = get-var_classname
TRANSFORMS-get-var_entityname = get-var_entityname
.... rest fields

transforms.conf

[get-var_classname]
REGEX = "var_classname": "([^"]+)"
FORMAT = var_classname::$1
WRITE_META = true

[get-var_entityname]
REGEX = "var_entityname": "([^"]+)"
FORMAT = var_entityname::$1
WRITE_META = true

.... rest extractions

 

fields.conf, this must be on all search heads

[var_classname]
INDEXED = true

[var_entityname]
INDEXED = true

... rest fields

 

 

bapun18
Communicator
0 Karma
Get Updates on the Splunk Community!

Register to Attend BSides SPL 2022 - It's all Happening October 18!

Join like-minded individuals for technical sessions on everything Splunk!  This is a community-led and run ...

What's New in Splunk Cloud Platform 9.0.2208?!

Howdy!  We are happy to share the newest updates in Splunk Cloud Platform 9.0.2208! Analysts can benefit ...

Admin Console: A Single, Unified Interface for All Your Cloud Admin Needs

WATCH NOWJoin us to learn how the admin console can save you time and give you more control over the Splunk® ...