Splunk Search

How to extract as below using universal forwarder props.conf?

bapun18
Communicator

I want to extract as below using universal forwarder props.conf

 

 

 

 

 

Whatever data I have before: should be the field name and after : would be the value
eg- for Class field value is Catalyst 9500
  "class": "Catalyst 9500", 
  "var_actionname": "Logstash - Chain", 
  "var_alertid": "4000", 
  "var_app_sys_id": "", 
  "var_assetfloor": "0", 
  "var_assetlocation": "", 
  "var_assetmake": "mycompany Systems", 
  "var_assetmodel": "Catalyst 9500", 
  "var_assetpanel": "", 
  "var_assetplate": "", 
  "var_assetpunch": "", 
  "var_assetrack": "", 
  "var_assetroom": "", 
  "var_assetserial": "", 
  "var_assetshelf": "", 
  "var_assettag": "", 
  "var_assetzone": "", 
  "var_autopolicyname": "Chain Active Events", 
  "var_autopolicynote": "", 
  "var_categoryid": "8", 
  "var_categoryname": "Network.Switches", 
  "var_classid": "6659", 
  "var_classname": "Catalyst 9500", 
  "var_classtype": "mycompany Systems", 
  "var_clearuser": "", 
  "var_collector": "csit2apacdca06", 
  "var_composite_criticality": 3, 
  "var_composite_id": "0", 
  "var_device_back_link": "https://123.121.12.13//index.?exec=registry&act=registry_device_management#devmgt_search.did=4526", 
  "var_deviceid": "4526", 
  "var_duty_pager": "", 
  "var_esp_class_name": "", 
  "var_event_back_link": "https://123.121.12.13//index.?exec=device_events&did=4526&etype=12708", 
  "var_event_guid": "EEBC704A15AFBB55FA19EF7D50A93993", 
  "var_eventcategory": "", 
  "var_eventcounter": "1", 
  "var_evententityid": "4526", 
  "var_evententityname": "ccntrx4-cn-bb-gw2.mycompany.com", 
  "var_evententitytype": "1", 
  "var_eventfirstoccurtime": "2022-09-22 22:32:05", 
  "var_eventid": "10784243", 
  "var_eventindexid": ".1199", 
  "var_eventlastoccurtime": "2022-09-22 22:32:05", 
  "var_eventmessage": "mycompany: Temperature problem. Currently, Temperature (TenGigabitEthernet1/0/40 Module Temperature Sensor) status: unavailable", 
  "var_eventpolicy": "mycompany: Temperature Unavailable", 
  "var_eventpolicycause": "<strong><!--StartFragment-->Description</strong><br>mycompany network device is reporting an &quot;unavailable&quot; status on temperature. Meaning that the agent presently can not report the temperature&apos;s sensor value.<br><br><strong>Probable Cause</strong><br><ul class=\"fr-tag\"><li class=\"fr-tag\">The sensor could have a hard failure (disconnected wire).</li><li class=\"fr-tag\">The sensor could have a soft failure such as out-of-range, jitter, or wildly fluctuating readings.</li></ul><br><strong>Resolution</strong><br>Manually check functioning of fan and replace if necessary.<!--EndFragment-->", 
  "var_eventpolicyexternalid": "", 
  "var_eventpolicyid": "12708", 
  "var_eventseverity_deprecated": "2", 
  "var_eventseveritylevel": "3", 
  "var_eventseveritytext": "MAJOR", 
  "var_eventsourceid": "4", 
  "var_eventsourcename": "Dynamic", 
  "var_eventstate": "Active", 
  "var_eventstateful": "1", 
  "var_eventsubentityid": "0", 
  "var_eventsubentityname": ".1199", 
  "var_eventsubentitytype": "0", 
  "var_eventticketid": "", 
  "var_eventtimeactive": "2022-09-22 22:32:05", 
  "var_eventtimedeleted": "None", 
  "var_eventurllink": "https://123.121.12.13//index.?exec=events&q_type=aid&q_arg=10784243&q_sev=1&q_sort=0&q_oper=0", 
  "var_eventusercleared": "", 
  "var_eventusernote": "", 
  "var_ipaddress": "10.79.194.32", 
  "var_orgbillingid": "", 
  "var_orgcrmid": "ff7ac89f1b5f8d94d73aec22b24bcbe9", 
  "var_orgid": "2", 
  "var_orgimpacted": "", 
  "var_orgname": "mycompany IT", 
  "var_parentid": "", 
  "var_parentname": "", 
  "var_priority": "", 
  "var_resultvalue": "unavailable", 
  "var_rootid": "", 
  "var_rootname": "", 
  "var_slsystemname": "", 
  "var_super_organization": "unknown", 
  "var_support_group": "", 
  "var_sysid": "fd19769ddb00c3ccdaeaf9551d961908", 
  "var_threshold": "", 
  "var_ticketemailsubject": "2", 
  "var_ticketid": "0", 
  "var_username": "", 
  "external_id": "ScienceLogic_", 
  "manager": "SCIENCELOGIC__ASSURED", 
  "signature": "ccntrx4-cn-bb-gw2.mycompany.com::Catalyst 9500::.1199", 
  "source": "ccntrx4-cn-bb-gw2.mycompany.com", 
  "source_id": "1234"

 

 

 

 

 

 I will attach the example of the log file that needs to be pushed with extracted fields, in the comment section

 

 

 

 

Labels (1)
Tags (1)
0 Karma

bapun18
Communicator

I want the values of the below fields
⦁ Date/time of the chain message
⦁ var_classname
⦁ var_entityname
⦁ var_deviceid
⦁ var_ipaddress
⦁ var_evententityid
⦁ var_eventpolicy
⦁ var_eventstate
⦁var_sys_id
⦁ var_composite_id
⦁ var_composite_criticality

0 Karma

isoutamo
SplunkTrust
SplunkTrust

Hi

You cannot do field extractions on UF. That needs HF or indexer if you want to do it on indexing time. Another option is do it on search time if you are ingesting all data on that event.

r. Ismo

0 Karma

bapun18
Communicator

I can do it on search time but, don't want to ingest unformatted data, can you help me with indexer-level filters.

0 Karma

isoutamo
SplunkTrust
SplunkTrust

I add here couple of example, as I'm sure that you can do rest by yourself with these instructions. If this is not enough, then please ask help from some local splunk partners or others who will do this as their business. 

regex101.com is your friend when you want to test regex.

The next examples will store the whole event and indexed the fields which you have defined on props+transforms.conf. 

props.conf and transforms.conf will go to HF / indexers which one is first on path from source (UF) to indexers.

[your-sourcetype]
TRANSFORMS-get-var_classname = get-var_classname
TRANSFORMS-get-var_entityname = get-var_entityname
.... rest fields

transforms.conf

[get-var_classname]
REGEX = "var_classname": "([^"]+)"
FORMAT = var_classname::$1
WRITE_META = true

[get-var_entityname]
REGEX = "var_entityname": "([^"]+)"
FORMAT = var_entityname::$1
WRITE_META = true

.... rest extractions

 

fields.conf, this must be on all search heads

[var_classname]
INDEXED = true

[var_entityname]
INDEXED = true

... rest fields

 

 

bapun18
Communicator
0 Karma
Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...