topic Re: Extract Fields from web_input app. in Splunk Search

How to Extract Fields from web_input app?

MGlass — Mon, 07 Aug 2023 19:27:12 GMT

How would you extract fields from this Data, I would like to extract the panel ID, watts, grid Hz, grid voltage and temp from the data, the grid data is on every set of 4 lines, there are 24 panels.

ID Watts Volts Freq Gvolts Temp

match="805000048512-1 98 W 33 V 60.0 Hz 251 V 35 °C

08/07/2023 12:58:53 UTC, _time="1691438333.0", title="ZEDTwo30", encoding="utf-8", browser="integrated_client", response_size="14555", response_code="200", url="http://192.168.2.178/index.php/realtimedata", request_time="734.2638969421387", content_md5="40acffc51f6d6213b2b1e1b379bc14f2", content_sha224="aae23ec01baaf5502794091e3cc7e00d1cb6ba265ef675a999e27dc0", raw_match_count="25", match="Inverter ID Current Power DC Voltage Grid Frequency Grid Voltage Temperature Reporting Time", match="805000048512-1 1 W 36 V 60.0 Hz 253 V 14 °C 2023-08-07 06:57:04", match="805000048512-2 0 W 36 V 253 V", match="805000048512-3 0 W 36 V 253 V", match="805000048512-4 1 W 36 V 253 V", match="805000050217-1 0 W 36 V 60.0 Hz 252 V 14 °C 2023-08-07 06:57:04", match="805000050217-2 1 W 36 V 252 V", match="805000050217-3 0 W 36 V 252 V", match="805000050217-4 1 W 36 V 252 V", match="805000048270-1 1 W 36 V 60.1 Hz 253 V 15 °C 2023-08-07 06:57:04", match="805000048270-2 1 W 36 V 253 V", match="805000048270-3 1 W 36 V 253 V", match="805000048270-4 1 W 36 V 253 V", match="805000051865-1 0 W 36 V 60.0 Hz 252 V 14 °C 2023-08-07 06:57:04", match="805000051865-2 0 W 36 V 252 V", match="805000051865-3 0 W 36 V 252 V", match="805000051865-4 0 W 36 V 252 V", match="805000050663-1 0 W 36 V 60.0 Hz 252 V 13 °C 2023-08-07 06:57:04", match="805000050663-2 0 W 36 V 252 V", match="805000050663-3 0 W 36 V 252 V", match="805000050663-4 0 W 36 V 252 V", match="805000048357-1 0 W 36 V 59.9 Hz 251 V 14 °C 2023-08-07 06:57:04", match="805000048357-2 0 W 36 V 251 V", match="805000048357-3 0 W 36 V 251 V", match="805000048357-4 0 W 36 V 251 V"

Re: Extract Fields from web_input app.

ITWhisperer — Mon, 07 Aug 2023 14:21:16 GMT

You need to expand the events by match=".... then you can extract the fields you want. This is because some match strings do not have all the values represented.

| rex max_match=0 "(?<match>match=\"[^\"]+)" | mvexpand match | rex field=match "match=\"(?<id>[\d-]+)\s(?<watts>\d+\sW)\s(?<volts>\d+\sV)\s((?<freq>[\d\.]+\sHz)\s)?(?<gvolts>\d+\sV)(\s(?<temp>\d+\s.C))?"

You may want to ignore events where match just contains the field headers

Re: Extract Fields from web_input app.

MGlass — Mon, 07 Aug 2023 16:22:44 GMT

Thank you, works perfect, I was struggling and couldn't wrap my head around the different strings, I have not used mvexpand before, looks like it breaks a single log into multiples?

Re: Extract Fields from web_input app.

ITWhisperer — Mon, 07 Aug 2023 21:48:53 GMT

More specifically, it duplicates events except for the expanded multi-value field which is split out across the copies of the event.