topic Re: How to extract an ip address from a field and store in another field? in Splunk Search

How to extract an ip address from a field and store in another field?

arindamlaha — Tue, 29 Sep 2020 13:51:18 GMT

I have a csv file with data in the following format...

logsource,Critical,Buffer Overflow,15:05:27 13 Mar 2017,,sourceserver_172.92.110.10 (172.92.110.10),,somedesthost_172.92.110.18 (172.92.110.18),",N/A/1080",Application Servers Protection Violation,buffer overflow,,,2,01548914,
logsource,Medium,Application Activity,12:11:23 31 Mar 2017,,172.92.110.83,,somedestination_172.92.110.88 (172.92.110.88),file-wbt-server [tcp/19380],,,,,22,01547076,
logsource,Informational,Application Activity,23:56:20 31 Mar 2017,India,202.91.201.54,United States,ABCDEF_172.74.18.13 (172.74.168.13),tcp/443,,,,,22,01553030,
logsource,Informational,Application Activity,23:49:22 31 Mar 2017,,somesource1_172.92.18.7 (172.92.18.7),United States,14.35.2.46,tcp/1443,,,,,3,01552984,

I need to extract the source IP address from the 6th fields in each row and save in a field "src_ip_address"
eg. from line 1, src_ip_address = 172.92.110.10
from line 2, src_ip_addres = 172.92.110.83

Similarly I need to extract the destination IP address from the 8th field and store the values in a "dst_ip_address" field
eg.
from line 1, dst_ip_address = 172.92.110.88
from line 2, dst_ip_address = 172.92.110.18

Is this possible? if yes, it will be of immense help if someone could show a clue.

Thanks !

Re: How to extract an ip address from a field and store in another field?

dineshraj9 — Thu, 27 Apr 2017 12:32:30 GMT

Try using rex in query this way -

| rex "([^\,]*\,){5}[^\,]*(?<src_ip_address>(\d+\.){3}\d+)[^\,]*\,[^\,]*\,[^\,]*(?<dst_ip_address>(\d+\.){3}\d+)"

Re: How to extract an ip address from a field and store in another field?

richgalloway — Thu, 27 Apr 2017 12:49:46 GMT

You used the regex tag on your message, but it's not clear if you want to the extraction at index time or search time.

For search-time extraction, try this rex command.

... | rex "(?:[^,]*?,){5}(?<src_ip_address>[^,]*?),[^,]*?,(?<dst_ip_address>[^,]*?)," | ...

For index-time extraction, try putting this stanza in your props.conf file.

[mysourcetype]
INDEXED_EXTRACTIONS = CSV
FIELD_NAMES = logsource, severity, condition, timestamp, field5, src_ip_address, field7, dst_ip_address, field9, field10, field11, field12, field13, field14, field15
TIMESTAMP_FIELDS = timestamp

Re: How to extract an ip address from a field and store in another field?

arindamlaha — Tue, 29 Sep 2020 13:51:55 GMT

Thank you for the regular expression, it's working except that it is not showing the first digit in the first octet of the src and dst ip addresses.

... | rex "([^\,]*\,){5}[^\,]*(?<src_ip_address>(\d+\.){3}\d+)[^\,]*\,[^\,]*\,[^\,]*(?<dst_ip_address>(\d+\.){3}\d+)"
| top 5 src_ip_address, dst_ip_address

src_ip_address dst_ip_address
0.92.46.23 0.92.138.71

0.92.46.25 0.92.138.71

trying to fix it... any quick suggestions?
thanks again.

Re: How to extract an ip address from a field and store in another field?

arindamlaha — Thu, 27 Apr 2017 21:09:06 GMT

This one did the magic!... could you please let me know if this is the right fix?

... | rex field=_raw "([^\,]\,){5}[^\,](?\b((\d+.){3}\d+))[^\,]\,[^\,]\,[^\,]*(?\b(\d+.){3}\d+)"

Re: How to extract an ip address from a field and store in another field?

arindamlaha — Fri, 28 Apr 2017 08:24:28 GMT

Hi, I could do the search - time extraction using the rex command -
... | rex field=_raw "([^\,]*\,){5}[^\,]*(?<src_ip>\b((\d+\.){3}\d+))[^\,]*\,[^\,]*\,[^\,]*(?<dst_ip>\b(\d+\.){3}\d+)"
Now is there a way to do the extraction at the time of indexing ?
I tried your suggestion to add a stanza to the props.conf file, it did not work.

Re: How to extract an ip address from a field and store in another field?

dineshraj9 — Fri, 28 Apr 2017 08:34:32 GMT

Add this under sourcetype -

EXTRACT_FIELDS = ([^\,]*\,){5}[^\,]*(?<src_ip>\b((\d+\.){3}\d+))[^\,]*\,[^\,]*\,[^\,]*(?<dst_ip>\b(\d+\.){3}\d+)

Re: How to extract an ip address from a field and store in another field?

arindamlaha — Fri, 28 Apr 2017 22:11:23 GMT

Thanks that worked !