Field Extraction needed

[mbasharat]

Hi,

I need extraction on below data. Variations are many but I need a specific string extracted from each variation. See blow:

cpe:/a:7-zip:7-zip
cpe:/a:adobe:acrobat
cpe:/a:adobe:acrobat_reader
cpe:/a:adobe:coldfusion
cpe:/a:adobe:flash_player
cpe:/a:adobe:reader
cpe:/a:adobe:shockwave_player
cpe:/a:apache:jmeter
cpe:/a:apache:poi
cpe:/a:oracle:jre
cpe:/a:oracle:jdk
cpe:/a:apache:struts
cpe:/a:apache:subversion
cpe:/a:apache:tomcat
cpe:/a:blackberry:blackberry_enterprise_service
cpe:/a:rim:blackberry_enterprise_server
cpe:/a:cisco:anyconnect_secure_mobility_client
cpe:/a:cisco:jabber
cpe:/a:emc:documentum_content_server
cpe:/a:firebirdsql:firebird
cpe:/a:flexerasoftware:flexnet_publisher
cpe:/a:git_for_windows_project:git_for_windows
cpe:/a:oracle:jre
cpe:/a:oracle:jdk

I need it like below:

7-zip 7-zip
adobe acrobat
adobe acrobat_reader
adobe coldfusion
adobe flash_player
adobe reader
adobe shockwave_player
apache jmeter
apache poi
apache struts
apache subversion
apache tomcat
blackberry blackberry_enterprise_service
cisco anyconnect_secure_mobility_client
cisco jabber
emc documentum_content_server
firebirdsql firebird
flexerasoftware flexnet_publisher
git_for_windows_project git_for_windows
oracle jre

If better suggestions, please provide. Thanks in-advance!!!

[jpolvino]

Another approach is to use the colon as a delimiter, and then pull the sought after fields out of the mv you create:

| makeresults
| eval myField="cpe:/a:adobe:shockwave_player"
| eval mvFields=split(myField,":")
| eval f1=mvindex(mvFields,2), f2=mvindex(mvFields,3)

While not as slick as a regex, it is easier for most mortals to understand. It's also handy in a lot of other cases where you need to pull fields out of text that has predictable delimiters.

[mbasharat]

Hi @ jpolvino,

See my response to t04kawa above. I doubt eval will be able to handle that variation will it?

[to4kawa]

| makeresults
| eval_raw="cpe:/a:7-zip:7-zip cpe:/a:adobe:acrobat cpe:/a:adobe:acrobat_reader cpe:/a:adobe:coldfusion cpe:/a:adobe:flash_player cpe:/a:adobe:reader cpe:/a:adobe:shockwave_player cpe:/a:apache:jmeter cpe:/a:apache:poi cpe:/a:apache:struts cpe:/a:apache:subversion cpe:/a:apache:tomcat cpe:/a:blackberry:blackberry_enterprise_service cpe:/a:cisco:anyconnect_secure_mobility_client cpe:/a:cisco:jabber cpe:/a:emc:documentum_content_server cpe:/a:firebirdsql:firebird cpe:/a:flexerasoftware:flexnet_publisher cpe:/a:git_for_windows_project:git_for_windows cpe:/a:oracle:jdk cpe:/a:oracle:jdk cpe:/a:oracle:jre cpe:/a:rim:blackberry_enterprise_server"
| rex max_match=0 "(?ms)a:(?<col1>[^:]+):(?<col2>\S+)"

[mbasharat]

Hi @ t04kawa,

I am seeing some additional patterns. How to consolidate these within above RegEx:

x-cpe:/a:microsoft:dynamics_365

x-cpe:/a:microsoft:azure_devops_server<br/>cpe:/a:microsoft:visual_studio_team_foundation_server

x-cpe:/a:hpe:smart_storage_administrator

cpe:/o:microsoft:windows<br/>cpe:/a:microsoft:xml_core_services

cpe:/o:microsoft:windows
cpe:/a:microsoft:windows_media_player

cpe:/o:microsoft:windows<br/>cpe:/a:microsoft:windows_defender

cpe:/o:microsoft:windows
cpe:/a:microsoft:sql_server
cpe:/a:microsoft:data_engine

cpe:/o:microsoft:windows
cpe:/a:microsoft:office
cpe:/a:microsoft:word_viewer
cpe:/a:microsoft:live_meeting_console
cpe:/a:microsoft:lync
cpe:/a:microsoft:lync_basic
cpe:/a:microsoft:lync_attendee
cpe:/a:microsoft:skype_for_business<br/>cpe:/a:microsoft:silverlight

There are several like the last one above. If you notice, there are multiple products within e.g. office, word_viewer etc... in variations enclosed in/separated with line breaker tag,

THANKS!!

[to4kawa]

use | rex mode=sed "s/\<br *\/\>/ /g" first

[mbasharat]

Two things:

The variation as is 2nd below with :/o: is not getting picked up. Any wildcard approach?

x-cpe:/a:microsoft:dynamics_365
x-cpe:/o:hpe:smart_storage_administrator

However, sed mode solution above for line breaker/br, I used it before the rex in your makeresults solution and that works but it is not working in my SPL where I have placed it first also. Any guidance on this piece pls?

[to4kawa]

1. use | in REGEX.
2. i don't know your query and logs. I can say nothing.

[mbasharat]

When I run like this to test combinations, it all works. But when I run in below query to test on one variation, it does not for html tags
:

| makeresults
| eval _raw="x-cpe:/a:microsoft:azure_devops_server
cpe:/a:microsoft:visual_studio_team_foundation_server"
| rex mode=sed "s/\/ /g"
| rex max_match=0 field=_raw "(?ms)\/\w:(?[^:]+):(?\S+)"
| eval Software=mvrange(0,mvcount(Software_Sub))
| mvexpand Software
| eval Software_Sub=mvindex(Software_Sub,Software)
| eval Software_Main=mvindex(Software_Main,Software)

| fields _raw, Software_Main, Software_Sub

index=abc sourcetype="xyz"
| bin span=24h _time
| makemv delim="," cve
| mvexpand cve

| search cpe="x-cpe:/a:microsoft:azure_devops_server<br/>cpe:/a:microsoft:visual_studio_team_foundation_server"

| stats dc(dns_name) AS Affected_Asset_Count,
latest(category) AS Vuln_Category,
latest(description) AS Description,
latest(vulnPubDate) AS Publication_Date,
latest(firstSeen) AS First_Seen_Date,
latest(lastSeen) AS Last_Seen_Date,
latest(hasBeenMitigated) AS Mitigation_Status,
latest(port) AS Port,
latest(seeAlso) AS Information,
latest(xref) AS External_Reference,
latest(plugin_name) AS Plugin_Name,
latest(plugin_version) AS Plugin_Version,
latest(plugin_family) AS Plugin_Family,
latest(pluginInfo) AS Plugin_Information,
latest(plugin_publication_date) AS Plugin_Publication_Date,
latest(plugin_modification_date) AS Plugin_Modification_Date,
latest(patch_publish_date) AS Patch_Publish_Date

by plugin_id, cve, cvss3_base_score, cpe

| rex mode=sed "s/\/ /g"
| rex max_match=0 field=cpe "(?ms)\/\w:(?[^:]+):(?\S+)"

| rename cpe AS Impacted_Software

| stats dc(Impacted_Software) as count by Impacted_Software, Software_Main, Software_Sub

Thanks in-advance!