topic Field Extraction in Getting Data In

Field Extraction

afamoyib — Wed, 22 Mar 2017 22:44:37 GMT

I am trying to extract a field but it is not working properly. I am able to extract single words but when spaces gets involved it fails. I am basically trying to extract all 4 tab keywords.

Event Sample
Description : Security Update
HotFixID : KB3212646
InstalledBy : NT AUTHORITY\SYSTEM
InstalledOn : 1/20/2017 12:00:00 AM

Description : Update
HotFixID : KB2952664
InstalledBy : NT AUTHORITY\SYSTEM
InstalledOn : 2/17/2017 12:00:00 AM

Re: Field Extraction

DalJeanis — Wed, 22 Mar 2017 23:15:06 GMT

Where exactly are you doing this? Is this an index-time extraction or a search-time extraction?

Are there CR/LFs in the _raw the way you have shown them, or are the events actually like this, or some other way ?

Description : Security Update HotFixID : KB3212646  InstalledBy : NT AUTHORITY\SYSTEM InstalledOn : 1/20/2017 12:00:00 AM

Are these the only fields you need to extract, or are there other keywords than these four?

Re: Field Extraction

JDukeSplunk — Thu, 23 Mar 2017 19:41:53 GMT

I don't know if you want to try a different approach, but if you're looking for Windows info like patch levels, uptime, and so on you might try WMI inputs. I created a new app called "wmi" and deployed it to my windows hosts. It's no-frills and basically consists of a directory named wmi and a sub-directory named local. Then plop this down as inputs.conf after editing what index you want things to go to. You can edit the wql = line to include whatever information that WMI class will output.

It just works.

# WMI FOR appdev INDEX
#replace the index = line with the correct index 
#place this file in C:\Program Files\SplunkUniversalForwarder\etc\apps\wmi\local as inputs.conf

[settings]
initial_backoff = 5
max_backoff = 20
max_retries_at_max_backoff = 0
checkpoint_sync_interval = 2

## Processes
[WMI:LocalProcesses]
interval = 120
wql = Select IDProcess,PrivateBytes,Name,PercentProcessorTime,TimeStamp_Sys100NS from Win32_PerfRawData_PerfProc_Process
index = idx_appdev
disabled = 0


## Scheduled Jobs

## Use the Win32_ScheduledJob  class. Note that this class can only return jobs that are created using either a script or AT.exe. 
## It cannot return information about jobs that are either created by or modified by the Scheduled Task wizard.
[WMI:ScheduledJobs]
disabled = 0
## Run once per day
interval = 86400
wql = SELECT Caption, Command, Description, InstallDate, InteractWithDesktop, JobId, JobStatus, Name, Notify, Priority, RunRepeatedly, Status FROM Win32_ScheduledJob
index = idx_appdev

## Services

## http://msdn.microsoft.com/en-us/library/aa394418(VS.85).aspx
## Lists all services registered on the system,if they are running,and the status
[WMI:Service]
disabled = 0
## Run once an hour
interval = 3600
wql = SELECT Name, Caption, State, Status, StartMode, StartName, PathName, Description FROM Win32_Service
index = idx_appdev


## Update
[WMI:InstalledUpdates]
disabled = 0
## Run once per day
interval = 86400
wql = SELECT Description, FixComments, HotFixID, InstalledBy, InstalledOn, ServicePackInEffect FROM Win32_QuickFixEngineering
index = idx_appdev


## Uptime
[WMI:Uptime]
disabled = 0
## Run once an hour
interval = 3600
wql = SELECT SystemUpTime FROM Win32_PerfFormattedData_PerfOS_System
index = idx_appdev

## index = idx_appdev


## Version
[WMI:Version]
disabled = 0
## Run once per day
interval = 86400
wql = SELECT Caption, ServicePackMajorVersion, ServicePackMinorVersion, Version FROM Win32_OperatingSystem
index = idx_appdev

Then I built a little dashboard around it..

Re: Field Extraction

afamoyib — Thu, 23 Mar 2017 20:40:53 GMT

I am trying to do index-time extraction. When i try to use to extract tool it extracts poorly.

Description : Security Update
HotFixID : KB3212646
InstalledBy : NT AUTHORITY\SYSTEM
InstalledOn : 1/20/2017 12:00:00 AM

For example I am trying to extract the contents for description and make it a field and i am trying to extract installedby contents and make it another field. I used this regex pattern ^\w+\s+:\s+(?P\w+) but i was able to extract only one word.

The tool appears to not be providing me the desired effect.

Re: Field Extraction

afamoyib — Tue, 21 Nov 2017 21:39:43 GMT

This was completed by writing regex statement to break properly and extract them into new words.