Getting Data In

How can I merge events from powershell input?

Builder

I want to get events from below powershell input.

inputs.conf

 [powershell://power_shell_sapmle]
 script = . "$SplunkHome\etc\apps\sample_app\bin\sample.ps1"
 schedule = */1 * * * *
 sourcetype = power_shell_sapmle

sample.ps1

 $Output = invoke-expression "wmic cpu list brief"
 Write-Output $Output

But, the event is divided and outputted line by line, even if I configured props.conf in Indexer like below.

props.conf

[power_shell_sapmle]
DATETIME_CONFIG = CURRENT
SHOULD_LINEMERGE = true
BREAK_ONLY_BEFORE_DATE = false
MAX_EVENTS = 1000
BREAK_ONLY_BEFORE = "NEVER_BREAKE_LINE"

How can I merge these events to 1 event?
Please someone help me.


*additional info

Output event of this powershell input like below.

alt text

0 Karma
1 Solution

Contributor

You could change your PowerShell script to output in json format:

Get-WmiObject Win32_Processor | Select-Object Caption,DeviceID,Manufacturer,MaxClockSpeed,Name,SocketDesignation | ConvertTo-Json -Compress

View solution in original post

0 Karma

Contributor

You could change your PowerShell script to output in json format:

Get-WmiObject Win32_Processor | Select-Object Caption,DeviceID,Manufacturer,MaxClockSpeed,Name,SocketDesignation | ConvertTo-Json -Compress

View solution in original post

0 Karma

Builder

I was able to get cpu data by changing powershell to yours.
But I still wonder if I can not acquire it as a single event unless I change the output of powershell itself...

0 Karma

Esteemed Legend

You are not taking me literally enough. The problem is that you have not deployed the props.conf settings to your Heavy Forwarder tier. The index-time settings of props.conf (including timestamping and line-splitting) happen on the FIRST FULL INSTANCE of Splunk that is encountered along the way. In your case, that is your Heavy Forwarders. Put props.conf there, restart Splunk everywhere, test the way that I mentioned, and you will be good-to-go.

0 Karma

Builder

I'm sorry, my explanation was wrong.

My environment, I have these instances.

  • Splunk Indexer
  • Splunk Universal forwarder(* not heavy forwarder)

My Universal Forwarder on Windows executes powershell script, and forwards to Indexer.
Therefore, I think that putting props.conf in Indexer is not wrong.

0 Karma

Esteemed Legend

Just try it. You have nothing to lose.

0 Karma

Builder

I have deleted props.conf of indexer and deployed props.conf to universal forwarder, then I have restarted both.

Thankfully for suggesting, but it still not work...

0 Karma

Esteemed Legend

Like this in props.conf:

 [power_shell_sapmle]
 DATETIME_CONFIG = CURRENT
 SHOULD_LINEMERGE = false
 LINE_BREAKER = (?!)
0 Karma

Builder

Thank you for answer!

But, I can't merge by that settings...

0 Karma

Esteemed Legend

I do not understand what you mean.

0 Karma

Builder

I mean that I put the setting you suggested in props.conf, but as before, the data captured by the powershell input was split into one event per line.

0 Karma

Esteemed Legend

You should NOT be using any merge settings. Just use my 3 and nothing else.
Make sure that if you are using sourcetype-override, that you use the original sourcetype.
Deploy all settings to your Heavy Forwarder and Indexer tiers.
Restart all Splunk instances there.
Send in new data and search for it with an All time value for Timepicker and index_earliest=-5m so that you FOR SURE are looking at recently-indexed events.
This will work.

0 Karma

Builder

I have just tried again, but it still not work.

I writes below things just in case.

  • I have used ONLY your 3 settings.
  • Also I have just created new sourcetype "powershellsapmle_20190213", so sourcetype-override is impossible.
  • I have deployed inputs.conf and powershell script to Heavy Forwarder, and put props.conf to Indexer with your 3 settings, then restarted both.
  • I have checked events by using real-time search, but it has never worked.
  • Splunk Ver is 7.2.3

If you can do it in your verification environment, would you please show me?

0 Karma

Splunk Employee
Splunk Employee

One option to troubleshoot yourself is to get a sample of your data into a file and then use the "Add data" wizard in your Splunk instance to see how Splunk with treat your data while playing with the props.conf parameters interactively.

This should help you quickly troubleshoot and get to a working configuration.

0 Karma

Champion

Can you share sample output you are getting by script?

0 Karma