topic Re: How can I merge events from powershell input? in Getting Data In

How can I merge events from powershell input?

yutaka1005 — Fri, 11 Jan 2019 06:50:30 GMT

I want to get events from below powershell input.

inputs.conf

 [powershell://power_shell_sapmle]
 script = . "$SplunkHome\etc\apps\sample_app\bin\sample.ps1"
 schedule = */1 * * * *
 sourcetype = power_shell_sapmle

sample.ps1

 $Output = invoke-expression "wmic cpu list brief"
 Write-Output $Output

But, the event is divided and outputted line by line, even if I configured props.conf in Indexer like below.

props.conf

[power_shell_sapmle]
DATETIME_CONFIG = CURRENT
SHOULD_LINEMERGE = true
BREAK_ONLY_BEFORE_DATE = false
MAX_EVENTS = 1000
BREAK_ONLY_BEFORE = "NEVER_BREAKE_LINE"

How can I merge these events to 1 event?
Please someone help me.

*additional info

Output event of this powershell input like below.

Re: How can I merge events from powershell input?

p_gurav — Fri, 11 Jan 2019 09:44:02 GMT

Can you share sample output you are getting by script?

Re: How can I merge events from powershell input?

ccl0utier — Fri, 11 Jan 2019 13:13:20 GMT

One option to troubleshoot yourself is to get a sample of your data into a file and then use the "Add data" wizard in your Splunk instance to see how Splunk with treat your data while playing with the props.conf parameters interactively.

This should help you quickly troubleshoot and get to a working configuration.

Re: How can I merge events from powershell input?

woodcock — Fri, 11 Jan 2019 21:17:34 GMT

Like this in props.conf:

 [power_shell_sapmle]
 DATETIME_CONFIG = CURRENT
 SHOULD_LINEMERGE = false
 LINE_BREAKER = (?!)

Re: How can I merge events from powershell input?

yutaka1005 — Tue, 15 Jan 2019 00:58:15 GMT

Thank you for answer!

But, I can't merge by that settings...

Re: How can I merge events from powershell input?

woodcock — Tue, 15 Jan 2019 03:14:24 GMT

I do not understand what you mean.

Re: How can I merge events from powershell input?

yutaka1005 — Tue, 15 Jan 2019 05:15:21 GMT

I mean that I put the setting you suggested in props.conf, but as before, the data captured by the powershell input was split into one event per line.

Re: How can I merge events from powershell input?

woodcock — Thu, 14 Feb 2019 01:33:21 GMT

You should NOT be using any merge settings. Just use my 3 and nothing else.
Make sure that if you are using sourcetype-override, that you use the original sourcetype.
Deploy all settings to your Heavy Forwarder and Indexer tiers.
Restart all Splunk instances there.
Send in new data and search for it with an All time value for Timepicker and index_earliest=-5m so that you FOR SURE are looking at recently-indexed events.
This will work.

Re: How can I merge events from powershell input?

yutaka1005 — Tue, 29 Sep 2020 23:16:48 GMT

I have just tried again, but it still not work.

I writes below things just in case.

I have used ONLY your 3 settings.
Also I have just created new sourcetype "power_shell_sapmle_20190213", so sourcetype-override is impossible.
I have deployed inputs.conf and powershell script to Heavy Forwarder, and put props.conf to Indexer with your 3 settings, then restarted both.
I have checked events by using real-time search, but it has never worked.
Splunk Ver is 7.2.3

If you can do it in your verification environment, would you please show me?

Re: How can I merge events from powershell input?

woodcock — Thu, 14 Feb 2019 02:17:07 GMT

You are not taking me literally enough. The problem is that you have not deployed the props.conf settings to your Heavy Forwarder tier. The index-time settings of props.conf (including timestamping and line-splitting) happen on the FIRST FULL INSTANCE of Splunk that is encountered along the way. In your case, that is your Heavy Forwarders. Put props.conf there, restart Splunk everywhere, test the way that I mentioned, and you will be good-to-go.

Re: How can I merge events from powershell input?

spayneort — Thu, 14 Feb 2019 03:13:45 GMT

You could change your PowerShell script to output in json format:

Get-WmiObject Win32_Processor | Select-Object Caption,DeviceID,Manufacturer,MaxClockSpeed,Name,SocketDesignation | ConvertTo-Json -Compress

Re: How can I merge events from powershell input?

yutaka1005 — Thu, 14 Feb 2019 07:02:27 GMT

I'm sorry, my explanation was wrong.

My environment, I have these instances.

Splunk Indexer
Splunk Universal forwarder(* not heavy forwarder)

My Universal Forwarder on Windows executes powershell script, and forwards to Indexer.
Therefore, I think that putting props.conf in Indexer is not wrong.

Re: How can I merge events from powershell input?

woodcock — Thu, 14 Feb 2019 07:14:43 GMT

Just try it. You have nothing to lose.

Re: How can I merge events from powershell input?

yutaka1005 — Thu, 14 Feb 2019 07:34:24 GMT

I have deleted props.conf of indexer and deployed props.conf to universal forwarder, then I have restarted both.

Thankfully for suggesting, but it still not work...

Re: How can I merge events from powershell input?

yutaka1005 — Thu, 14 Feb 2019 07:37:04 GMT

I was able to get cpu data by changing powershell to yours.
But I still wonder if I can not acquire it as a single event unless I change the output of powershell itself...

Re: How can I merge events from powershell input?

ololdach — Tue, 22 Jun 2021 13:54:54 GMT

powershell inputs expect the command to return PS Objects and every object will be a separate event. Find a template to convert string output to objects here: https://community.splunk.com/t5/Getting-Data-In/powershell-input-working-example-and-some-pitfalls/m-p/556695#M92128