Getting Data In

Are my props.conf and transforms.conf correct in setting metadata from TCP input?

ericmck2000
Explorer

So... I am attempting to setup a TCP input, which will automatically set metadata, from the event.

The _Raw looks like: {"time":"2017-01-31T15:51:56.9081571-06:00","index":"main","source":"ToolsTesting","host":"348SR52-OGS","Event":"Hello world"}

With the pretty version looking like:

{   [-] 
     Event:  Hello world    
     host:   348SR52-OGS    
     index:  main   
     source:     ToolsTesting   
     time:   2017-01-31T15:51:56.9081571-06:00  
}

So, the "Event" field contains the actual event, whether it be hello world, or a complex object, That piece is working....

But, I want to be able to set the source, host, index... etc, via fields located in my raw input.

So far, I have this....

Props:

[toolsjson]
INDEXED_EXTRACTIONS = json
KV_MODE = json
NO_BINARY_CHECK = true
category = Structured
description = Parse out messages
pulldown_type = 1
disabled = false
TRANSFORM-setsource=set_source_value

Transforms:

[set_source_value]
DEST_KEY = MetaData:Host
REGEX = .
FORMAT = host::$1

I guess the issue is.... I am not very familiar with how to properly use transforms to set the fields. I have looked over a few examples, and I am still slightly lost. Can somebody give me a bit of help?

As a bonus, I would like to strip the "metadata" out of my raw event, and only display the actual event.

0 Karma
1 Solution

ericmck2000
Explorer
0 Karma

ericmck2000
Explorer

http://docs.splunk.com/Documentation/Splunk/6.4.1/Data/Assignmetadatatoeventsdynamically

I found the answer after spending a few hours using google.

0 Karma

ericmck2000
Explorer

Write-up

I posted a full write-up of everything I did for anybody who finds this information useful.

http://xtremeownage.com/index.php?threads/sending-events-to-splunk-via-tcp-using-c.1648/#post-3083

Got questions? Get answers!

Join the Splunk Community Slack to learn, troubleshoot, and make connections with fellow Splunk practitioners in real time!

Meet up IRL or virtually!

Join Splunk User Groups to connect and learn in-person by region or remotely by topic or industry.

Get Updates on the Splunk Community!

SOC4Kafka - New Kafka Connector Powered by OpenTelemetry

The new SOC4Kafka connector, built on OpenTelemetry, enables the collection of Kafka messages and forwards ...

Event Series: Level up your SOC: Advancing with Splunk Enterprise Security

AI has fundamentally raised the stakes for security operations, and this three-part series is your guide to ...

Announcing Modern Navigation: A New Era of Splunk User Experience

We are excited to introduce the Modern Navigation feature in the Splunk Platform, available to both cloud and ...