I have a script that pulls the data at the bottom into a file and then splunk pull the files from the corresponding directory
However one of the 3 hosts works properly and 2 hosts split into 2 events, that are different sizes. All the files should be the same in terms of structure. I cannot seem to solve the source type to fix it.
I am still fairly new to splunk, Any advice?
Script to retrieve log data
#!/bin/bash
cd /home/max
hosts=$(pcregrep -M '^192.*-.*\n' /etc/hosts | sed -e 's/[0-9]\{1,3\}\.[0-9]\{1,3\}\.[0-9]\{1,3\}\.[0-9]\{1,3\}/\n/g' | tail -n +2 | tr -d '[:blank:]')
rm -rf logs/*
rm -rf logs2/*
for host in $hosts
do
if ping -c 1 -W 1 $host; then
if [[ $host == *"SW"* ]]; then
mkdir -p logs2/$host
ssh -oIdentitiesOnly=yes -i /home/max/.ssh/id_rsa_2 marq-net@$host -q 'ubntbox mca-status' | sed 's/,/\n/g' | sed '/^[[:space:]]*$/d' > logs2/$host/$host.log
else
mkdir -p logs/$host
ssh -oIdentitiesOnly=yes -i /home/max/.ssh/id_rsa_2 marq-net@$host -q 'ubntbox mca-status' | sed 's/,/\n/g' | sed '/^[[:space:]]*$/d' > logs/$host/$host.log
fi
fi
done
echo $(TZ='America/Chicago' date)
echo $(date)
They should be the exact same structure
max@splunk:~$ cat logs/Silo-Link02/Silo-Link02.log| wc -l
67
max@splunk:~$ cat logs/Marq-Link01/Marq-Link01.log| wc -l
67
Here is the log from Marq-Link01
deviceName=Marq-Link01
deviceId=XX:CC:XX:CC:XX:CC
firmwareVersion=WA.ar934x.v8.6.2.41239.190822.1633
platform=LiteBeam 5AC Gen2
deviceIp=192.168.0.50
apMac=XX:CC:XX:CC:XX:CC
wlanOpmode=ap-ptp-ac
wlanConnections=1
wlanUptime=235026
essid=DataLink
security=WPA2
freq=5520
centerFreq=5550
txPower=24
chanbw=80
signal=-70
chain0Signal=-72
chain1Signal=-74
noise=-88
cinr=23
evm=24
uptime=497206
airTime=0.1
cpuUsage=33.3
loadavg=7
memTotal=61952
memFree=13764
memBuffers=3376
distance=3605
netrole=bridge
lanIpAddress=0.0.0.0
wlanIpAddress=0.0.0.0
wlanTxRate=351.0
wlanRxRate=468.0
txModRate=4x
rxModRate=6x
wlanTxLatency=0
wlanPolling=1
wlanScanStatus=0
wlanDownlinkCapacity=263250
wlanUplinkCapacity=238680
lanRxBytes=3192091802
lanRxPackets=17133341
lanRxErrors=0
lanTxBytes=11338063946
lanTxPackets=10570286
lanTxErrors=0
lanPlugged=1
lanSpeed=1000Mbps-Full
cableLen=29
wlanRxBytes=8409699951
wlanRxPackets=7949519
wlanRxErrors=0
wlanTxBytes=2920142565
wlanTxPackets=14584597
wlanTxErrors=0
wlanRxErrNwid=19662
wlanRxErrCrypt=0
wlanRxErrFrag=0
wlanRxErrRetries=0
wlanRxErrBmiss=0
wlanRxErrOther=0
latitude=42
longitude=-97
boardCrc=e06fb32b
cfgCrc=39387dfa
status_flags=4
Here is the log that works properly from Silo-Link2
deviceName=Silo - link02
deviceId=B4:FB:E4:B8:8E:85
firmwareVersion=WA.ar934x.v8.6.2.41239.190822.1633
platform=LiteAP AC
deviceIp=192.168.0.52
apMac=B4:FB:E4:B8:8E:85
wlanOpmode=ap-ptmp-ac
wlanConnections=0
wlanUptime=0
essid=Marq-DataLink2
security=WPA2
freq=5680
centerFreq=5680
txPower=24
chanbw=10
signal=0
chain0Signal=0
chain1Signal=0
noise=0
cinr=0
evm=0
uptime=93652
airTime=1.4
cpuUsage=65.0
loadavg=0
memTotal=61952
memFree=17580
memBuffers=3264
distance=100000
netrole=bridge
lanIpAddress=0.0.0.0
wlanIpAddress=0.0.0.0
wlanTxRate=0.0
wlanRxRate=0.0
txModRate=1x
rxModRate=1x
wlanTxLatency=0
wlanPolling=1
wlanScanStatus=0
wlanDownlinkCapacity=0
wlanUplinkCapacity=0
lanRxBytes=262240342
lanRxPackets=2100983
lanRxErrors=0
lanTxBytes=443438977
lanTxPackets=448666
lanTxErrors=0
lanPlugged=1
lanSpeed=1000Mbps-Full
cableLen=41
wlanRxBytes=0
wlanRxPackets=0
wlanRxErrors=0
wlanTxBytes=174479589
wlanTxPackets=1408700
wlanTxErrors=0
wlanRxErrNwid=0
wlanRxErrCrypt=0
wlanRxErrFrag=0
wlanRxErrRetries=0
wlanRxErrBmiss=0
wlanRxErrOther=0
latitude=42
longitude=-97
boardCrc=ef03416f
cfgCrc=d59a5d30
status_flags=4
It's likely not a script problem, but a props.conf problem. In the inputs.conf file that defines the scripted input, add sourceype = foo
. Then in the corresponding props.conf file add:
[foo]
DATE_TIME_CONFIG = current
SHOULD_LINEMERGE = false
LINE_BREAKER = ([\r\n]+)deviceName
It's likely not a script problem, but a props.conf problem. In the inputs.conf file that defines the scripted input, add sourceype = foo
. Then in the corresponding props.conf file add:
[foo]
DATE_TIME_CONFIG = current
SHOULD_LINEMERGE = false
LINE_BREAKER = ([\r\n]+)deviceName
I apologize, I'm not sure where those files are.
my splunk is at /opt/splunk which file is the one I want to edit
max@splunk:~$ sudo find /opt/splunk | grep inputs.conf
/opt/splunk/etc/apps/introspection_generator_addon/default/inputs.conf
/opt/splunk/etc/apps/splunk_app_infrastructure/README/inputs.conf.spec
/opt/splunk/etc/apps/splunk_app_infrastructure/local/inputs.conf
/opt/splunk/etc/apps/splunk_app_infrastructure/default/inputs.conf
/opt/splunk/etc/apps/SplunkLightForwarder/default/inputs.conf
/opt/splunk/etc/apps/splunk_instrumentation/default/inputs.conf
/opt/splunk/etc/apps/sample_app/default/inputs.conf
/opt/splunk/etc/apps/snmp_ta/README/inputs.conf.spec
/opt/splunk/etc/apps/splunk_httpinput/default/inputs.conf
/opt/splunk/etc/apps/splunk_monitoring_console/default/inputs.conf
/opt/splunk/etc/apps/search/local/inputs.conf
/opt/splunk/etc/system/README/inputs.conf.spec
/opt/splunk/etc/system/README/inputs.conf.example
/opt/splunk/etc/system/local/inputs.conf
/opt/splunk/etc/system/default/inputs.conf
/opt/splunk/etc/modules/distributedDeployment/classes/deployable/inputs.conf
/opt/splunk/lib/python2.7/site-packages/slim/config/conf-specs/inputs.conf.spec
max@splunk:~$ sudo find /opt/splunk | grep props.conf
/opt/splunk/etc/apps/splunk_app_infrastructure/default/props.conf
/opt/splunk/etc/apps/learned/local/props.conf
/opt/splunk/etc/apps/splunk_archiver/default/props.conf
/opt/splunk/etc/apps/SplunkLightForwarder/default/props.conf
/opt/splunk/etc/apps/legacy/default/props.conf
/opt/splunk/etc/apps/splunk_instrumentation/default/props.conf
/opt/splunk/etc/apps/sample_app/default/props.conf
/opt/splunk/etc/apps/snmp_ta/default/props.conf
/opt/splunk/etc/apps/splunk_monitoring_console/default/props.conf
/opt/splunk/etc/apps/search/local/props.conf
/opt/splunk/etc/apps/search/default/props.conf
/opt/splunk/etc/system/README/props.conf.spec
/opt/splunk/etc/system/README/props.conf.example
/opt/splunk/etc/system/default/props.conf
/opt/splunk/lib/python2.7/site-packages/slim/config/conf-specs/props.conf.spec
max@splunk:~$
That did the trick Sir. Thank you so much! If you dont mind explaining it a bit it would be much appreciated.
First, do NOT edit any file in a 'default' directory or your changes will be lost the next time you upgrade Splunk. See the note at the top of $SPLUNK_HOME/etc/system/default.props.conf. It's better to create a custom apps, myorg_foo_inputs and myorg_foo_props, to hold the config files then install those apps in the appropriate places (search head, indexer, forwarder).
The sourcetype
attribute in inputs.conf answers the original question: how to set a source type for a data source. Every unique source type (format) should have a sourcetype attribute and a corresponding stanza in props.conf.
The props.conf stanza tells Splunk how to parse the data of that type. Splunk is pretty good at guessing, but, as you've seen, doesn't always get it right. Use props to prevent Splunk from guessing wrong.
DATE_TIME_CONFIG = current
says to use the current time for events since the data does not contain a timestamp.
SHOULD_LINEMERGE = false
says not to join lines together.
'LINE_BREAKER = ([\r\n]+)deviceName' tells Splunk events begin when "deviceName" is seen immediately after a newline.
All of these settings are in the docs and in $SPLUNK_HOME/etc/system/README/props.conf.spec.
Nevermind got it 🙂
For anyone like me
/opt/splunk/etc/system/default/props.conf