Getting Data In

Is it possible to process a raw log file and give a out put in tabular structure?.Please help me out.My logs are like this only.

New Member

Hi I am new to splunk. My log file sample is given below:
Raw log data given below:

2011-01-01T00:00:50.665998Z : Starting eth11
2011-01-01T00:00:50.689468Z : <<<<< Iniializing IM XN >>>>>>>>
2011-01-01T00:00:50.714001Z : <<<<<<<XNIMInit - Registered the Events >>>>>>>>
2001-01-01T00:00:50.714407Z : XN IM init success
2001-01-01T00:00:50.718647Z : Started eth1
2001-01-01T00:00:52.219925Z : Found device
2001-01-01T00:00:52.223080Z : gateway device serial number is uty200866625
2001-01-01T00:00:52.223705Z : gateway device receiver id is u0115613936
2001-01-01T00:00:52.380948Z : Host mac address is p0:n5:02:1d:k8:56
2001-01-01T00:00:52.381392Z : isgateway is true
2001-01-01T00:00:52.391026Z : Item U200866625 not found in the list
2001-01-01T00:00:52.395788Z : Inserted device uty200866625 in the list
2001-01-01T00:00:52.397047Z : Calling gateway script /lib/rdk/gwSetup.sh "169.254.197.230" "search hsd1.ga.comcast.net.;nameserver 75.75.75.75;nameserver 75.75.76.76;" "73.207.244.96 PaceXG1v3;73.207.244.96 PaceXG1v3;" "eth1" "50" &
2001-01-01T00:00:52.413514Z : Output is
{
"xme":
[
{
"a":"uty200866625",
"b":"yes",
"c":"1.5.9.3",
"d":"a00:b5:c62:1d:e8:f56",
"e":"US/Eastern",
"f":"-18000",
"g":"3600",
"h":"0",
"i":"yes",
"j":"http://fakeaddress",
"k":"true",
"l":"ws://1.2.3.4:5",
"m":"http://nextfakeaddress",
"n":"search hsd1.name2.;nameserver 5.5.5.5;nameserver 5.5.6.6;",
"o":"7.2.2.9 Pa;7.2.2.9 Pa;",
"p":"Id1:1030;Id2:19004;Id3:08;Id4:1028801",
"q":"09uop0115613936"
}
]
}

2011-01-01T00:00:52.414854Z : <<<<< Iniializing IM XN >>>>>>>>
2011-01-01T00:00:54.048400Z : <<<<<<<IM XN API Call received>>>>>>>>
2011-01-01T00:00:54.048791Z : <<<<<<<IM XN Call Reached API>>>>>>>>
2011-01-01T00:00:54.049162Z : Output string is

2001-01-01T00:00:52.413514Z : Output is
{
"xme":
[
{
"a":"uty200866600",
"b":"yes",
"c":"1.2.1.2",
"d":"an0:cb5:s62:1de:ef8:gh56",
"e":"US/Eastern",
"f":"-18000000",
"g":"3600000",
"h":"0",
"i":"yes",
"j":"http://fakeaddress",
"k":"true",
"l":"ws://1.2.3.4:5",
"m":"http://nextfakeaddress",
"n":"search hsd1.name2.;nameserver 5.5.5.5;nameserver 5.5.6.6;",
"o":"7.2.2.9 Pa;7.2.2.9 Pa;",
"p":"Id1:1030;Id2:19004;Id3:08;Id4:1028801",
"q":"09uop0115613936"
}
]
}

My out put table should look like:

**timestamp|  device_serial_number | receiver_id | mac_address|  a |b| c |d| e| f |g| h| I| j|  m | q 
2001-01-01  00:00:52|uty200866625|u0115613936|p0:n5:02:1d:k8:56|uty200866625|yes|1.5.9.3|a00:b5:c62:1d:e8:f56|US/Eastern|-18000|3600|0|yes|http://fakeaddress|http://nextfakeaddress|09uop0115613936**

In log file upto end of json data will be consider as my one output row or one record. like my 1 st row of output will be .My question is using splunk can we achieve this?

0 Karma

Esteemed Legend

This should get you most of the way there:

... | extract pairdelim=",[\r\n]" kvdelim=":" | rex "gateway device serial number is (?<device_serial_number>.*)" | rex "gateway device receiver id is (?<receiver_id >.*)" | rex "Host mac address is (?<mac_address >.*)" | transaction maxpause=5s | fieldformat _time=strftime(_time, "%Y-%m-%d %H:%M:%S") | table _time device_serial_number receiver_id mac_address a b c d e f g h I j m q
0 Karma