Windows Vista Forums
Vista Forums Home Join Vista Forums Donate Vista Tutorials Tags

Welcome to Vista Forums we are your forum to discuss Windows Vista x64 and x86 systems. Whether you need help or just want to post an idea you have on Vista, this is the forum for you.
Register at Vista forums...the world biggest Windows Vista resource Join Vista Forums Now

Go Back   Vista Forums > Microsoft Technical Newsgroups > PowerShell

Parse Log files into a strongly type Object

Update your Vista Drivers Update Your Drivers Now!!
Closed Thread
 
Thread Tools Display Modes
Old 01-11-2007   #1 (permalink)
Karlos
Guest


 

Parse Log files into a strongly type Object

Fellow humans,

I'm trying to process the following data from a log file into a
strongly typed object for further analysis:-

05-Nov-06 12:00:06.287 ( -1 4360 4356)
dual_flsrv.exe:dual_flsrv.c,v:267: connection accepted from client
SVRPCSA address 158.89.5.16
05-Nov-06 12:00:06.287 ( -1 4360 8220)
dual_flsrv.exe:rcv_u_fl.c,v:165: serving client SVRPCSA address
158.89.5.16

This is the script that I'm using to parse the data into an object:-

switch -file c:\svrlog.txt -regex {

"^\s*(\d\d-\w\w\w-\d\d)\s*(\d\d:\d\d:\d\d\.\d\d\d)\s*(.*[0-9]\))\s*(.*)"
{
$obj = New-Object -TypeName System.Management.Automation.PSObject
$prop = New-Object System.Management.Automation.PSNoteProperty 'Date',
$matches[1]
$obj.PSObject.Properties.Add($prop)
$prop = New-Object System.Management.Automation.PSNoteProperty 'Time',
$matches[2]
$obj.PSObject.Properties.Add($prop)
$prop = New-Object System.Management.Automation.PSNoteProperty 'Code',
$matches[3]
$obj.PSObject.Properties.Add($prop)
$prop = New-Object System.Management.Automation.PSNoteProperty 'Desc',
$matches[4]
$obj.PSObject.Properties.Add($prop)
$obj
}

}


The question is, how can I collect the output of the above script into
a collection of an object for further analysis. e.g say sort on code
and list.

say like $x = switch -file c:\svrlog.txt -regex { .........} (which
doesn't work.)

Any idea's

Many thanks in advance


My System SpecsSystem Spec
Old 01-11-2007   #2 (permalink)
Sung M Kim
Guest


 

RE: Parse Log files into a strongly type Object

"Karlos" wrote:
> The question is, how can I collect the output of the above script into
> a collection of an object for further analysis. e.g say sort on code
> and list.
> say like $x = switch -file c:\svrlog.txt -regex { .........} (which
> doesn't work.)

I have modify the script just a bit so instead of using "-file" switch for
the "switch",
the script will get content and try to match each line in a file
(I haven't tried to change the script more PowerShell like though)

# get the content of log file and match each line one by one
# each matched data will be stored in "$result"
$result = (gc .\log.txt) | % {
switch -regex ($_)
{

"^\s*(\d\d-\w\w\w-\d\d)\s*(\d\d:\d\d:\d\d\.\d\d\d)\s*(.*[0-9]\))\s*(.*)" {
$obj = New-Object -TypeName System.Management.Automation.PSObject
$prop = New-Object System.Management.Automation.PSNoteProperty
'Date', $matches[1]
$obj.PSObject.Properties.Add($prop)
$prop = New-Object System.Management.Automation.PSNoteProperty
'Time', $matches[2]
$obj.PSObject.Properties.Add($prop)
$prop = New-Object System.Management.Automation.PSNoteProperty
'Code', $matches[3]
$obj.PSObject.Properties.Add($prop)
$prop = New-Object System.Management.Automation.PSNoteProperty
'Desc', $matches[4]
$obj.PSObject.Properties.Add($prop)

$obj
}
}}
$result

# the result will display the following
# and the type of $result is "System.Object[]"
Date Time Code
Desc
---- ---- ----
----
05-Nov-06 12:00:06.287 ( -1
4360 4356)
05-Nov-06 12:00:06.287 ( -1
4360 8220)

My System SpecsSystem Spec
Old 01-11-2007   #3 (permalink)
Keith Hill [MVP]
Guest


 

Re: Parse Log files into a strongly type Object

"Karlos" <kbeigan@gmail.com> wrote in message
news:1168531851.839808.105750@o58g2000hsb.googlegroups.com...
> switch -file c:\svrlog.txt -regex {
>
> "^\s*(\d\d-\w\w\w-\d\d)\s*(\d\d:\d\d:\d\d\.\d\d\d)\s*(.*[0-9]\))\s*(.*)"
> {
> $obj = New-Object -TypeName System.Management.Automation.PSObject
> $prop = New-Object System.Management.Automation.PSNoteProperty 'Date',
> $matches[1]
> $obj.PSObject.Properties.Add($prop)
> $prop = New-Object System.Management.Automation.PSNoteProperty 'Time',
> $matches[2]
> $obj.PSObject.Properties.Add($prop)
> $prop = New-Object System.Management.Automation.PSNoteProperty 'Code',
> $matches[3]
> $obj.PSObject.Properties.Add($prop)
> $prop = New-Object System.Management.Automation.PSNoteProperty 'Desc',
> $matches[4]
> $obj.PSObject.Properties.Add($prop)
> $obj
> }
>
> }
>
>
> The question is, how can I collect the output of the above script into
> a collection of an object for further analysis. e.g say sort on code
> and list.
>
> say like $x = switch -file c:\svrlog.txt -regex { .........} (which
> doesn't work.)
>
> Any idea's


Try this:

$x = $(switch .... )

Apparently after the "=" of an assignment statement you can use a command
directly but if you use a language keyword like switch it has to appear in a
subexpression - $().

BTW you could simplify the above:

$obj = new-object psobject
add-member -inp $obj -MemberType NoteProperty Time $matches[2]
....

--
Keith


My System SpecsSystem Spec
Old 01-12-2007   #4 (permalink)
Lee Holmes [MSFT]
Guest


 

Re: Parse Log files into a strongly type Object

You might find this helpful:
http://www.leeholmes.com/blog/parset...Vengeance.aspx


[C:\temp]
PS:10 > $parseExpression =
"^\s*(\d\d-\w\w\w-\d\d)\s*(\d\d:\d\d:\d\d\.\d\d\d)\s*(.*[0-9]\))\s*(.*)"

[C:\temp]
PS:11 > $propertyName = "Date","Time","Code","Description"

[C:\temp]
PS:12 > gc log.txt | Parse-TextObject -ParseExpression
$parseExpression -PropertyName $propertyName

Date Time Code
Description
---- ---- ----
-----------
05-Nov-06 12:00:06.287 ( -1 4360 4356)
dual_flsrv.exe:dual_flsrv....
05-Nov-06 12:00:06.287 ( -1 4360 8220)
dual_flsrv.exe:rcv_u_fl.c,...

--
Lee Holmes [MSFT]
Windows PowerShell Development
Microsoft Corporation
This posting is provided "AS IS" with no warranties, and confers no rights.

"Keith Hill [MVP]" <r_keith_hill@no.spam.thank.u.hotmail.com> wrote in
message news:uY1pCKaNHHA.4000@TK2MSFTNGP06.phx.gbl...
> "Karlos" <kbeigan@gmail.com> wrote in message
> news:1168531851.839808.105750@o58g2000hsb.googlegroups.com...
>> switch -file c:\svrlog.txt -regex {
>>
>> "^\s*(\d\d-\w\w\w-\d\d)\s*(\d\d:\d\d:\d\d\.\d\d\d)\s*(.*[0-9]\))\s*(.*)"
>> {
>> $obj = New-Object -TypeName System.Management.Automation.PSObject
>> $prop = New-Object System.Management.Automation.PSNoteProperty 'Date',
>> $matches[1]
>> $obj.PSObject.Properties.Add($prop)
>> $prop = New-Object System.Management.Automation.PSNoteProperty 'Time',
>> $matches[2]
>> $obj.PSObject.Properties.Add($prop)
>> $prop = New-Object System.Management.Automation.PSNoteProperty 'Code',
>> $matches[3]
>> $obj.PSObject.Properties.Add($prop)
>> $prop = New-Object System.Management.Automation.PSNoteProperty 'Desc',
>> $matches[4]
>> $obj.PSObject.Properties.Add($prop)
>> $obj
>> }
>>
>> }
>>
>>
>> The question is, how can I collect the output of the above script into
>> a collection of an object for further analysis. e.g say sort on code
>> and list.
>>
>> say like $x = switch -file c:\svrlog.txt -regex { .........} (which
>> doesn't work.)
>>
>> Any idea's

>
> Try this:
>
> $x = $(switch .... )
>
> Apparently after the "=" of an assignment statement you can use a command
> directly but if you use a language keyword like switch it has to appear in
> a subexpression - $().
>
> BTW you could simplify the above:
>
> $obj = new-object psobject
> add-member -inp $obj -MemberType NoteProperty Time $matches[2]
> ...
>
> --
> Keith
>
>



My System SpecsSystem Spec
Old 01-13-2007   #5 (permalink)
Kbgo
Guest


 

Re: Parse Log files into a strongly type Object

Lee,

Your function / cmdlet looks really brilliant, however I'm still getting my
head around Powershell, and I'm unsure about how to implement
parse-textobject.

I've read your article and copied the entire contents of the text box from
your blog :

----- Start of textfile: Parse-TextObject.ps1
param([string] $parseExpression, [string[]] $propertyName, [type[]]
$propertyType, [string] $delimiter, [switch] $unitTest)

################################################################################################
##
## Parse-TextObject.ps1 -- Parse a simple string into a custom MshObject.
##
....
....
....

Assert "return-Property2 * 2 should be '56785678'"
$(($return.Property2 * 2) -eq "56785678")
}

Main $input $parseExpression $propertyType $propertyName $delimiter $unitTest
----- end of textfile: Parse-TextObject.ps1

I copyied this file to the c:\Program Files\Windows PowerShell\v1.0\

and then ran this command:

$parExp =
"^\s*(\d\d-\w\w\w-\d\d)\s*(\d\d:\d\d:\d\d\.\d\d\d)\s*(.*[0-9]\))\s*(.*)"
$ObjDef = @("Time","Date","Code","Desc")
$res = gc c:\svr.txt | parse-textobject -ParseExpression:$pasExp
-ObjectDefinition:$ObjDef

---------- svr.txt -------------
26-Dec-06 08:30:46.772 mooring(Mooring.cpp):1124: still alive counter = 6002
26-Dec-06 08:30:46.912 ( -1 4416 22636)
hscopcserv.exe:hscopcservobject.cpp,v:1930: OPC Read error for item 87 with
pointno 59839 and paramno 32755
26-Dec-06 08:30:47.115 GDAerror 0xa8194d14 0x800003e8: Device is off-net
26-Dec-06 08:30:47.115 ( -1 4416 22636)
hscopcserv.exe:hscopcservobject.cpp,v:1930: OPC Read error for item 88 with
pointno 59839 and paramno 32756
26-Dec-06 08:30:47.162 GDAerror 0xa8194d14 0x800003e8: Device is off-net
--------------------------------

The powershell shell reported this :

The term 'parse-textobject' is not recognized as a cmdlet, function,
operable program, or script file. Verify the term
and try again.
At line:1 char:40
+ $res = gc c:\svr.txt | parse-textobject <<<< -ParseExpression $pasExp
-ObjectDefinition $ObjDef

Any ideas?


"Lee Holmes [MSFT]" wrote:

> You might find this helpful:
> http://www.leeholmes.com/blog/parset...Vengeance.aspx
>
>
> [C:\temp]
> PS:10 > $parseExpression =
> "^\s*(\d\d-\w\w\w-\d\d)\s*(\d\d:\d\d:\d\d\.\d\d\d)\s*(.*[0-9]\))\s*(.*)"
>
> [C:\temp]
> PS:11 > $propertyName = "Date","Time","Code","Description"
>
> [C:\temp]
> PS:12 > gc log.txt | Parse-TextObject -ParseExpression
> $parseExpression -PropertyName $propertyName
>
> Date Time Code
> Description
> ---- ---- ----
> -----------
> 05-Nov-06 12:00:06.287 ( -1 4360 4356)
> dual_flsrv.exe:dual_flsrv....
> 05-Nov-06 12:00:06.287 ( -1 4360 8220)
> dual_flsrv.exe:rcv_u_fl.c,...
>
> --
> Lee Holmes [MSFT]
> Windows PowerShell Development
> Microsoft Corporation
> This posting is provided "AS IS" with no warranties, and confers no rights.
>
> "Keith Hill [MVP]" <r_keith_hill@no.spam.thank.u.hotmail.com> wrote in
> message news:uY1pCKaNHHA.4000@TK2MSFTNGP06.phx.gbl...
> > "Karlos" <kbeigan@gmail.com> wrote in message
> > news:1168531851.839808.105750@o58g2000hsb.googlegroups.com...
> >> switch -file c:\svrlog.txt -regex {
> >>
> >> "^\s*(\d\d-\w\w\w-\d\d)\s*(\d\d:\d\d:\d\d\.\d\d\d)\s*(.*[0-9]\))\s*(.*)"
> >> {
> >> $obj = New-Object -TypeName System.Management.Automation.PSObject
> >> $prop = New-Object System.Management.Automation.PSNoteProperty 'Date',
> >> $matches[1]
> >> $obj.PSObject.Properties.Add($prop)
> >> $prop = New-Object System.Management.Automation.PSNoteProperty 'Time',
> >> $matches[2]
> >> $obj.PSObject.Properties.Add($prop)
> >> $prop = New-Object System.Management.Automation.PSNoteProperty 'Code',
> >> $matches[3]
> >> $obj.PSObject.Properties.Add($prop)
> >> $prop = New-Object System.Management.Automation.PSNoteProperty 'Desc',
> >> $matches[4]
> >> $obj.PSObject.Properties.Add($prop)
> >> $obj
> >> }
> >>
> >> }
> >>
> >>
> >> The question is, how can I collect the output of the above script into
> >> a collection of an object for further analysis. e.g say sort on code
> >> and list.
> >>
> >> say like $x = switch -file c:\svrlog.txt -regex { .........} (which
> >> doesn't work.)
> >>
> >> Any idea's

> >
> > Try this:
> >
> > $x = $(switch .... )
> >
> > Apparently after the "=" of an assignment statement you can use a command
> > directly but if you use a language keyword like switch it has to appear in
> > a subexpression - $().
> >
> > BTW you could simplify the above:
> >
> > $obj = new-object psobject
> > add-member -inp $obj -MemberType NoteProperty Time $matches[2]
> > ...
> >
> > --
> > Keith
> >
> >

>
>
>

My System SpecsSystem Spec
Old 01-14-2007   #6 (permalink)
Kbgo
Guest


 

Re: Parse Log files into a strongly type Object


Lee, Many thanks for the excellent response which is now working (my lame
error).

great reusable script,

Cheers


"Lee Holmes [MSFT]" wrote:

> You might find this helpful:
> http://www.leeholmes.com/blog/parset...Vengeance.aspx
>
>
> [C:\temp]
> PS:10 > $parseExpression =
> "^\s*(\d\d-\w\w\w-\d\d)\s*(\d\d:\d\d:\d\d\.\d\d\d)\s*(.*[0-9]\))\s*(.*)"
>
> [C:\temp]
> PS:11 > $propertyName = "Date","Time","Code","Description"
>
> [C:\temp]
> PS:12 > gc log.txt | Parse-TextObject -ParseExpression
> $parseExpression -PropertyName $propertyName
>
> Date Time Code
> Description
> ---- ---- ----
> -----------
> 05-Nov-06 12:00:06.287 ( -1 4360 4356)
> dual_flsrv.exe:dual_flsrv....
> 05-Nov-06 12:00:06.287 ( -1 4360 8220)
> dual_flsrv.exe:rcv_u_fl.c,...
>
> --
> Lee Holmes [MSFT]
> Windows PowerShell Development
> Microsoft Corporation
> This posting is provided "AS IS" with no warranties, and confers no rights.
>
> "Keith Hill [MVP]" <r_keith_hill@no.spam.thank.u.hotmail.com> wrote in
> message news:uY1pCKaNHHA.4000@TK2MSFTNGP06.phx.gbl...
> > "Karlos" <kbeigan@gmail.com> wrote in message
> > news:1168531851.839808.105750@o58g2000hsb.googlegroups.com...
> >> switch -file c:\svrlog.txt -regex {
> >>
> >> "^\s*(\d\d-\w\w\w-\d\d)\s*(\d\d:\d\d:\d\d\.\d\d\d)\s*(.*[0-9]\))\s*(.*)"
> >> {
> >> $obj = New-Object -TypeName System.Management.Automation.PSObject
> >> $prop = New-Object System.Management.Automation.PSNoteProperty 'Date',
> >> $matches[1]
> >> $obj.PSObject.Properties.Add($prop)
> >> $prop = New-Object System.Management.Automation.PSNoteProperty 'Time',
> >> $matches[2]
> >> $obj.PSObject.Properties.Add($prop)
> >> $prop = New-Object System.Management.Automation.PSNoteProperty 'Code',
> >> $matches[3]
> >> $obj.PSObject.Properties.Add($prop)
> >> $prop = New-Object System.Management.Automation.PSNoteProperty 'Desc',
> >> $matches[4]
> >> $obj.PSObject.Properties.Add($prop)
> >> $obj
> >> }
> >>
> >> }
> >>
> >>
> >> The question is, how can I collect the output of the above script into
> >> a collection of an object for further analysis. e.g say sort on code
> >> and list.
> >>
> >> say like $x = switch -file c:\svrlog.txt -regex { .........} (which
> >> doesn't work.)
> >>
> >> Any idea's

> >
> > Try this:
> >
> > $x = $(switch .... )
> >
> > Apparently after the "=" of an assignment statement you can use a command
> > directly but if you use a language keyword like switch it has to appear in
> > a subexpression - $().
> >
> > BTW you could simplify the above:
> >
> > $obj = new-object psobject
> > add-member -inp $obj -MemberType NoteProperty Time $matches[2]
> > ...
> >
> > --
> > Keith
> >
> >

>
>
>

My System SpecsSystem Spec
Old 01-15-2007   #7 (permalink)
Lee Holmes [MSFT]
Guest


 

Re: Parse Log files into a strongly type Object

My pleasure!

--
Lee Holmes [MSFT]
Windows PowerShell Development
Microsoft Corporation
This posting is provided "AS IS" with no warranties, and confers no rights.

"Kbgo" <Kbgo@discussions.microsoft.com> wrote in message
news:19D31DE7-5D28-4DFA-85E3-C0DAC4D8A24E@microsoft.com...
>
> Lee, Many thanks for the excellent response which is now working (my lame
> error).
>
> great reusable script,
>
> Cheers
>
>
> "Lee Holmes [MSFT]" wrote:
>
>> You might find this helpful:
>> http://www.leeholmes.com/blog/parset...Vengeance.aspx
>>
>>
>> [C:\temp]
>> PS:10 > $parseExpression =
>> "^\s*(\d\d-\w\w\w-\d\d)\s*(\d\d:\d\d:\d\d\.\d\d\d)\s*(.*[0-9]\))\s*(.*)"
>>
>> [C:\temp]
>> PS:11 > $propertyName = "Date","Time","Code","Description"
>>
>> [C:\temp]
>> PS:12 > gc log.txt | Parse-TextObject -ParseExpression
>> $parseExpression -PropertyName $propertyName
>>
>> Date Time Code
>> Description
>> ---- ---- ----
>> -----------
>> 05-Nov-06 12:00:06.287 ( -1 4360
>> 4356)
>> dual_flsrv.exe:dual_flsrv....
>> 05-Nov-06 12:00:06.287 ( -1 4360
>> 8220)
>> dual_flsrv.exe:rcv_u_fl.c,...
>>
>> --
>> Lee Holmes [MSFT]
>> Windows PowerShell Development
>> Microsoft Corporation
>> This posting is provided "AS IS" with no warranties, and confers no
>> rights.
>>
>> "Keith Hill [MVP]" <r_keith_hill@no.spam.thank.u.hotmail.com> wrote in
>> message news:uY1pCKaNHHA.4000@TK2MSFTNGP06.phx.gbl...
>> > "Karlos" <kbeigan@gmail.com> wrote in message
>> > news:1168531851.839808.105750@o58g2000hsb.googlegroups.com...
>> >> switch -file c:\svrlog.txt -regex {
>> >>
>> >> "^\s*(\d\d-\w\w\w-\d\d)\s*(\d\d:\d\d:\d\d\.\d\d\d)\s*(.*[0-9]\))\s*(.*)"
>> >> {
>> >> $obj = New-Object -TypeName System.Management.Automation.PSObject
>> >> $prop = New-Object System.Management.Automation.PSNoteProperty 'Date',
>> >> $matches[1]
>> >> $obj.PSObject.Properties.Add($prop)
>> >> $prop = New-Object System.Management.Automation.PSNoteProperty 'Time',
>> >> $matches[2]
>> >> $obj.PSObject.Properties.Add($prop)
>> >> $prop = New-Object System.Management.Automation.PSNoteProperty 'Code',
>> >> $matches[3]
>> >> $obj.PSObject.Properties.Add($prop)
>> >> $prop = New-Object System.Management.Automation.PSNoteProperty 'Desc',
>> >> $matches[4]
>> >> $obj.PSObject.Properties.Add($prop)
>> >> $obj
>> >> }
>> >>
>> >> }
>> >>
>> >>
>> >> The question is, how can I collect the output of the above script into
>> >> a collection of an object for further analysis. e.g say sort on code
>> >> and list.
>> >>
>> >> say like $x = switch -file c:\svrlog.txt -regex { .........} (which
>> >> doesn't work.)
>> >>
>> >> Any idea's
>> >
>> > Try this:
>> >
>> > $x = $(switch .... )
>> >
>> > Apparently after the "=" of an assignment statement you can use a
>> > command
>> > directly but if you use a language keyword like switch it has to appear
>> > in
>> > a subexpression - $().
>> >
>> > BTW you could simplify the above:
>> >
>> > $obj = new-object psobject
>> > add-member -inp $obj -MemberType NoteProperty Time $matches[2]
>> > ...
>> >
>> > --
>> > Keith
>> >
>> >

>>
>>
>>



My System SpecsSystem Spec
Closed Thread

Thread Tools
Display Modes



Similar Threads
Thread Thread Starter Forum Replies Last Post
Passing credential object - what's the type? bobuva PowerShell 2 05-09-2008 10:57 AM
Re: Unable to cast COM object of type 'ADODB.RecordsetClass' to class type 'System.Object[]' Michel Posseth [MCP] .NET General 2 03-27-2008 10:04 AM
Parse XML files from Powershell? Duncan Smith PowerShell 8 03-21-2007 07:31 AM
parse just ip addresses from syslog files Steve Schofield PowerShell 4 01-17-2007 12:08 AM
Parse Log files into a strongly type Object Karlos PowerShell 1 01-11-2007 05:16 PM


Update your Vista Drivers Update Your Drivers Now!!

Vistax64.com is an independent web site and has not been authorized,
sponsored, or otherwise approved by Microsoft Corporation.
"Windows Vista", the Start Orb, and related materials are trademarks of Microsoft Corp.
© Designer Media 2005-2008