Windows Vista Forums
Vista Forums Home Join Vista Forums Windows 7 Forum Vista Tutorials Tags
Welcome to Windows Vista Forums. Our forum is dedicated to helping you find solutions with any problems, errors or issues you are experiencing with Windows Vista. The Vista forum also covers news and updates and has an extensive Windows Vista tutorial section that covers a wide range of tips and tricks.

Go Back   Vista Forums > Misc Newsgroups > PowerShell

Vista - Read a huge text file from bottom up

Reply
 
Old 04-18-2008   #1 (permalink)
Hayato Iriumi


 
 

Read a huge text file from bottom up

I just wanted to share some code with the community today. I had a
need to take a look at a few thousand bytes of a huge text file from
the bottom and I was able to do it in PowerShell.

[Problem]
I manage build servers at my company and somehow the build process
generated a huge text file. The size was well over 600MB and opening
it in Notepad would not have been an option. I didn't even try. So I
had to come up with a way to peek at the last part of the huge text
file.

[Solution]
I thought of Get-Content Cmdlet, but I couldn't find any option that
allowed me to read text file from the bottom. So I turned to .NET
Framework. Here is the code. Please change $TextFilePath and
$BytesToRead as you see fit and please feel free to suggest anything
if there is better to accomplish this.

$TextFilePath = "D:\Temp\HugeTextFile"
$BytesToRead = 2048

$fs = [System.IO.File]::OpenRead($TextFilePath)
$fs.Position = $fs.Length - $BytesToRead

$sr = New-Object System.IO.StreamReader($fs)

$text = $sr.ReadToEnd()

$sr.Close()
$fs.Close()

$text

My System SpecsSystem Spec
Old 04-19-2008   #2 (permalink)
Bob Landau


 
 

RE: Read a huge text file from bottom up

Thanks, I'm going to save this snippet.

I've also wanted the last M liines in a file while this may not do exactly
that for practical purposes its much faster than reading the entire file just
to return the last 10 lines.

Like you said you can aways adjust the bytes to read

"Hayato Iriumi" wrote:
Quote:

> I just wanted to share some code with the community today. I had a
> need to take a look at a few thousand bytes of a huge text file from
> the bottom and I was able to do it in PowerShell.
>
> [Problem]
> I manage build servers at my company and somehow the build process
> generated a huge text file. The size was well over 600MB and opening
> it in Notepad would not have been an option. I didn't even try. So I
> had to come up with a way to peek at the last part of the huge text
> file.
>
> [Solution]
> I thought of Get-Content Cmdlet, but I couldn't find any option that
> allowed me to read text file from the bottom. So I turned to .NET
> Framework. Here is the code. Please change $TextFilePath and
> $BytesToRead as you see fit and please feel free to suggest anything
> if there is better to accomplish this.
>
> $TextFilePath = "D:\Temp\HugeTextFile"
> $BytesToRead = 2048
>
> $fs = [System.IO.File]::OpenRead($TextFilePath)
> $fs.Position = $fs.Length - $BytesToRead
>
> $sr = New-Object System.IO.StreamReader($fs)
>
> $text = $sr.ReadToEnd()
>
> $sr.Close()
> $fs.Close()
>
> $text
>
My System SpecsSystem Spec
Old 04-19-2008   #3 (permalink)
Hayato Iriumi


 
 

Re: Read a huge text file from bottom up

Hi, Bob.
Quote:

> I've also wanted the last M liines in a file while this may not do exactly
> that for practical purposes its much faster than reading the entire file just
> to return the last 10 lines.
I guess it depends on what kind of text file you have. I personally
think it's safer to return certain number of bytes rather than number
of lines because the text file could be an XML file that has only one
line. In that case, you would be reading the entire file.

I was thinking... Maybe I could make this a custom Cmdlet. Hmm... but
it may be worth it because you can accomplish this in such a small
amount of effort accessing .NET Framework. I hope PowerShell team will
include this kind of Cmdlet in the next release.
My System SpecsSystem Spec
Old 04-19-2008   #4 (permalink)
Hayato Iriumi


 
 

Re: Read a huge text file from bottom up

> I was thinking... Maybe I could make this a custom Cmdlet. Hmm... but
Quote:

> it may be worth it because you can accomplish this in such a small
> amount of effort accessing .NET Framework. I hope PowerShell team will
> include this kind of Cmdlet in the next release.
oops, I meant it may NOT be worth it.
My System SpecsSystem Spec
Old 04-25-2008   #5 (permalink)
Keith Hill [MVP]


 
 

Re: Read a huge text file from bottom up

"Hayato Iriumi" <hiriumi@xxxxxx> wrote in message
news:0ea7b860-14ce-477b-9d2b-07f1b90afa43@xxxxxx
Quote:

> Hi, Bob.
>
Quote:

>> I've also wanted the last M liines in a file while this may not do
>> exactly
>> that for practical purposes its much faster than reading the entire file
>> just
>> to return the last 10 lines.
>
> I guess it depends on what kind of text file you have. I personally
> think it's safer to return certain number of bytes rather than number
> of lines because the text file could be an XML file that has only one
> line. In that case, you would be reading the entire file.
>
> I was thinking... Maybe I could make this a custom Cmdlet. Hmm... but
> it may be worth it because you can accomplish this in such a small
> amount of effort accessing .NET Framework. I hope PowerShell team will
> include this kind of Cmdlet in the next release.
Ideally, the team could add an "offset" parameter to Get-Content to allow
this use case. You should suggest it on http://connect.microsoft.com.

--
Keith

My System SpecsSystem Spec
Reply

Thread Tools


Similar Threads
Thread Forum
Read a line from a text file, without loading the entire file inmemory PowerShell
How to read text file VB Script
How to read and write a text file? VB Script
How do I read a text file and sort text by fixed positions? PowerShell
Using ADO to read a text file PowerShell


Vista Forums is an independent web site and has not been authorized,
sponsored, or otherwise approved by Microsoft Corporation.
"Windows Vista", the Start Orb, and related materials are trademarks of Microsoft Corp.
© Designer Media Ltd

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46