Perl Tail ?
Hi,
In windows I am using a nice piece of perl code to tail a log file. It does a nice job until the file becomes very large, ie over 20 meg, then the perl process starts to be a memory hog. I tried using a cygwin tail, but it locks the file so the original process can't write to it. This perl code does NOT lock the file. Can someone suggest how I might grab new chunks of a large file efficiently ? The current code is like this
while(-s $file ) {
open(TF,$file) || last ;
seek(TF,$curpos,0);
@lines=;
$cursorposition = tell(TF);
last if ((stat(_))[7] < $cursorposition);
foreach $nline (@lines){
$newline = substr $nline, 24;
# write out to tailfile $newline
}
close(TF);
sleep 1;
}
}
else
{ sleep 5; }
}
Thanks
Re: Perl Tail ?
created: 2005-10-04 11:31:01
Use [mod://File::ReadBackwards]. It's very efficient.
Update:
my $file_name = 'file.log';
my $n = 5;
my $fh = File::ReadBackwards->new($file_name)
or die("Can't read input file: $!\n");
my @lines;
for (1..$n) {
my $line = $fh->readline();
last if not defined $line;
unshift(@lines, $line);
}
print foreach @lines;
Re:
created: 2005-10-04 11:32:02
Is there any need for you to read in the whole file?
$\=~s;s*.*;q^|D9JYJ^^qq^\//\\\///^;ex;print
@lines=Why not simply read line by line and don't store them?;
$\=~s;s*.*;q^|D9JYJ^^qq^\//\\\///^;ex;print
Re^2:
created: 2005-10-04 11:46:14
So this is an efficient way to do the Unix equivalent ?
tail -f logfile > tailfileI would like to tail it for 10 minutes, restart the tail process for another 10 mintues, copying off tailfile to tailfile.chunk before writing to tailfile again. COuld I use this to do it ? I will check tail archives too.
Re^3:
created: 2005-10-04 14:04:58
I recall many years ago that unix tail is broken on large files, such that I wrote my own Perl tail for one job. You might want to look at tail for Perl Power Tools.
I would like to tail it for 10 minutes, restart the tail process for another 10 mintues, copying off tailfile to tailfile.chunk before writing to tailfile againPlease be more specific. Are you intending to skip (lose) 10 minutes of logfile? Do you just want to split the logfile into chunks at 10 minute intervals?
What is the fundamental problem you're trying to solve?
-QM
--
Quantum Mechanics: The dreams stuff is made of
Re^4:
created: 2005-10-04 16:28:50
upon reflection I would like to make my posted code NOT read from the begining. that is all really.
thanks
Re: Perl Tail ?
created: 2005-10-04 11:34:26
Re: Perl Tail ?
created: 2005-10-04 11:36:43
A super search on tail will give examples of how to do this, including my post node 353907.
--
Oh Lord, wont you burn me a Knoppix CD ?
My friends all rate Windows, I must disagree.
Your powers of persuasion will set them all free,
So oh Lord, wont you burn me a Knoppix CD ?
(Missquoting Janis Joplin)
Re: Perl Tail ?
created: 2005-10-04 11:50:03
i tried Perl Power Tools v0.14: tail and it seems to work ok in windows. i havent tried it on a 20 meg file though...
hth
hth
Re: Perl Tail ?
created: 2005-10-04 18:57:01
I tried using a cygwin tail, but it locks the file so the original process can't write to it.
LOL -- that's the best joke I've seen about MS Windows in a while. I suppose it could just be taken as a (yet another?) case of midguided implementation on the part of cygwin, but based on the whole brain-damaged approach to file locking foisted by MS, cygwin should not take all the blame (or should at least admit only to justifiable failure).