[Perldl] How to find out cause of out of memory

David Mertens dcmertens.perl at gmail.com
Wed Feb 15 01:37:06 HST 2012


Mark (and anybody else following along) -

First, to your idea. Your code creates a line lookup table for a text file.
This isn't data compression and you're not loading all 3 gigabytes of data
into 40 megabytes of ram. You're simply creating a lookup table that is 40
megabytes which allows you to quickly locate the position of the data you
need. The system still has to go back to the disk to fetch the data with

    # I'm using $fh instead of DATA here:
    seek $fh, $offset[$entered], 0;
    $line = <$fh>;

Second, to your code. This code works, but not for the reasons you might
suspect. Try running your code under "use strict" and you will see errors.
Why? Every time you surround code with a block you create a lexical scope.
Any "my" variables within that scope are not accessible outside that scope
(PadWalker shenanigans aside). As such, lines like

  {my $line_num = 2;}

do the following: (1) creates a scoped block, (2) creates a lexical
variable within that scoped block called , (3) assigns the value of 2 to
that lexical variable, and (4) exits the scoped block, destroying all
lexicals in the process. Your code works because you are essentially using
global variables instead of lexical variables without realizing it and
Perl's default interpretation of undef in numeric context evaluates to
zero. In more modern and idiomatic Perl, your code might look like this:

-------------------%<-------------------

use strict;
use warnings;

# Get the file:
print "What file would you like to examine? ";
chomp (my $file_name = <>);
die "File does not exist!\n" unless -f $file_name;
open my $fh, '<', $file_name or die "Unable to open $file_name\n";

# Build lookup table (recall: $. is the current record number)
my @offset = (0);
$offset[$.] = tell $fh while (<$fh>);

# Retrieve data for the user:
while (1) {
    # Get the requested line number
    print "Give me a line offset between 0 and $#offset (or q to quit): ";
    chomp (my $line = <>);
    # Exit if that's what they want
    exit if lc $line eq 'q';
    # Verify it:
    if ($line =~ /^\d+$/ and $line < @offset) {
        # Go to the lookup position and read the data:
        seek $fh, $offset[$line], 0;
        my $data = <$fh>;
        # No need to chomp; we need the \n anyway:
        print $data;
    }
    else {
        print "Bad input, try again\n";
    }
}

------------------->%-------------------

the trick here is to put your file data into decimal
> then to pack the file data like this
> ##################################################
> $data = pack "w*", $_;
> #################################################
> then just use unpack to view the numerical data
>

What are you doing here? I've never seen the 'w' pack template, but perldoc
tells me it is "a BER compressed integer." And you're only packing a single
number, whatever happens to be in "it" at the moment.

David

-- 
 "Debugging is twice as hard as writing the code in the first place.
  Therefore, if you write the code as cleverly as possible, you are,
  by definition, not smart enough to debug it." -- Brian Kernighan
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.jach.hawaii.edu/pipermail/perldl/attachments/20120215/33c05a2e/attachment.html>


More information about the Perldl mailing list