Er, this is an extremely solved problem. Why don't you use one of the fifteen million solutions already in existance?

Are you sure you want to use fixed-length record sizes for forum posts? Do you have a small maximum length for the body of the post? If not, all of your records will be the size of the largest possible record -- and if that's not small, you'll have to do a lot of IO to read each message from the disk, pull it through the cache, and get it where perl can handle it. If you really think that will improve your speed (and I don't, even without benchmarking it), you need some sort of unique ID per message which you can use as an offset into the file. Multiply that by the size of a record and use seek and sysread on a filehandle to extract just the record you want.

If that doesn't seem fun to you (and it's not), you could use a unique record separator between records and let CPAN://Tie::File take care of reading and writing.

If this were my project, I would use CPAN://DBD::SQLite and avoid all of the tedious mucking about with O(n) access and let indexed columns get that down to O(log n) or better, while not having to process all of the data myself. Please consider that instead.

Taking your performance/io problems in Perl always reads in 4K chunks and writes in 1K chunks... Loads of IO! into account, i think you could use a flat file for storing post titles and a reference to a file that contains the actual post.

This node was taken out by the NodeReaper on 2006-01-02 01-56-18
Reason: [chromatic]: (reap) personal attack

You may view the original node and the consideration vote tally.

See pack and unpack for record handling; sysopen, sysseek, sysread, syswrite for retrieving or writing records; and flock for keeping the files from being modified during writes. As mentioned above, post contents should be in a separate file, with the post record containing just start and end offsets. This keeps your records neat and fixed-length. If your users are allowed to do edits, you should allocate a few extra characters for post content so small increases don't require moving it to a new spot in the file. After the time limit for edits is reached, your clean-up utility can rewrite the content to exact size.

It would, however, be simpler to just do this with mySQL. Certain things like indexes can get quite complicated to program.