POD Standards
ptum
created: 2006-01-06 14:17:37

I've recently embarked on a quest to bring the POD (Plain 'Ole Documentation) in our shop up to some level of measurable quality (or at least density/quantity). I half-heartedly googled around and did a Super Search here at PM but didn't come up with much in the way of a standardized way to do POD. So I decided to write my own rules. Here's a first draft:

  1. All Perl modules should be documented with POD.
  2. All Perl scripts (anything with the shebang line) should be documented with POD.
  3. All POD documentation should include, at a minimum, the following =head1 sections:
    • DESCRIPTION -- This should be a paragraph or two about the purpose(s) for the module/script and a high-level narrative about what it does and how it works.
    • SYNOPSIS -- This should be an example as to how one would use a module or call a script.
      • if the script is intended to be directly executed, it should also include a =head2 section for PARAMETERS, outlining each argument or parameter which can be passed to the script
    • AUTHOR -- This should include at least the last name of the person who 'owns' or wrote the script or module.
  4. Each public method or subroutine should be documented, using an =item or =headN tag to identify it by name, matching the subroutine name.
  5. Private methods or subroutines can be commented rather than POD-ed, but their names should follow the leading underscore convention so they can be easily excluded from any coverage calculations.
  6. Additional POD =headN sections can be used optionally: METHODS, NAME, SEE ALSO, PUBLIC, PRIVATE, COPYRIGHT, etc.
  7. All POD should pass podchecker (POD::Checker) with no errors

I've rolled my own customized substitute for POD::Coverage to profile our codebase documentation; it depends on (or at least expects) our code POD to adhere to these standards. So far it is being well-received by the other developers, although they may just be humoring me. :)

This seems to be an area where a little structure may go a long way ... I've tried to keep the standard pretty minimal to avoid makework. Recommendations welcome.

Re: POD Standards
created: 2006-01-06 14:29:30
I'm not sure how NAME could be considered optional. And I would consider COPYRIGHT to be essential for all but in-house modules (and important even then).
Re: POD Standards
created: 2006-01-06 14:53:20
Hi

You might look at item 3 of this article by TheDamian to see what his template for a module is. I found it very useful.

There was a similar template for scripts on the London PM mailing list, but it currently seems to be having a massage.

- j

Re: POD Standards
xdg
created: 2006-01-06 15:36:51

Also worth noting perlmodstyle. It has a section on documenting with Pod.

-xdg

Code written by xdg and posted on PerlMonks is public domain. It is provided as is with no warranties, express or implied, of any kind. Posted code may not have been tested. Use of posted code is at your own risk.

Re: POD Standards
created: 2006-01-06 15:58:52

An alternative viewpoint is that for anything other than temporarially blocking out a chunk of code, POD should not be used. If these are programmers comments, then use comment cards.

If this is proper documentation, then use a proper documetnation tool/markup. Something that knows how to:

  • build tables;
  • include diagrams;
  • build tables of contents;
  • create indexes;
  • knows how to hyperlink;

If you use the lowest common denominator, you reap what you sow.


Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
Lingua non convalesco, consenesco et abolesco. -- Rule 1 has a caveat! -- Who broke the cabal?
"Science is about questioning the status quo. Questioning authority".
In the absence of evidence, opinion is indistinguishable from prejudice.
Re^2: POD Standards
created: 2006-01-06 16:00:57
Pod::PseudoPod knows how to do most of that.
Re^3: POD Standards
created: 2006-01-06 16:10:08

Which is great if I want to have to write a parser to extract the documentation from the source file, convert it to another format, write it to a separate file and use another tool to view it--but why?

Why not just write it in that destination format to start with?


Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
Lingua non convalesco, consenesco et abolesco. -- Rule 1 has a caveat! -- Who broke the cabal?
"Science is about questioning the status quo. Questioning authority".
In the absence of evidence, opinion is indistinguishable from prejudice.
Re^4: POD Standards
created: 2006-01-06 16:13:50
Well I just use a half-dozen line pod.cgi or pseudopod.cgi and simply type in the name of the module as a query to the cgi and I see the pod in HTML. No manual parsing, no writing to a separate file, only a single tool to auto-parse and display.
Re^5: POD Standards
created: 2006-01-06 16:45:45
I just use a half-dozen line pod.cgi or pseudopod.cgi

Which means that everyone who wants to read the documentation has to be running a webserver.

And every time anyone wants to read a document, it has to be reparsed. Over and over all over the world.

And that pre-supposes that you know which document contains the information you need. If you come across an unfamiliar language construct in someone elses source code, there are about twenty different places you might find the relevant documentation. Once you found it, the documentation is generally quite good, and well maintained. But finding it? Oh boy!

As a for instance, if you do not already know which core document CHECK{} blocks are documented in, try finding it:

P:\test>perldoc CHECK
No documentation found for "CHECK".

P:\test>perldoc /f CHECK
No documentation found for "/f".
No documentation found for "CHECK".

P:\test>perldoc -f CHECK
No documentation for perl function `CHECK' found

P:\test>perldoc -q CHECK
Found in C:\Perl\lib\pod\perlfaq4.pod
  How do I verify a credit card checksum?
    Get the Business::CreditCard module from CPAN.

Found in C:\Perl\lib\pod\perlfaq8.pod
  How do I check whether input is ready on the keyboard?
    The easiest way to do this is to read a key in nonblocking mode with the
    Term::ReadKey module from CPAN, passing it an argument of -1 to indicate
    not to block:

        use Term::ReadKey;

        ReadMode('cbreak');

        if (defined ($char = ReadKey(-1)) ) {
            # input was waiting and it was $char
        } else {
            # no input was waiting
        }

        ReadMode('normal');                  # restore normal tty settings

Found in C:\Perl\lib\pod\perlfaq9.pod
  How do I check a valid mail address?
    You can't, at least, not in real time. Bummer, eh?

    Without sending mail to the address and seeing whether there's a human
[SNIP]

There are many more examples.

Even once converted to html per AS docs, there is still no easy way to find where in the IO::Socket::INET docs the recv call is documented. Or where in the LWP::* suite you should look to find the numeric/text mapping for HTTP error codes.


Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
Lingua non convalesco, consenesco et abolesco. -- Rule 1 has a caveat! -- Who broke the cabal?
"Science is about questioning the status quo. Questioning authority".
In the absence of evidence, opinion is indistinguishable from prejudice.
Re^6: POD Standards
created: 2006-01-06 17:15:27
Ok, point taken, you're right, POD's not perfect. What do you use instead?
Re^7: POD Standards
created: 2006-01-06 17:22:13

In previous lives, HTML and a decent html editor.

If I were starting a project tomorrow, it would be http://www.stack.nl/~dimitri/doxygen/.


Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
Lingua non convalesco, consenesco et abolesco. -- Rule 1 has a caveat! -- Who broke the cabal?
"Science is about questioning the status quo. Questioning authority".
In the absence of evidence, opinion is indistinguishable from prejudice.
Re^7: POD Standards
created: 2006-01-06 19:12:53
I use three types of documentation: POD, mostly for Perl (and C and shell) programs, and only to generate manual pages from. LaTeX for most things that exceeds manual pages (long time ago, I used to use *roff and friends for that). And plain ASCII for anything else (and that's quite a lot).

But to solve the problem of finding the right documentation, you need an index. (La)TeX is good in generating indices.

Perl --((8:>*
Re^6: POD Standards
created: 2006-01-08 07:33:59
As a for instance, if you do not already know which core document CHECK{} blocks are documented in, try finding it:

P:\test>perldoc

I don't get it what you are trying to prove. perldoc doesn't search (except for the faq), nor does it do an index lookup yet (except for functions). It's pod doesn't claim that it does. Once you've learned that, you wouldn't ever try any of that again. In fact, I'd hope you'd do perldoc perldoc and not even bother in the first place.

Something that does work? Look at perltoc, search for CHECK, go to the indicated page (perlmod). If that doesn't work, use an actual search tool.

That said, work is being done to add X<> indexing tags to the core pods; I'd hope perldoc would take advantage of them someday soon.

Re^7: POD Standards
created: 2006-01-08 08:31:11

I'm not trying to prove anything.

I'm trying to show that whilst the information contained within the current documentation set is generally very good. All of the time and effort that went into producing and maintaining that information is let down by the inadaquate means of locating the information you require. If you know where to look, you know, but if you don't, you are stymied.

Manually produced indexes are no substitute for machine generated TOCs and indexes.

And grepping your entire perl installation is no substitue either. Being quite selective about what I grepped, a search for "CHECK" turns up 288 references spread across 67 files. Most of those are embedded within html markup which makes trying to subset the list visually very difficult.

But mostly, I gave the OP an alternate viewpoint to his question, and posted a little justification for why I arrived at that alternate viewpoint in direct response to questions asked of me.

If POD has merit, then it can surely stand up to a little scrutiny?

I don't consider that it does stand up to scrutiny and for me, both defenses of POD--your "use a proper [...] tool", and jZed's "but there is Pos::PseudoPod"--better confirm ('prove') my conclusions than anything I could have said.

However, with the information available, the OP can and will make up his mind what is correct for his purposes. Whichever way he chooses to go, the provision of the alternative viewpoint will not have harmed his decision making process.


Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
Lingua non convalesco, consenesco et abolesco. -- Rule 1 has a caveat! -- Who broke the cabal?
"Science is about questioning the status quo. Questioning authority".
In the absence of evidence, opinion is indistinguishable from prejudice.
Re^8: POD Standards
created: 2006-01-08 08:41:25
I don't consider that it does stand up to scrutiny and for me, both defenses of POD--your "use a proper [...] tool",
Just to clarify, I wasn't trying to defend Pod, I was trying to defend perldoc. Pod is a file format, and as such doesn't really have a designated user interface. You are perfectly free to brew your own.
Re^2: POD Standards
created: 2006-01-06 16:35:58
So which is your preferrred proper documentation tool/markup?
Re^3: POD Standards
created: 2006-01-06 17:13:21

I am undecided--Doxegen looks pretty good, as do one or two of the wiki style tools--javadoc has a lot going for it (except the presence of java in the name:).

I really like the D language approach of allowing you to embed the code within an html document. The compiler simply ignores the surrounding HTML.

Haskell has a similar concept called Literate Haskell. Again, the source code is embedded within the documentation, and the compiler simple ignores everything except the source code, in this case indicated by the presence of a '>' in the first column of the line. If that isn't present, the compiler ignores the line.

Generally speaking, html is not the greatest markup in the world, but it is, or at least can be, pretty simple, as evidenced by this site, and there are dozens if not hundreds of tools around for editing, indexing, creating TOCs etc.

It is also as ubiquitous as anything is ever likely to be. It would be a rare machine these daya that doesn't have some form of HTML browser available, even if it is a text-only one like lynx.

And the tools available go much further in accomodating the needs of the user. The browser I am using allows me to zoom the display (increasing the size of fonts and graphics using Ctrl-mousewheel, which is very useful for those who's eyesight is not so good. It also has a (downloadable) option to verbalise the contents--if you are completely blind. HTML is far from perfect, but it isn't going away anytime soon, and it does bring a wealth of useful tools with it.

And for the CLI-only crowd, lynx is far more useable when browsing nested documentation (like IO::Socket*/ LWP::*) than perldoc or man.


Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
Lingua non convalesco, consenesco et abolesco. -- Rule 1 has a caveat! -- Who broke the cabal?
"Science is about questioning the status quo. Questioning authority".
In the absence of evidence, opinion is indistinguishable from prejudice.
Re: POD Standards
created: 2006-01-06 16:02:19
About five years ago, I built a skeleton for Perl consisting of:

#! /usr/local/bin/perl

package FILL-IN-THE-NAME;

use strict;
use warnings;

CODE-GOES-HERE

1;

__DATA__

=pod

=head1 NAME

=head1 SYNOPSIS

=head1 DESCRIPTION

=head1 EXAMPLES

=head1 AUTHOR

=cut
Basically, I took the format of the UNIX man() pages and added the 'EXAMPLES' header. In addition, I have occasionally added 'TODO', 'NOTES', or 'SEE ALSO' headers when they were warranted. As I write the code, I update the POD.

I did this for a project that had all the ear-marks of becoming humongous (due to a certain lack of detail in the specifications and requirements). I could see myself quickly losing what little mind I had left if I didn't take steps. So I cooked up the template and started using it religiously.

Six months into the mess, I (finally) got another body assigned to the project. She was somewhat boggled by the idea of doing the Doco as you went along, but she came around.

What convinced her was her first bi-weekly 'status review meeting' with the "Project Team". I wandered in with a hundred and thirty-five pages of printed POD and spent the next three hours answering questions with variations on a theme of: 'Yes, that has been implemented, and see page mumble for the Doc'; and 'No, we can't do that, it will break the interface to X, unless we do the following .... ". (I wrote a bit of code that generated an Index from the 'NAME' headers and a Quick-Ref from the 'SYNOPSIS' headers to pull this off.)

Over the next eighteen months that 'structured documentation' saved our bacon so many times....

----
I Go Back to Sleep, Now.

OGB

Re: POD Standards
created: 2006-01-06 16:05:59
Private methods or subroutines can be commented rather than POD-ed, but their names should follow the leading underscore convention so they can be easily excluded from any coverage calculations.

I use normal POD for subroutines, but use the "=begin" pod command designate a different translator to process those sections. I call my programmer documentation sections "programmers" docs, and I created a little "translator program" that just identifies the programmerdocs sections, strips them out, and passes them on to the standard pod translators.

That way, 'perldoc' gives me the user documentation, and my 'programmerdocs' script gives me the programmer documentation. This also allows a tester who is unfamiliar with the program to black box test each function without knowing the internals of the code: if the code doesn't match it's documentation, one of them needs fixing. This would be more useful if we had a testing department, but hopefully it's still of use to document.

I consistantly use the same template for documenting each function: first the name and arguments, then a one line description of the purpose of the function, then a detailed description, then the return values section, the global variables used (indicating read/write status), any preconditions (such as open filehandles, etc.), any postconditions (program termination conditions, warnings emitted, filesystem modifications, etc).

By documenting the pre and post conditions carefully, I usually end up thinking about the error handling of my functions (ie. "the routine emits a warning: but should it throw an exception instead?" or "The program dies if we hit an error condtion. Can/should it recover from this state instead?"). Thinking about formally documenting my decisions makes me think harder about the justifications I have for making each of those choices.

That's what works for me. I don't claim it should work for anyone else, though. :-)
--
Ytrew

Re: POD Standards
created: 2006-01-06 18:50:45
Back in the good old days, pod2man would complain if you would not have a NAME section (excpet if you used the --lax option). All my documentation starts with NAME, SYNOPSIS and DESCRIPTION, as good manual pages do. For me, POD is primary used to generate man pages, and hence, it should follow the man pages conventions, hence the first three sections. And while in extreme cases a SYNOPSIS or a DESCRIPTION may be omitted, NAME would not be optional.

As for other sections, BUGS, SEE ALSO, COPYRIGHT/LICENSE, AUTHOR/MAINTAINER, and HISTORY, I use frequently. To a lesser extent: EXAMPLES, FILES (if additional files, typically configuration files, are used by the program), STANDARDS (if it conforms to standards), REFERENCES (for instance, to literature).

I presume that you list methods in a section METHODS. I would describe methods in the DESCRIPTION section. I've no idea what to expect from sections named PUBLIC and PRIVATE.

Private methods or subroutines can be commented rather than POD-ed, but their names should follow the leading underscore convention so they can be easily excluded from any coverage calculations.
I would not document private methods in the POD - as I said, for POD is to generate manual pages. That is, user documentation. It should document the interface: what to pass in to the program, subroutines or methods, and what to expect to happen in return (return values, side-effects). Discussion of the internals should be kept to a minimum, and only done if relevant. Hence, no private methods.

I don't like having a (potentially) large number of subroutines starting with an underscore. It looks unnatural and distracts. Doing so for the reason you mention is double bad. Programming languages shouldn't force arbitrary rules upon a programmer - it's the programming language that should be the slave, and the programmer the master. Not the other way around.

Perl --((8:>*
Re: POD Standards
created: 2006-01-07 00:24:23
Please be sure to look at Test::Pod and Test::Pod::Coverage to automate your checking. I think that Test::Pod::Coverage ought to be extensible enough to handle your substitute.

xoxo,
Andy

Re^2: POD Standards
created: 2006-01-07 14:24:12
But if you include your POD tests as part of your distributed module, please consider using "skip unless require Test::Pod" logic.

I recently tried to install a module on my ISP and it installed and tested fine except the one test that depended on Test::POD which failed because Test::POD wasn't installed. Off CPAN.pm went to intall Test::POD and its many prereqs and somewhere in that very long list it came on something that needed Module::Build which refused to install (No, please let's not get into a MakeMaker/Build war). Yes the situation is b0rked and I should fix it, but I needed the original module running right away. I ended up doing a force install on the original module since its only failed test was the one requiring Test::POD.

Personally, I don't think it's the purpose of a distribution test to refuse to install because it can't test its POD. Hooray, your module gets a perfect phalanx score. Boo, it cost me an extra twenty minutes to install it.

Re^3: POD Standards
created: 2006-01-07 20:55:22
I agree 100%. That's why the sample 4-line code that I suggest people use as t/pod.t does that exclusion.

xoxo,
Andy

Re^4: POD Standards
created: 2006-01-10 12:01:56
Um, where? I can't find your "4-line code". Thanks.
Re^5: POD Standards
created: 2006-01-11 11:56:26
[http://search.cpan.org/dist/Test-Pod/Pod.pm]
use Test::More;
eval "use Test::Pod 1.00";
plan skip_all => "Test::Pod 1.00 required for testing POD" if $@;
all_pod_files_ok();

xoxo,
Andy

perlmonks.org content © perlmonks.org and Anonymous Monk, BrowserUk, jimbojones, jZed, Old_Gray_Bear, palmerp, Perl Mouse, petdance, ptum, xdg, ysth

prlmnks.org © 2006 edmund von der burg (eccles & toad)

v 0.03