I'm fact-checking a tutorial presentation I've written. Does anyone know if any of the [mod://Data::Dumper] clones have a hooking mechanism similar to [mod://Storable]? E.g.:
package My::Class;
sub STORABLE_freeze {
# my custom freezing routine
}
sub STORABLE_thaw {
# my custom thawing routine
}
Given that the dumpers usually offer eval-able code in a consistent manner, I'd presume that a hook implementation would be hard to implement. I haven't seen one and am stating in the presentation that they don't exist, but I'd be happy to be proven wrong.
Thanks,
-xdg
Code written by xdg and posted on PerlMonks is public domain. It is provided as is with no warranties, express or implied, of any kind. Posted code may not have been tested. Use of posted code is at your own risk.
* $Data::Dumper::Freezer or $OBJ->Freezer(NEWVAL)Can be set to a method name, or to an empty string to disable the feature. Data::Dumper will invoke that method via the object before attempting to stringify it. This method can alter the contents of the object (if, for instance, it contains data allocated from C), and even rebless it in a different package. The client is responsible for making sure the specified method can be called via the object, and that the object ends up containing only perl data types after the method has been called. Defaults to an empty string.
Jim Keenan
In the case of Storable, the hook functions are responsible for returning a string that represents the data -- it's a custom serialization routine. It's not so much the order that matters, but for handling data that isn't part of a Perl data structure. That might be C data structures for an XS module, or -- for the purposes of the presentation -- an inside-out object where the reference is just an index and the real data is kept in lexical hashes that Storable doesn't know about.
For Data::Dumper, etc., it's not clear how an inside-out object should be dumped as eval-able code. Object::InsideOut addresses this by providing it's own dump/pump routines that dump and recreate an object. However, that doesn't work well if some programmer throws an ordinary data structure with inside-out objects at Data::Dumper.
use Some::InsideOut::Class;
use Data::Dumper;
my @list;
for ( 1 .. 10 ) {
push @list, Some::InsideOut::Class->new( data => $_ );
}
print Dumper \@list;
The earlier comment about Data::Dumper::Freezer suggests that there is probably a way to convert the inside-out object to a regular Perl data structure to be dumped. A couple problems that I see:
Programmers expect a dump to reflect the underlying code structure and the dump they get would be just a convenient fiction.
Registering the Freezer/Toaster requires loading Data::Dumper -- so regardless of whether a dump is ever called, just supporting Data::Dumper requires the overhead of loading it.
-xdg
Code written by xdg and posted on PerlMonks is public domain. It is provided as is with no warranties, express or implied, of any kind. Posted code may not have been tested. Use of posted code is at your own risk.
Programmers expect a dump to reflect the underlying code structure and the dump they get would be just a convenient fiction.
You cant help that, can you? Going around diddling innards according to a Data::Dumper reading is a bad idea anyway; I dont think you should feel obligated to support this notion. What you want to do is support Data::Dumper-based serialisation; thats another matter altogether.
Registering the Freezer/Toaster requires loading Data::Dumper so regardless of whether a dump is ever called, just supporting Data::Dumper requires the overhead of loading it.
How so? You can always assign a value to $Data::Dumper::Freezer, whether the module is loaded or not.
sub f { warn };
$Data::Dumper::Freezer = 'f';
require Data::Dumper;
Data::Dumper::Dumper( bless \$_, 'main' );
__END__
Warning: something's wrong at - line 1.
The real problem I see is that multiple modules may want to use this hook, and they will probably all have a different idea of what the right value for $Data::Dumper::Freezer is.
You could try to be a little more cooperative by using something like this:
$Data::Dumper::Freezer = "DD_freezer" unless defined $Data::Dumper::Freezer;
*{; do { no strict 'refs'; \*{$Data::Dumper::Freezer} } } = sub {
# serialisation code here
};
But that gives you no guarantee that everyone else will attempt to be similarly cooperative. :-/
Makeshifts last the longest.
The real problem I see is that multiple modules may want to use this hook, and they will probably all have a different idea of what the right value for $Data::Dumper::Freezer is.
Good point. I misunderstood how $Data::Dumper::Freezer worked. I thought you had to do something like this and it registered per package:
package Foo;
use Data::Dumper;
Data::Dumper->Freezer( "__FREEZE" ); # Incorrect!
sub __FREEZE {
# custom class dump here
}
It looks like it's just a global, which is even worse. (Unless you want to use Data::Dumper objects, which can have a per D::D object freezer, but that's doesn't help either.)
As the discussion below suggests, I think Data::Dumper::Streamer comes closest with its FreezeClass handlers.
-xdg
Code written by xdg and posted on PerlMonks is public domain. It is provided as is with no warranties, express or implied, of any kind. Posted code may not have been tested. Use of posted code is at your own risk.
Data::Dumper has limited support for this. It requires a single freeze/thaw method for all objects being dumped. Older versions would insist that the method actually existed for all the objects being dumped. It also (afair) suffers the problem that it leaves the objects in their frozen state after the dump.
Data::Dump::Streamer has more sophisticated support for dumping and thawing objects in various ways. Including support for different freeze/thaw methods per class as well as allowing the object to be able to provide a totally seperate proxy when it is frozen. DDS, does NOT leave the objects in a frozen state when it dumps. I will admit that the documentation of this isnt so hot, but id happy to decrypt my mutterings on the subject as required. (I've never received any feedback on this particular issue tho.)
I like to think that I made Data::Dump::Streamer a lot more flexible in this area than either Storable or Data::Dumper. Which is not to mention all the other positives from using DDS. :-) (Of course it does have the disadvantage that it only has a Perl-XS hybrid implementation, but if compiling isnt a problem then DDS should work fine out of the box.)
Update: For those that can't be bothered to look here is the documentation from DDS on this subject:
Controlling Object Representation (Freeze/Thaw)
This module provides hooks for specially handling objects. Freeze/Thaw for generic handling, and FreezeClass/ThawClass for class specific handling. These hooks work as follows (and it should be understood that Freeze() below refers to both it and FreezeClass as does Thaw() refer to ThawClass() as well.
If a Freeze() hook is specified, then it is called on the object during the Data() phase prior to traversing the object. The freeze hook may perform whatever duties it needs and change its internal structure, _or_ it may alter $_0 providing a substitute reference to be dumped instead (note that this will not alter the data structure being dumped). This reference may even be a totally different type!
If a Thaw() hook is specified then as part of the dump code it will be included to rebless the reference and then call the hook on the newly created object. If the code was originally frozen (not replaced) the method will be called on the object to unfreeze it during the Out() phase of the dump, leaving the structure unmodified after the dump. If the object was replaced by the freeze hook this doesn't occur as it is assumed the data structure has not changed. A special rule applies to Thaw() hooks in that if they include the prefix "->" then they are not executed inline, and as such expected to return the object, but as an independent statement after the object hash been created created, and the return of the statement is ignored. Thus a method that simply changes the internal state of the object but doesn't return an object reference may be used as a Thaw() handler.
For now these options are specified as string values representing the method names. Its possible a later version will extend this to also handle codrefs.
Note that the Freeze/Thaw methods will NOT be executed on objects that don't support those methods. The setting in this case will be silently ignored.
BTW, I presented code to implement a serializable insideout object framework in node 219924. Why i never uploaded it to CPAN O dont remember.
$world=~s/war/peace/g
Great reference about serializing inside-out objects. If I give an extended version of the talk at a conference, I'll probably include that.
For now these options are specified as string values representing the method names. Its possible a later version will extend this to also handle codrefs.
I'm still a little fuzzy on DDS's freeze/thaw handlers. Part of your description reads as if DDS might look for a Freeze() method, but the docs read like one has to register the freezing method name via DDS->Freeze (or rather DDS->FreezeClass) first. Could you give a code example?
So I think that's different from what Storable does looking for a STORABLE_freeze method. Any thought of supporting a DDS_freeze method directly if one exists?
-xdg
Code written by xdg and posted on PerlMonks is public domain. It is provided as is with no warranties, express or implied, of any kind. Posted code may not have been tested. Use of posted code is at your own risk.
Regarding the equivelent to Storable, just use the Freeze()/Thaw() method (as per Data::Dumper) to register the method. Im reluctant to hard code the method names as I dont think it scales well.
use DDS;
sub Popsicle::Freeze {
my ($self)=@_;
$_[0]=bless \do{my $x=join "-",@$self},ref $self;
}
sub Popsicle::Thaw {
my ($self)=@_;
$_[0]=bless [ map {split /-/,$_ } $$self ],ref $self;
}
my $ig=bless ["A".."C"],"Popsicle";
my %h=(One=>1,Two=>2,Three=>$ig);
# "fix" statement thaw...
Dump->Names('first')
->FreezeClass('Popsicle'=>'Freeze')
->ThawClass('Popsicle'=>'Thaw')
->Data( \%h )->Out;
print "\n";
# inline thaw....
Dump->Names('second')
->FreezeClass('Popsicle'=>'Freeze')
->ThawClass('Popsicle'=>'->Thaw')
->Data( \%h )->Out;
print "\n";
# clear the hooks for Popsicle
Dump->Names('third')
->FreezeClass('Popsicle'=>'')
->ThawClass('Popsicle'=>'')
->Data( \%h )->Out;
print "\n";
# Using FreezeThaw to make it a bit easier...
Dump->Names('fourth')
->FreezeThaw('Popsicle'=>'Freeze','->Thaw')
->Data( \%h )->Out;
print "\n";
# Using generic hooks and not class specific ones
Dump->Names('fifth')
->Freeze('Freeze')
->Thaw('->Thaw')
->Data( \%h )->Out;
print "\n";
__END__
$first = {
One => 1,
Three => bless( \do { my $v = 'A-B-C' }, 'Popsicle' ),
Two => 2
};
$first->{Three}->Thaw();
$second = {
One => 1,
Three => bless( \do { my $v = 'A-B-C' }, 'Popsicle' )->Thaw(),
Two => 2
};
$third = {
One => 1,
Three => bless( [
'A',
'B',
'C'
], 'Popsicle' ),
Two => 2
};
$fourth = {
One => 1,
Three => bless( \do { my $v = 'A-B-C' }, 'Popsicle' )->Thaw(),
Two => 2
};
$fifth = {
One => 1,
Three => bless( \do { my $v = 'A-B-C' }, 'Popsicle' )->Thaw(),
Two => 2
};
$world=~s/war/peace/g
Im reluctant to hard code the method names as I dont think it scales well.
I'd like to better understand your reasoning. One advantage of the Storable approach is that it's defines an interface that modules can implement in order to manage their interoperability with Storable. E.g. Object::InsideOut and Class::Std::Storable.
-xdg
Code written by xdg and posted on PerlMonks is public domain. It is provided as is with no warranties, express or implied, of any kind. Posted code may not have been tested. Use of posted code is at your own risk.
I'm glad you challenged me on that. To be clear: I'm not convinced im right in my opinion. Basically my concern is that using such an approach means that you end up with a proliferation of freeze/thaw mechanisms. IE, if somebody wants their class to work with Storable they have one set of method names, if they want their class to work with DDS they would need another set. And if the concept was extended you'd need the same for Data::Dumper and for YAML.
But maybe this is actually not such a big deal. There arent that many production worthy serialization tools out there, so may the proliferation is managable from the start.
Alternatively i wonder if maybe DDS should just define a hash so that modules can provide system wide defaults. IE, Class::InsideOut might include a line like:
$Data::Dump::Streamer::FreezeThaw{__PACKAGE__}=['foo','->bar'];
If YAML and Data::Dumper used the same approach then you might get code like:
$Data::Dump::Streamer::FreezeThaw{__PACKAGE__}=['foo','->bar'];
$Data::Dumper::FreezeThaw{__PACKAGE__}=['foo','bar'];
$YAML::FreezeThaw{__PACKAGE__}=['foo','bar'];
What do you think? I'm very open to ideas about this. And willing to be convinced that another approach is better.
$world=~s/war/peace/g
I like it. Among other things, it might allow one serializer to fall back to another. E.g. if YAML doesn't find a freezer defined for it, it could theoretically fall back to DDS.
The only other way that I think might make sense is for a package to have the hash, e.g.
our %FREEZE_THAW_FOR;
%FREEZE_THAW_FOR{ "Data::Dump::Streamer" } = [ 'foo', '->bar' ];
%FREEZE_THAW_FOR{ "Data::Dumper" } = [ 'foo', 'bar' ];
%FREEZE_THAW_FOR{ "YAML" } = [ 'foo', 'bar' ];
It all amounts to the same thing really -- mapping between packages and serializers in some package variable space. I think I moderately prefer your approach, though a part of me is leery of the globals either way. Seems like there ought to be a better way -- maybe a third party module that packages can register with and serializers can query, but then you wind up in dependency hell.
I think the KISS and YAGNI principles would suggest doing it your way.
-xdg
Code written by xdg and posted on PerlMonks is [http://creativecommons.org/licenses/publicdomain|public domain]. It is provided as is with no warranties, express or implied, of any kind. Posted code may not have been tested. Use of posted code is at your own risk.
I think I moderately prefer your approach, though a part of me is leery of the globals either way.
I agree. In general I'm leery of the globals too.* I would like to know why you moderately prefer my approach as your reversal hadn't occured to me at all and I'm wondering what tilts you in the direction of my proposal.
* Globals have the advantage that you can use local on them to affect dynamic changes. For instance to resolve one of your issues from another part of this thread you could create a wrapper sub that localizes $Data::Dumper::Freeze and $Data::Dumper::Thaw before calling into Data::Dumper. Using a pure object approach (like DDS) means that you have to have seperate serialization objects, which in turn makes it difficult to "override" a default set of properties for a limited duration. Anyway, I apologise for the digression, I just thought I should use this chance to explain why I think that Data::Dumper's dynamic var approach actually makes sense. (Even though at first glance it doesnt :-)
$world=~s/war/peace/g
I'm wondering what tilts you in the direction of my proposal.
Off the cuff:
Not having to worry about strict or use vars or our and backwards compatibility. Your approach just explicitly forces fully qualified names.
External fully-qualified names makes it clear where the mappings are used. I think that will help those maintaining the code follow the action.
Doesn't pollute the package namespace with a special variable the package doesn't even use itself.
(Regarding globals and local: Yes, indeed. Have you seen Object::LocalVars?)
-xdg
Code written by xdg and posted on PerlMonks is public domain. It is provided as is with no warranties, express or implied, of any kind. Posted code may not have been tested. Use of posted code is at your own risk.
Warning this node contains a u-turn of opinion that would impress even a well salted politician...
I think I finally clicked to why Storable uses the approach it does and why despite the other points raised I think it actually makes more sense. In two words:
Respect Inheritance!
The point is that if you define a Storable_freeze() method all Storable has to do is say $obj->can('Storable_freeze') (in a heavy XS accent of course ;-) and it knows if the object needs special handling. With my approach I have to do a hash lookup on the classname, so subclasses won't inherit their freeze/thaw handlers.
Which brings me to the point of starting to think that my approach is actually subpar and that it makes a lot more sense to copy Storable's approach. Which leaves me in the uncomfortable situation of wanting to support both. Sigh.
I think that given a classname register would break inheritance I'm going to have to provide DDS_freeze and DDS_thaw support. I guess supporting both approaches has its merits as they are to a certain extent complementary. IE, you could use the existing mechansim to override the DDS_ method hooks if you wanted to.
Anyway, live and learn I guess.
$world=~s/war/peace/g