[JOB] The Perl Foundation seeks Windows Developer
adamk
created: 2006-04-01 07:13:04
perlmonks
prlmnks
I couldn't resist the title given the date, but this isn't actually a joke :)

Yesterday Ovid put out a call for grant applications, and ideas for things that could be sponsored.

I wrote in mentioning a project that needs to be done, but which I don't have the time or qualifications to handle. The grant committee is interested, and so I've been asked to put out a call for anyone interested in spending a month or so on TPF's dime.

In response to my CPAN-capable Windows Installer challenge in January, Carl Franks responded with Vanilla Perl, which is an installer containing MinGW, dmake, and core Perl compiled with them.

Vanilla could be classed as a research distribution. It allows people working on the Perl toolchain itself to work on Win32 compatibility, it provides a way to examine how the Perl code interacts with Win32.

But it isn't really useful to anyone doing development, and is still loaded with bugs, although both David Golden and myself have been working in bits and pieces to slowly squash the bugs in it.

What would be much more useful would be a different flavour of Perl, a distribution tentatively code-named Strawberry Perl.

The purpose of Strawberry Perl is to expand on Vanilla, bundling in all of Bundle::CPAN, as well as bundling some strategic upgrades of some dual CPAN/core modules that have had Win32-related upgrades since 5.8.8 was released.

The aim is to provide CPAN authors a Win32 Perl platform that is similar in use and style to the Unix Perls, to make it easier for them to port to Win32, and to provide a suitable basic Perl platform for experienced Perl developers that prefer to install from CPAN, instead of using binary packages.

Because nobody currently working on Vanilla can contribute very much time, the grant would involve the TPF paying someone to take responsibility for the creation of delivery of the first production release of Strawberry Perl.

This means tracking and keeping an eye on ALL of the bugs that are preventing the release, hunting down and diagnosing bugs, creating patches to fix them when the module's author isn't very Win32 savvy or doesn't have time to fix them, taking co-maint bits to release new versions of modules where the author is awol or doesn't have time to do releases, and generally dealing with any bits and pieces holding up Strawberry Perl.

This is the direction Vanilla is moving anyway, but currently it's moving in dribs and drabs, and it would be great to have someone push it through to completion.

Interested parties should have experience with both Win32 and Perl, and given the nature of the task, will need to play well with others.

So how about it, want to be funded to work on a Perl distribution for a while?

Note this is a grant, not a job, and so the rules of the grant process still apply. But the grants committee would look highly favourably on anyone interested in this project.
Re: [JOB] The Perl Foundation seeks Windows Developer
xdg
created: 2006-04-01 08:20:38
perlmonks
prlmnks

Let me add that the Win32 experience needed includes both Perl and the Windows API/C. Some of the bigger challenges may be getting unix-y programs to compile or unravelling problems in distributions that may build OK with Visual C++ and nmake, but break with MinGW and dmake (the tools in VanillaPerl). E.g. Win32::API is in the latter category.

-xdg

Code written by xdg and posted on PerlMonks is public domain. It is provided as is with no warranties, express or implied, of any kind. Posted code may not have been tested. Use of posted code is at your own risk.

Re^2: [JOB] The Perl Foundation seeks Windows Developer
adamk
created: 2006-04-02 07:07:42
perlmonks
prlmnks
I'll note that you aren't expected to do 100% of everything yourself.

There are certainly other people around doing bits and pieces.

A great deal of the work is just locating and diagnosing the problem, and reporting it to the author so they can do the quick fix.

But what we lack most of all is someone to really manage the process. Keep an eye on _all_ the bugs and all the authors, and step in wherever there are gaps to get everything sorted out.

So while you may well need to do a lot of different things, hopefully you won't have to do it along.
Re: [JOB] The Perl Foundation seeks Windows Developer
Steve_p
created: 2006-04-01 08:26:18
perlmonks
prlmnks

An important dependency of this project is the ability to compile Perl and XS-based modules with Visual C++ 8.0, a.k.a. Visual Studio 2005 or the .NET 2.0 SDK C++ compiler. Change made in how C executables and DLLs are compiled with .NET 2.0 have caused significatant problems in building a Perl with this compiler. As of now, Perl cannot successfully built with this compiler.

Re^2: [JOB] The Perl Foundation seeks Windows Developer
jimbojones
created: 2006-04-01 20:12:13
perlmonks
prlmnks
Hi

could you provide more details on this? I thought the Vanilla/Strawberry Perls were bundled with a gcc compiler. Is this a requirement of Strawberry perl or of the grant?

thanks, j

Re^3: [JOB] The Perl Foundation seeks Windows Developer
adamk
created: 2006-04-01 22:27:48
perlmonks
prlmnks
No, it is not part of the grant to do any additional work to adapt to changes introduced by commercial compilers that may have broken existing code.

Vanilla/Strawberry are built with gcc, and while some people may feel it is important to do the work needed to handle changes in commercial compilers, I don't personally feel it is the responsibility of the TPF to be funding compatability with non-open-source compilers, particularly where the blame for the code not working lies with those commercial compilers.

So while obviously you shouldn't be introducing any NEW bugs that might break existing working scenarios, fixing problems caused by commercial compiler vendors will not be part of the grant, or at least part of THIS grant. The grant committee may feel it is worth seperately funding this work, but I'd prefer to deal with seperate problems seperately.

In addition, and as an aside, Windows Vista is incompatible with nmake, and there is no obvious upgrade path for name, seeing as we have been using a very very old legacy build of nmake this entire time.

So from Vista onwards, I would expect that all Perl distributions will need to transition away from nmake to something like dmake.
Re^4: [JOB] The Perl Foundation seeks Windows Developer
Steve_p
created: 2006-04-02 00:14:03
perlmonks
prlmnks

Work has already been done by Gisle Aas of ActiveState to allow XS modules to be compiled with either the MinGW gcc or Visual C++, regardless of the compiler that Win32 Perl was built with (see Perl change #27555 for details). I'm pretty sure that Perl has been able to be built with either dmake or nmake with Visual C++ for some time now. It would be nice, however, and I'm sure you would agree, to allow users their choice of compilers on whatever operating system they choose. I would hate to see this project be a step backwards in Perl support on Win32.

With Windows Vista, all indications are that support for the free nmake will go away. The nmakes provided with recent Visual Studio's or .NET SDKs will still work. In exchange for losing free nmake support, however, we will have a usable and programmable shell that may be able to be used to provide a sane Configure script. So, I see Vista as a great new opportunity and challenge for porting Perl rather than an obstacle.

As far as working with commercial compilers, Perl is everywhere and should be able to be compiled with any sane C compiler, commercial or not, just as it is supported on almost every free and commercial operating systems. Whether the C compiler is commercial or not should be irrelevant. Support for Visual C++ 8.0 will be provided. Its simply a matter of time. I consider it to be a major bug that it isn't supported at this time. This is why I see this support as important for this project.

Re^5: [JOB] The Perl Foundation seeks Windows Developer
adamk
created: 2006-04-02 05:45:11
perlmonks
prlmnks
Let me restate my previous point.

I'm NOT saying that it isn't a good thing to support more compilers. Playing well with others is a GOOD thing.

And most of this grant is not about adding MinGW compatibility, because almost everything works already anyway.

This grant is firstly about fixing things in Bundle::CPAN that don't work, or have major bugs, on any Win32 platform, Vanilla or Strawberry or ActivePerl or otherwise.

Because there's a lot of bugs for things as simple as Term::ReadLine::Perl, or that to relate to the fact CPAN.pm doesn't play well with the Windows firewall. All the normal portability and platform-specific stuff.
And the second part is fixes to deal with other secondary problems that have emerged because the bulk of Windows users currently don't need to use CPAN, and a few bug fixes to remove assumptions caused by CPAN authors treating ActivePerl bundled modules as if they are in the core and thus not listing those dependencies in their (Makefile|Build).PL

That's what THIS grant is about. IT's about making all the modules in the Perl toolchain intrinsically sane on Windows.

This doesn't invalidate the importance of adding support for newer and different compilers. But it is a mostly orthogonal task, and so it would be the subject of a different grant.

And in the process, this grant creates an option for people to use a Perl distribution that is both simple to install, without requiring a knowledge of C, and entirely libre from the compiler on up.

Because I for one don't write C, can't stand Visual Studio, and just want my Perl modules to install, without having to care about C compilers.
Re^6: [JOB] The Perl Foundation seeks Windows Developer
demerphq
created: 2006-04-02 16:18:25
perlmonks
prlmnks

and a few bug fixes to remove assumptions caused by CPAN authors treating ActivePerl bundled modules as if they are in the core and thus not listing those dependencies in their (Makefile|Build).PL

IMO the much better solution is to just put those modules in Core in the first place.

---
$world=~s/war/peace/g

Re^7: [JOB] The Perl Foundation seeks Windows Developer
adamk
created: 2006-04-03 00:19:05
perlmonks
prlmnks
Even if certain Windows-specific modules were added to the core _today_, these fixes would still need to be done to deal with older installed Perl versions.

Whether or not something should be in the core is an orthogonal problem to this grant, but I'll address it in my reply to one of your other comments, below.
Re^8: [JOB] The Perl Foundation seeks Windows Developer
demerphq
created: 2006-04-03 02:42:33
perlmonks
prlmnks

Even if certain Windows-specific modules were added to the core _today_, these fixes would still need to be done to deal with older installed Perl versions.

Ive said it a lot of times Adam, a lot of these fixes can't be done without proper access to the Win32 API. And as long as that API is reachable only through a non core module set the issues involved with adding better support become much more difficult. You can't just say "am I on Win32? Yes ok, better use the Win32 tools" you have to say "am I on Win32 AND do i have the required modules available". Which complicates things and means that less Win32 folk are going to bother to try to add proper Win32 support to core modules.

Not only that, but look at the mantra of P5P, it basically says "fix what you like, but you can't use non-core modules to do it". Which means for instance that functionality in EU-I simply cant be used or tested under a core build. (Although what I find really amusing is that you can added Win32 API support to core modules, but only via XS, for some reason providing that access through a perl layer is verbotem. So we have Win32API calls for specialized purposes scattered throughout the core, but always hidden so that end users can't make use of them in interesting new ways. Which is bad because it raises the skills requirement for improving Win32 perl, not only do you need to be conversant with the API, you need to be a good perl programmer, and a good C/XS programmer.)

As an example using core perl there is only one sharing mode that you can open a file with, using Win32API::File there are a variety, including the mode that Windows itself uses to lock DLL's but still allow them to be renamed. (Annoyingly the default open mode for Cygwin is different from the default open mode of native builds. This is why you can delete files opened with Cygwin perl, but not AS perl)

Similarly since Win32API::File is omitted from Core it is unsurprising to me that File::Temp uses broken *NIX based logic to do its thing on Win32. Wheras if Win32API::File were included in core then the author of F::T (apparently not a win32 developer) might have looked at the documentation for Win32API::File and found out that Windows supports an OS level temporary file which will be autodeleted once all filehandles to it are closed. Or with even better API support he might have found the API call that windows provides to produce a guaranteed unique temporary file.

But since this stuff isn't available to Core, and since non Win32 developers seem to have this strange aversion to using MSDN library search, (maybe because they know that since there is no support for such in core there isnt any point in searching) the fact is that non-win32 programmers will never even know that there are better ways to do things on Win32.

So I stick to my guns: adding proper win32 API support to core is not a nice to have, its an essential part of improving Perl support on Win32. And doing it in blead even will be a signal to the community that the investment in making the code work better on Win32 will not be a wasted one.

And as a closing thought consider this: In VB you can bind to a Win32 API call with one line of code. In Perl you cant bind to API at all out of the box. That VB can provide better OS access than Perl does is just wrong.

---
$world=~s/war/peace/g

Re^9: [JOB] The Perl Foundation seeks Windows Developer
BrowserUk
created: 2006-04-03 11:42:35
perlmonks
prlmnks

You've hit the nail on the head.

The biggest problem (for the none *nix developer), that I've seen in my forays into Perl sources, is that (understandably given it's origins), most of the Perl sources are written directly to the architecture and semantics of *nix system calls and methods. Where the underlying OS does not directly support those architectures and semantics, it is up to each developer on each non-*nix OS to add conditional code in-situ, at the site of each use, to emulate the *nix behaviours.

It is the combined effect of the conditional codes for all the non-*nix OSs, (along with variations of minor tweaks and fixes for the myriad not-quite-*nix-compatible *nix-like platforms), that is responsible for a large part of the complexity and fragility of the sources.

It is made even worse by the need to apply the same conditional fix/work-around in-situ at each place in the sources where it is used. This just exacerbates the potential for the multiple implementations of the same fix to get out of synch. It's the cardinal sin of c&p code reuse. Your example of file open & sharing modes above relects the problem exactly.

It gets worse where a pair (or more), of related but otherwise atomic posix system calls, which get used directly in-line in the Perl sources, require a combination of 2 or 3 Win32 (or other platform) system APIs to achieve the same function. And it gets even worse when there is a need to convey some information (like OS handles) between the open and close sequences of a bracketing pair of calls. If the posix pair do not require the passing of a handle, then there is no provision in the architecture for this.

The architectural solution to this problem, for all platforms, not just Win32, would be to isolate OS dependant calls from the perl sources. Essentially, this would require building a Perl virtual OS layer and removing all system calls from the core sources in favour of calls to the virtual OS layer. The Perl sources would call the virtual OS functions and the platform dependant equivalents are conditionally compiled in to perform that function. In many places this has already been done--PerlIO is a good example.

The problem is that in a lot of cases, the virtualisation is done at too low a level. They require a one to one correspondance between the virtualisation and the underlying POSIX apis. When the OS is unable to easily provide that one to one correspondance, the need to cut&paste platform dependant fixes all over the source tree returns.

Of course, it is entirely the wrong time, too late, to attempt to address this properly in P5. Unfortunately, unless a cross-platform white knight has become involved in Parrot recently, that was headed in exactly the same direction.

However, I think there is a solution (for win32), that could be retro-fitted to the P5 sources without creating wholesale mayhem. It would require considerable effort; at least 3 or 4 commited, knowledgable, Win32 developers--though not necessarily all full time; and the involvement and support of 1 or 2 of the deeply knowledgable p5p source gurus. Essentially, there are already a bunch of Win32 OS specific virtual functions conditionally build into and exported from perl5x.dll. All the stuff that resides in win32\Win32.c.

The project would involve removing as much of the Win32 conditional code from the core of the sources and replacing it with calls to new virtualised functions added into Win32.c (or probably better Win32v.c or similar). This would benefit all p5 programmers. Core *nix programmers would have considerably less Win32 conditional code to work around. Win32 p5p developers (like yourself & Steve) could concentrate your efforts in one place more easily. And cross-platform developers and module writers would gain access to the Win32 equivalents of *nix functions exported from the Win32\* sources rather than needing to invent their own workarounds or cut&paste existing code.

Like you, I absolutely applaud and encourage any moves/projects that will enable Perl on Win32 to

  1. Be less of a problem for non-Win32 authors to write modules that will easily port to Win32.
  2. Be less of a source of bug-reports and grief to the p5p efforts in general.
  3. Give me greater access to the facilities and semantics of my chosen platform.

My fear is that, as currently envisaged and described, the Vanilla/Strawberry projects, and this particular effort are not likely to achieve any of these.

I feel that the situation is analogous to General Motors designing and proposing a solution to the current fuel crisis and expecting Toyota (and Honda and Ford and BMW) to implement their solution without consultation or input.

The current proposal seems to revolve around allowing more modules written originally for non-Win32 platforms to be seen as functional on Win32, as measured by some statistic (currently the AS autobuild process). However, I think that as currently conceived, the project will achieved very little in the long term. Perhaps the problems that prevents the AS autobuild process from passing more modules is that it is automated. And as the endemic solution to compatibility problems is to hand-code in-situ, conditional work arounds to each problem as it arises, the automated approach cannot do that.

Whilst a short, concentrated effort now may result in a bunch of immediate fixes to individual incompatibilities, without an architected solution that allows those fixes to be virtualised and embedded into the core, the efforts would only allow a short term hiatus in the problems because new authors and new modules would be forced back into writing individual fixes to known problems with known solutions as soon as the manual 'bug hunt and fix' effort subsided.

Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
Lingua non convalesco, consenesco et abolesco. -- Rule 1 has a caveat! -- Who broke the cabal?
"Science is about questioning the status quo. Questioning authority".
In the absence of evidence, opinion is indistinguishable from prejudice.
Re^10: [JOB] The Perl Foundation seeks Windows Developer
adamk
created: 2006-04-06 16:10:15
perlmonks
prlmnks
Unfortunately for this idea though, you have to find a way to make it happen.

The reason the grant committee will probably end up getting behind this, is that it will work, today.

You are proposing something without a clear path to the end point. The Strawberry Perl grant is saying "Here's something I want to do, here's how it can be done. I could do it ANYWAY, but we could use some resources to make it happen faster".

Allowing more modules written originally for non-Win32 platforms, or for non-specific platforms to work on Win32 is a good thing. It can be done now, and doesn't require a multi-year, multi-developer effort to entirely re-architect the way Perl works.

Which is why it is happening.
Re^11: [JOB] The Perl Foundation seeks Windows Developer
BrowserUk
created: 2006-04-06 17:17:24
perlmonks
prlmnks

I concur. I wasn't suggesting this as an alternative to your project, just that I believe that the problems that stop many of the modules from building on Win32, run much deeper than either the compiler that is used to build them, or artificially false statistics generated by a broken, down-level, automated build process. I felt that the aims of the project as described are fuzzy and incomplete, and I requested clarification of those aims.

The bottom line from my perspective is that if any modules that are successfully transferred to the Win32 platform as a result of your project are binary compatible with AS builds, which a couple of people have said they will be, then everyone is a winner.

I still have my reservations as to how many of the incompatibilities your project will resolve. I also have doubts as to the efficacy of hacking solutions to each module individuallym rather than attempting to package generic fixes to the underlying problems and feed them back into the core so that future modules pick them up automatically from there. As I said in my first post in this thread, maybe Strawberry is the right way to start, and maybe pointers to how to achieve generic fixes will come from it.

Either way, I wish you the best of luck in your endeavours, and I am sorry that my attempt to ellicit clarifications have lead to misunderstandings of the 'us & them' variety.

Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
Lingua non convalesco, consenesco et abolesco. -- Rule 1 has a caveat! -- Who broke the cabal?
"Science is about questioning the status quo. Questioning authority".
In the absence of evidence, opinion is indistinguishable from prejudice.
Re: [JOB] The Perl Foundation seeks Windows Developer
BrowserUk
created: 2006-04-02 12:33:15
perlmonks
prlmnks

I keep reading this, and I'm still lost for an explanation of what the Vanilla and Strawberry projects are aiming to achieve?

As a Win32 Perl user,

  1. I can already build my own perl

    I actively sic choose to use Active State Perl for various reasons.

    • They've already solved most of the problems.
    • They've added value in the form of all the must have modules they bundle with their distributions.
    • They've added value in the form of their html-ised documentation sets which are 1000% more usable that perldoc.
    • They've added value in the form of the PPM manager.

      Whilst I can build most modules myself, having a quick, simple and convenient 'just do it' install for the more complex or tricky modules is a huge time saver.

      Why expend time and energy re-discovering all the problems and solutions to installing packages on win32 for every installation, when the problems have already been solved and are available for one-step installation?

  2. I can already build my own modules, with or without the use of CPAN.pm

    I can use the CPAN shell. I choose not to, because it carries with it so much baggage that simply does nothing for me.

    For the vast majority of modules, I can manually download, build, test and install them--including 1 or 2 small dependencies before the CPAN shell has finished configuring itself, which it seems to need to do every time?

On those occasions I fail to be able to build a module, (most recently CPAN://Devel::FastProf), the root cause seems to be much more to do with the way the authors write their modules to make use of inherently *nix-only features of the Perl source code, or by directly accessing parts of the Perl runtime bypassing the good work done the developers to isolate OS dependencies by virtualising them.

I guess what I am saying is, I certainly have the time, and (some of), the skills that would allow me to contribute to this project, but I am at a loss to understand what it is that you envisage that I, and other win32 users, would gain from this effort?

Also, what does the overall Perl community gain from the TPF funding this project?

The bottom line is, you are asking for someone to apply for a TPF grant, and therefore raise a grant application, but it's not at all clear to me what the objectives of the project are? Who is it intended to help and how?

I also think that a much clearer brief is required. In this short thread we've already seen that there is more than one opinion as to the direction that should be pursued. Your brief justification of the motivation for Vanilla & Strawberry

The aim is to provide CPAN authors a Win32 Perl platform that is similar in use and style to the Unix Perls, to make it easier for them to port to Win32, and to provide a suitable basic Perl platform for experienced Perl developers that prefer to install from CPAN, instead of using binary packages

Leaves me uneasy that it would benefit Win32 Perl users much. Despite the almost obligatory "I don't/only/have to use Win32" disclaimer, the biggest problem facing Win32 Perl users is that the vast majority of Perl developers don't want to develop on Win32, and this project sounds like yet another attempt to make win32 look as much like *nix as possible to allow reluctant but willing CPAN authors to more easily test and bug fix their modules on the win32 platform. Maybe that is the right way to go, but based on my own limited experience of trying to build the 'awkward squad' of modules on Win32, the problems go deeper than the compiler, CRT or build utilities.

As an example of what I mean, the problem I encountered with Devel::FastProf was that although the MSVC CRT has perfectly normal C getc() routine, once you've bypassed various errors to do with simple unixisms, the base Perl headers files are converting the getc() used in the source, to a call to 

FastProf.xs(97) : warning C4013: 'PerlSIO_getc' undefined; assuming extern returning int

And the PerlIO/SIO subsystem doesn't appear to provide this routine. I've tried to unwind the macros to work out why getc() is redefined as PerlSIO_getc(), when that routine does not appear to exist anywhere in the entire Perl source tree, not just the Win32 part, without success.

I imagine that any *nix programmer that encountered this problem when trying to port their module to Win32 would probably take one look, blame it on a broken MS C compiler/CRT and understandably give up, but the problem lies within the the perl sources. Does anyone anywhere use the whole PerlSIO subsystem? What was the purpose of that layer of headers files? And was it ever achieved/completed?

And this is not an isolated incident. Many of the other problems of compiling code written to run on GCC with the MS compiler come down to compiler settings. Things like declaring for loop variables inline, which gcc allows even though it isn't part of the C standard, have to be explicitly enabled using the MS compilers.

My point is, I am not at all sure that simply moving to a perl distribution that builds with gcc is going to fix very much, or help Win32-reluctant module developers to more easily port their modules. And unless Strawberry also included all, (or at least most), of the value added features that come with the AS distributions, then Win32 Perl users, (as opposed to developers), are going to be very reluctant to adopt it. It would be of little benefit to anyone if the only people that made use of Vanilla/Strawberry Perl were only developers looking to simplify the porting of their modules to Win32, because it is quite likely that the results of their efforts would not work for the majority of Win32 users.

Please don't misunderstand me, I would hate to be seen as dismissive of any attempt to make Perl more Win32 friendly. Indeed, I would love to be a part of such an effort and dog knows if there was a little income to be earnt from such efforts it certainly would go amiss. My fear is that as I understand the direction outlined here, (which of course my be a completely different thing from the intent), may not, even probably would not, do much to help in this regard, and that any monies that the TPF feels are justified in making Perl more Win32 friendly/complete might be better directed.

Maybe Strawberry is only the starting point and would enable/encourage more people to become actively involved in making Perl work better on Win32? I think I am the wrong person to make that judgement and so I would encourage an expanded discussion of the overall goals and the best way of achieving them, to help convince me, and others, that this is the right way to go.

Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
Lingua non convalesco, consenesco et abolesco. -- Rule 1 has a caveat! -- Who broke the cabal?
"Science is about questioning the status quo. Questioning authority".
In the absence of evidence, opinion is indistinguishable from prejudice.
Re^2: [JOB] The Perl Foundation seeks Windows Developer
rhesa
created: 2006-04-02 14:41:53
perlmonks
prlmnks
adamk wrote:
The aim is to provide CPAN authors a Win32 Perl platform that is similar in use and style to the Unix Perls, to make it easier for them to port to Win32, and to provide a suitable basic Perl platform for experienced Perl developers that prefer to install from CPAN, instead of using binary packages.

I didn't read that as aiming for a replacement of ActivePerl. Having a sensible toolchain on Win32 would remove much of the perceived reluctance to develop on that platform, because it would be much easier to start working _right away_, without having to dig around for compilers, header files, and what not.

I'm also guessing this would benefit the AS distribution in the long run, since it would mean more modules will build on Win32, which means AS will be able to package those too.

You wrote:

I imagine that any *nix programmer that encountered this problem when trying to port their module to Win32 would probably take one look, blame it on a broken MS C compiler/CRT and understandably give up, but the problem lies within the the perl sources.

That might very well be true right now. If this project succeeds, that argument would no longer be true, since we'd be using the same tool chain on both platforms. The grant also asks for fixing bugs like these.

You wrote:

Maybe Strawberry is only the starting point and would enable/encourage more people to become actively involved in making Perl work better on Win32?

I believe that's the goal, yes, absolutely.

Even though I said goodbye to Win32 long ago (a personal and professional luxury, I know), I do see this as a valuable plan: it would improve perl for a very large user base, integrating their development practices with those of unixy platforms; making more modules available to them means an expanded user base, which means more eyeballs, which implies improved code. I feel that's a win-win situation for both camps. Having used that word, I even have hope we won't be speaking of "both camps" in the future.

Re^3: [JOB] The Perl Foundation seeks Windows Developer
BrowserUk
created: 2006-04-02 15:22:30
perlmonks
prlmnks
I'm also guessing this would benefit the AS distribution in the long run, since it would mean more modules will build on Win32, which means AS will be able to package those too.

Um. The problem is, will it?

Are the DLLs produced by compiling XS code with MinGW compatible with a Perl built with MS VC++? If not, how will allowing module authors to "get away" with making minimal changes to their modules in order to achieve successful tests under Vanilla/Strawberry enable those same packages to pass the AS autobuild process?

Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
Lingua non convalesco, consenesco et abolesco. -- Rule 1 has a caveat! -- Who broke the cabal?
"Science is about questioning the status quo. Questioning authority".
In the absence of evidence, opinion is indistinguishable from prejudice.
Re^4: [JOB] The Perl Foundation seeks Windows Developer
rhesa
created: 2006-04-02 15:51:18
perlmonks
prlmnks
Aren't you implying that the AS distribution is the reference to which all Perl module authors should conform? Why wouldn't it be the other way around?

I don't see why AS couldn't switch to gcc for their builds (theoretically speaking). At the very least, Strawberry will give _them_ something to build and test against.

In any case, Strawberry would give module authors a tool to test these things for themselves. The barriers to that are currently too high, in my opinion. Whether they will make use of that options is another question, but it seems obvious that they won't _without_ these tools.

Re^5: [JOB] The Perl Foundation seeks Windows Developer
BrowserUk
created: 2006-04-02 16:33:57
perlmonks
prlmnks
Aren't you implying that the AS distribution is the reference to which all Perl module authors should conform?

No. I'm saying that for the Win32 Perl user, AS is the defacto standard that makes Perl usable on Win32. It maybe that non-win32 Perl users see POSIX as the ultimate expression of cross systems compatibility, but it just misses the mark in so many ways for win32 users and programmers, that moving 2 steps closer to POSIX compatibilty and 15 steps further away from being able to utilise the features of our OS is no progress at all.

Presumably, the purpose of the exercise is to make Perl more usable on Win32? If so, I do not think that you achieve that by throwing away much of what AS perl already provides in order to gain what? I can already build most of CPAN on Win32 with existing tools and most of what I cannot build will not be fixed by throwing away half of what I already have.

Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
Lingua non convalesco, consenesco et abolesco. -- Rule 1 has a caveat! -- Who broke the cabal?
"Science is about questioning the status quo. Questioning authority".
In the absence of evidence, opinion is indistinguishable from prejudice.
Re^4: [JOB] The Perl Foundation seeks Windows Developer
demerphq
created: 2006-04-02 16:13:45
perlmonks
prlmnks

Are the DLLs produced by compiling XS code with MinGW compatible with a Perl built with MS VC++?

Assuming that the VC++ is VC6 then yes, if its VC7 then yes, but not if you use C++, if its VC8 then no.

The rule is that it doesnt matter what compiler was used, it matters what CRT is in use. Since stuff built with both VC6 and MINGW link against MSVCRT60.dll they are able to coexist peacefully. Similarly MSVCRT60 and MSVCRT70 are compatible in terms of C linking, but C++ exceptions will cause a problem. Luckily the use of C++ is rare in Perl.

Also as of later AS Perl builds you can compile using either MINGW or VC. If VC isnt available and MINGW is then EUMM will automatically use the latter.

---
$world=~s/war/peace/g

Re^4: [JOB] The Perl Foundation seeks Windows Developer
syphilis
created: 2006-04-02 22:11:21
perlmonks
prlmnks
Are the DLLs produced by compiling XS code with MinGW compatible with a Perl built with MS VC++?

Yes they are - and vice versa, too. I've often mixed things in this way (in both directions). Exceptions may exist - such as Inline::C and Inline::CPP, where you've built an extension that expects you to be using gcc (or cl) and run it with a perl that expects you to be using cl (or gcc). If the VC++ compiler in question is other than version 6.0 then you might strike an additional (very rare) problem because of different runtimes being used.

(I also wondered what Strawberry will achieve. Having read this thread, I can't see that it has anything to offer me ... but that's about all I've been able to deduce ...)

Cheers,
Rob
Re^4: [JOB] The Perl Foundation seeks Windows Developer
adamk
created: 2006-04-03 00:28:01
perlmonks
prlmnks
Yes, it will absolutely benefit AS in the long fun, and in fact in the short run too.

Because Vanilla has allowed the feedback loop to be shortened, we've been able to fix more than a dozen bugs in various areas that afflict ActivePerl as well, in places like CPAN.pm, Term::ReadLine, Win32API::Registry and others.

There are not bugs in C code related, they are bugs in Perl code. In fact, in some cases, I cannot see how the module could possibly build AT ALL so that it could be bundled into AP.

The VAST majority of bugs encountered so far have nothing to do with compilers at all. They are ordinary bugs in ordinary code. But perhaps because they are bundled, or perhaps because the feedback loop is at 6 month intervals, they aren't getting fixed.
Re^4: [JOB] The Perl Foundation seeks Windows Developer
adamk
created: 2006-04-03 01:35:47
perlmonks
prlmnks
This is not about DLLs. The problems with compiler compatibility has been largely fixed.

95% of the problems are in non-C areas.
Re^2: [JOB] The Perl Foundation seeks Windows Developer
xdg
created: 2006-04-02 14:59:55
perlmonks
prlmnks
Also, what does the overall Perl community gain from the TPF funding this project?

In short: diversity and choice.

I think the Perl community gains by opening up Perl usage on Windows to a wider audience. Until now, Perl on Win32 has really meant two choices: Cygwin or ActiveState, both of which bring a lot of baggage along with their benefits.

One of the downsides of relying on, for example, ActiveState, is that while they solve some problems for you, you wind up held hostage to choices that they make. One of the goals of CamelPack and VanillaPerl, etc. is to make having your own Perl distribution on Win32 as easy as it is on unix-based platforms -- with direct reliance on CPAN rather than intermediate repositories to allow for access to the most up to date modules available.

Here's a real-life example: Scalar::Util. ActiveState's PPM repository targets the first ActiveState release for a particular major version of Perl to maximize backwards compatibility and (presumably) minimize support costs. Because the version of Scalar::Util frozen into the first ActiveState Perl 8xx release didn't include the critical refaddr function, every CPAN module that requires a Scalar::Util with refaddr fails the automated build process for 8xx PPMS -- even though later releases of ActiveState Perl 8xx include more modern versions of Scalar::Util. That's a lot of modules -- including my own Class::InsideOut.

With VanillaPerl, I and other users don't have to rely upon PPM repositories -- they just use CPAN directly and can rely on the dependency management code already built into the CPAN toolchain. And, because a compiler is bundled (unlike with ActiveState), that includes XS modules.

I certainly don't think that's for everyone. Many end users may be perfectly happy with a PPM-dependent approach. But I think TPF and the community will benefit by creating more choices.

-xdg

Code written by xdg and posted on PerlMonks is public domain. It is provided as is with no warranties, express or implied, of any kind. Posted code may not have been tested. Use of posted code is at your own risk.

Re^3: [JOB] The Perl Foundation seeks Windows Developer
BrowserUk
created: 2006-04-02 16:34:14
perlmonks
prlmnks

I just went to CPAN and pulled [cpan://Scalar::Util]. It took less than 10 seconds to download via my dial-up connection. And doing the payer in 4 parts took almost exactly the same amount of time. And that included me mis-typing the name once. There is no reason why AS' autobuild process couldn't download the latest/required version of module dependancies. That's a procedural problem for AS, not a problem for users.

It sucks that perfectly workable modules are getting flagged as unbuildable by the AS process, especially for want of the latest version of a module, (in this case developed by an AS developer?), that they already know builds correctly as they ship it with later versions of their own distributions.

However, the version I just downloaded and built was Scalar::Util v1.17. The latest version available on cpan is v1.18. This fails to build under AS 5.8.6 because of an unresolved external 

   Creating library blib\arch\auto\List\Util\Util.lib and object blib\arch\auto\List\Util\Util.exp
Util.obj : error LNK2019: unresolved external symbol _Perl_seed referenced in function _XS_List__Util_shuffle
blib\arch\auto\List\Util\Util.dll : fatal error LNK1120: 1 unresolved externals

And that comes down to my having 5.8.6 rather the 5.8.8. A change 26054, was made to Scalar::Util::shuffle to address a problem whereby dprof would "break" shuffle. That change included calling Perl_seed(), which (apparently) was not previously exported. So change 26836 was required to export Perl_seed and fix the linker error.

My point is, this is standard version control ripples and I don't see how that would be fixed by using a different compiler?

Vanilla/Strawberry Perl may make it easier and simpler for developers to get set up to test code on Win32, but unless the vast majority of users also move to using it, the only benefit woudl be in the statistics of successful Win32 builds--which would be pretty useless unless users can benefit. It could even make things worse, as I could see the standard response to questions about build problems of packages on win32 being "it works with Vanilla Perl, so it must be Active State that is broken", which wouldn't help many users.

Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
Lingua non convalesco, consenesco et abolesco. -- Rule 1 has a caveat! -- Who broke the cabal?
"Science is about questioning the status quo. Questioning authority".
In the absence of evidence, opinion is indistinguishable from prejudice.
Re^4: [JOB] The Perl Foundation seeks Windows Developer
xdg
created: 2006-04-02 17:15:17
perlmonks
prlmnks
I could see the standard response to questions about build problems of packages on win32 being "it works with Vanilla Perl, so it must be Active State that is broken", which wouldn't help many users.

It wouldn't help many users of ActiveState Perl, you mean. If ActiveState is broken and something else works, isn't that an improvement for users in general? Maybe some friendly co-opetition would improve ActiveState, too.

As to the Scalar::Util thing -- it has more to do with the fact that ActiveState's auto-build procedure uses their earliest 5.8 Perl release as the the basis for PPM generation without any upgrades to any modules in core. 8xx PPM's are supposed to be ones that will work with the earliest 5.8 Perl released by ActiveState -- which was several years ago. So anyone depending on ActiveState-built PPM's has to accept a 3-year lag in compatibility.

It's compounded by the fact that Perl can't easily replace core modules on Windows if they are in-use. demerphq has recently patched ExtUtils::Install to use the Win32 API to do this on Windows (and the old files are removed at the next reboot). That's great news for VanillaPerl, but doesn't help ActiveState as, again, they're stuck with ExtUtils::Install from 5.8.0.

The inability to rely upon any upgrades to core modules in 5.8 is a major flaw in my opinion. I'm sure there have been substantial bug-fixes to numerous core modules from 5.8.0 to 5.8.8. But any distribution that depends on certain behavior of upgraded modules will fail a dependency version requirements checks for building a PPM or may not behavior properly if the dependency version is ignored.

A good example of that is File::Spec and company. Look at the Changes file and look at how many Win32 bug fixes there have been in the last year alone. There are a lot. Any any code which relies upon those bugs being fixed can't require a modern File::Spec and still pass ActiveState's PPM build process.

The point is that this is not "standard version control ripples" as you put it. This is a fundamental difference in the philosophy of module packaging, distribution and support. Both are valid and they just serve different "customers" who have different wants.

-xdg

Code written by xdg and posted on PerlMonks is public domain. It is provided as is with no warranties, express or implied, of any kind. Posted code may not have been tested. Use of posted code is at your own risk.

Re^5: [JOB] The Perl Foundation seeks Windows Developer
BrowserUk
created: 2006-04-02 17:47:24
perlmonks
prlmnks
It's compounded by the fact that Perl can't easily replace core modules on Windows if they are in-use. The inability to rely upon any upgrades to core modules in 5.8 is a major flaw in my opinion

I don't get this? I install upgrades to core modules all the time and I never have to re-boot afterwards. What is this all about?

Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
Lingua non convalesco, consenesco et abolesco. -- Rule 1 has a caveat! -- Who broke the cabal?
"Science is about questioning the status quo. Questioning authority".
In the absence of evidence, opinion is indistinguishable from prejudice.
Re^6: [JOB] The Perl Foundation seeks Windows Developer
xdg
created: 2006-04-03 00:30:46
perlmonks
prlmnks

As I understand it, if a file is open, e.g. C:\perl\lib\CPAN.pm, that file can't be deleted by Windows. You can install a newer version as C:\perl\site\lib\CPAN.pm, but @INC lists C:\perl\lib before C:\perl\site\lib, so you'll wind up using the older version when you think you're using the newer one.

demerphq's patch (not yet on CPAN) uses Win32 API calls that effectively rename the open file, install the new file in place of the old one, and schedule the renamed old file for deletion upon next reboot. (Ever see how Windows install/uninstall programs tell you they can't complete until rebooting? Same thing.)

-xdg

Code written by xdg and posted on PerlMonks is public domain. It is provided as is with no warranties, express or implied, of any kind. Posted code may not have been tested. Use of posted code is at your own risk.

Re^7: [JOB] The Perl Foundation seeks Windows Developer
BrowserUk
created: 2006-04-03 09:54:28
perlmonks
prlmnks
As I understand it, if a file is open, e.g. C:\perl\lib\CPAN.pm, that file can't be deleted by Windows.

Whilst true, this is a Good Thing(TM) not a bad one. And the simple answer is to close the file before you try and delete it. Of course, that doesn't solve the real problem, but it would for the problem as you describe it.

But the bigger problem is that your understanding of the actual problem that demerphq has addressed in incorrect.

Ever see how Windows install/uninstall programs tell you they can't complete until rebooting?

Of course.

Same thing.

Actually not!

For the real problem see demerphqs post at Re^6: JOB The Perl Foundation seeks Windows Developer.

And that succinctly demonstrates my biggest fear with respect to the proposal in the OP. It is being driven by the needs of people who do not know the platform and it's APIs, and are proposing solutions without a full understanding of the real problems, or the possibilities available for solving them.

Did you ever hear/read the lyrics to the old XTC song, Making plans for Nigel?

Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
Lingua non convalesco, consenesco et abolesco. -- Rule 1 has a caveat! -- Who broke the cabal?
"Science is about questioning the status quo. Questioning authority".
In the absence of evidence, opinion is indistinguishable from prejudice.
Re^8: [JOB] The Perl Foundation seeks Windows Developer
demerphq
created: 2006-04-03 10:17:48
perlmonks
prlmnks

Well, to be fair to xdg, I dont think his characterization is wrong. Its a touch on the simplisitic side, and glosses over some of the finer points, but hes not really wrong.

I think you have a point when you say

It is being driven by the needs of people who do not know the platform and it's APIs, and are proposing solutions without a full understanding of the real problems, or the possibilities available for solving them.
However I see this as an issue of education. Since the VP and SP projects are intended to broaden access to Win32/Perl development enviornments hopefully we can expect to see people start to ramp up their Win32 knowledge and long term we can put aside of sillier UNIX-centric workarounds that have popped up.

Anyway, it would be really nice to have someone like yourself contributing to this effort, so I really hope you decide to help out.

---
$world=~s/war/peace/g

Re^6: [JOB] The Perl Foundation seeks Windows Developer
demerphq
created: 2006-04-03 02:00:13
perlmonks
prlmnks

Classic example is Cwd. CPAN uses Cwd internally. This means the Cwd.dll is locked when you try to build and install Pathtools from CPAN shell. The simple solution is to exit CPAN and go to the Pathtools directory and do the build by hand. But that only works if you are using ExtUtils::MakeMaker to do the build. If you use Module::Build then you have the same problem as CPAN would as MB also uses Cwd.dll

Likewise this problem shows up in situations where you are upgrading XS modules that are in use elsewhere in the system. For instance i have some processes that run 24 hours a day. With the updated ExtUtils::Install I can install new versions of XS modules that this process uses, and it Just Works. Without the new version EU::I Just Fails because the dlls are locked and cannot be deleted.

The strategy employed by the newer EU-I code is to try to rename the locked dll to a temporary name, install the new code, and schedule the renamed dlls for deletion at boot. If the rename fails it installs using a temporary name and arranges the replacement of the in use file at reboot.

So, there are a variety of ways that you could avoid needing this functionality, but that doesnt mean the functionality is unneeded.

---
$world=~s/war/peace/g

Re^7: [JOB] The Perl Foundation seeks Windows Developer
BrowserUk
created: 2006-04-03 10:10:36
perlmonks
prlmnks

Needless to say, (or perhaps it does need to be said?), I do understand both the problem you addressed and the MoveFileEx & MOVEFILE_DELAY_UNTIL_REBOOT solution that I assume you have used to address it.

As an aside to the primary subject of this thread, I believe that for most situations, (Ie. excluding the one you cite of a longing running Perl process that is using the DLL that is to be replaced), there is a somewhat easier solution that removes the need for the above API. I need to test it, but I will post/email you the details once I have and you can see if you think that it merits inclusion as a part of your fix.

In simple terms, if the module to be replaced is in use only by the current perl process (Ie. the one attempting to perform the replacement), then calling FreeLibrary(), or possibly FreeLibraryAndExitThread(), then overwrite the DLL, and the call LoadLibrary() to make the new version available to the current process--if required.

Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
Lingua non convalesco, consenesco et abolesco. -- Rule 1 has a caveat! -- Who broke the cabal?
"Science is about questioning the status quo. Questioning authority".
In the absence of evidence, opinion is indistinguishable from prejudice.
Re^8: [JOB] The Perl Foundation seeks Windows Developer
demerphq
created: 2006-04-03 10:28:28
perlmonks
prlmnks

MoveFileEx & MOVEFILE_DELAY_UNTIL_REBOOT solution that I assume you have used to address it.

Yep, thats the one. And some weird logic with renaming and stuff but MoveFileEx is the tricky part.

In simple terms, if the module to be replaced is in use only by the current perl process

Yeah, i thought about this too. Actually we thought of a number of tricks, but eventually there was something like a consensus that using MoveFileEx would be best as its covers all the scenarios thus means its a fairly simple mechanism. Using FreeLibrary() would only really resolve DLL related problems within a limited context, and we have reason to believe that other contexts are common enough that they should be dealt with. I mean, one advantage to the current setup is that a program could reinstall something like the MSVCRT dll if it needed to.

There is also another use case to consider, the default open mode for native Win32 perl is to open with r/w sharing, but not delete sharing. This means that a module that ends up leaving a filehandle open (say because it has a DATA section) will be opened in a form that cant be deleted nor renamed. Using MoveFileEx even such a file as this could be dealt with, albeit with a potential reboot.

Anyway, unfortunately I scheduled the ex-ExtUtils-Install releases for deletion in preparation of releasing a new production version of ExtUtils-Installs, but there is a good chance you can still get it from CPAN. The latest contains a thinko relating to MANIFEST.SKIP so it might be best to wait for the "release" version which wont happen until weds or so. Anyway, id love to hear any feedback you might have, or if you feel up to it, any patches you might want to contribute.

---
$world=~s/war/peace/g

Re^5: [JOB] The Perl Foundation seeks Windows Developer
syphilis
created: 2006-04-02 23:28:59
perlmonks
prlmnks
demerphq has recently patched ExtUtils::Install to use the Win32 API to do this on Windows (and the old files are removed at the next reboot). That's great news for VanillaPerl, but doesn't help ActiveState as, again, they're stuck with ExtUtils::Install from 5.8.0.

Not so. The latest build of ActiveState perl has EU::MM-6.30, which includes EU::I-1.33 - all of which is stock-standard perl 5.8.8.

Cheers,
Rob
Re^6: [JOB] The Perl Foundation seeks Windows Developer
xdg
created: 2006-04-03 00:50:34
perlmonks
prlmnks

The latest ActiveState build is standard 5.8.8. But the ActiveState automated PPM build farm is the first 5.8 release by ActiveState.

That means that for any module to be automatically built into a PPM, it has to be backwards compatible with Perl 5.8.0 core modules, even if the latest ActiveState release includes updates.

Here's an example from the [http://ppm.activestate.com/BuildStatus/5.8-C.html|PPM Build Status] for my own Class::InsideOut:

#     Tried to use 'Class::InsideOut'.
#     Error:  "refaddr" is not exported by the Scalar::Util module
# Can't continue after import errors at D:\cpanrun\depot\main\contrib-patched\perl\CPAN\src\Class-InsideOut\blib\lib/Class/InsideOut.pm line 16
# BEGIN failed--compilation aborted at D:\cpanrun\depot\main\contrib-patched\perl\CPAN\src\Class-InsideOut\blib\lib/Class/Idubious

Now let's climb in the way-back machine to Perl 5.8.0 in July 2002 and look at the [http://search.cpan.org/src/JHI/perl-5.8.0/ext/List/Util/lib/Scalar/Util.pm|Scalar::Util source]:

# Scalar::Util.pm
#
# Copyright (c) 1997-2001 Graham Barr . All rights reserved.
# This program is free software; you can redistribute it and/or
# modify it under the same terms as Perl itself.

package Scalar::Util;

require Exporter;
require List::Util; # List::Util loads the XS

our @ISA       = qw(Exporter);
our @EXPORT_OK = qw(blessed dualvar reftype weaken isweak tainted readonly openhandle);
our $VERSION   = $List::Util::VERSION;

Notice how refaddr is missing? For reference, in Perl 5.8.0, $List::Util::VERSION happens to be 1.07.

By Perl 5.8.1 in September 2003, Scalar::Util was up to 1.13 and refaddr was available in core Perl.

But -- to my point -- Class::InsideOut can't be built automatically into a PPM by ActiveState because their build farm is designed to be backwards compatible nearly 4 years all the way to 5.8.0.

-xdg

Code written by xdg and posted on PerlMonks is public domain. It is provided as is with no warranties, express or implied, of any kind. Posted code may not have been tested. Use of posted code is at your own risk.

Re^7: [JOB] The Perl Foundation seeks Windows Developer
demerphq
created: 2006-04-03 04:32:47
perlmonks
prlmnks

But -- to my point -- Class::InsideOut can't be built automatically into a PPM by ActiveState because their build farm is designed to be backwards compatible nearly 4 years all the way to 5.8.0.

Oh, theres a simple and easy solution to that. Dont use Scalar::Util to get refaddr(). Get it from Data::Dump::Streamer instead. At the same time you can add the DDS serilization support you mentioned a while back. And since DDS builds just fine under the PPM infrastructure everything will work out nicely.

And if you want a less painful to use reftype() function you can get it from DDS too. :-)

---
$world=~s/war/peace/g

Re^8: [JOB] The Perl Foundation seeks Windows Developer
xdg
created: 2006-04-03 07:09:48
perlmonks
prlmnks

Heh! Well, that goes to show that TIMTOWTDI still applies!

In this specific example, I wouldn't use DDS in practice because I'm trying to limit dependencies to 5.8 core modules only.

Once Class::InsideOut reaches its 1.0 release, I was going to requiest that a PPM be generated in the University of Winnipeg PPM repository anyway.

-xdg

Code written by xdg and posted on PerlMonks is public domain. It is provided as is with no warranties, express or implied, of any kind. Posted code may not have been tested. Use of posted code is at your own risk.

Re^9: [JOB] The Perl Foundation seeks Windows Developer
demerphq
created: 2006-04-03 08:01:14
perlmonks
prlmnks

What I would do in this situation is fake it. Set it up so on 5.8.0 or whatever weirdo version AS is using DDS becomes a prereq. Then structure your code so you use refaddr from either source. That should get it sorted enough to distribute through their infrastructure, and shouldn't really affect normal users. I reckon only a little bit of Makefile.PL magic would be required. Just do a check like 

 (   $^O eq 'MSWin32' 
  && eval "use Scalar::Util qw(refaddr); 1") 
 ? ( 'Data::Dump::Streamer' ) : ()
---
$world=~s/war/peace/g

Re^10: [JOB] The Perl Foundation seeks Windows Developer
xdg
created: 2006-04-03 11:50:04
perlmonks
prlmnks

I might yet do that for Class::InsideOut, but I don't think that's a really good long-term fix. A quick spin through Module;:CoreList shows 299 of 359 modules in the perl 5.8.8 core being new or changed since 5.8.0 -- who knows what kind of differences are lurking. Keeping track and finding/writing proxies anytime I use a core modules is more headache than I'd like. I'd much rather be able to specify an up-to-date core module and be done with it.

-xdg

Code written by xdg and posted on PerlMonks is public domain. It is provided as is with no warranties, express or implied, of any kind. Posted code may not have been tested. Use of posted code is at your own risk.

Re^4: [JOB] The Perl Foundation seeks Windows Developer
adamk
created: 2006-04-03 01:33:12
perlmonks
prlmnks
There is no reason why AS' autobuild process couldn't download the latest/required version of module dependancies. That's a procedural problem for AS, not a problem for users.
You assert there is no reason, and yet here we are without AS autobuild support for vast quantities of things. It is a problem for users because AS are the defacto gatekeepers for what is and is not supported on Windows. Lets look at just the CPAN packages that start with "P".

http://ppm.activestate.com/BuildStatus/5.8-P.html

Vast swaths of modules no longer work in the autobuild system, the grand irony being that Windows is the platform with the WORST support in the PPM autobuild system, and Mac OS X being the best, when one would think given their relative userbases, Windows would be the best supported.

Looking at some common things from that list...

PAR does not work, PDL does not work, PGP does not work, my new PITA testing project does not work, POE (which has NO non-core deps by default) does not work, PPI (which has almost no non-core deps) does not work, and thus most of Perl:: does not work, Plucene does not work, POSY doesn't, Process doesn't, Perl-GPS doesn't, plus dozens of less well used modules.

Many of these use plain ordinary pure Perl code with proper platform-neutral File::Spec usage.

Although it may APPEAR there is no reason, there most certainly is. And despite efforts to fix the autobuild system, it is increasingly appearing that these are systematic faults, inherent to the design of the PPM system itself, and may well be unfixable without breaking back-compatibility.

I and others have raised this problem a number of times, via the online forms, emails, and even in person at OSCON. I've spent a year in the rediculous position that as the AUTHOR of PPI I cannot install it on my own Windows laptop so I can do demonstrations during my PPI talk.

Or at least, I cannot without a vast amount of work, installing the compiler and make myself, then downloading the entire chain of 20 distributions manually and hand-installing all of them.

The point of installers is to install things.

And as long as everything Just Worked, the PPM system was a perfectly acceptable solution.

But despite gozer's and other people's efforts, it has now reached a point where the problems have become too much for me. When something like this goes wrong, I am in a position where there is nothing at all I can do to fix the problem, other than to bitch to ActiveState. And I hate to bitch about things which I can't fix myself if needed.

So if there are prodedural problems inside of ActiveState PPM system that I cannot fix, that even THEY might not be able to fix, then it is clear to me that what the Perl community needs is a way to at least have the OPTION to help itself in these situations.

The CamelPack has helped solve the short term problem by at least making it easy to set up a compiler and a make with AP.

Vanilla has provided an excellent research distribution so we can look at Win32 problems very close to and in the Perl core, and look at problems in the toolchain without creating an expectation that it is useful for authors or developers.

Strawberry is the next step, and is about making a minimal but complete and bug free distribution. It's the third step of an unknown number.

This is NOT about compilers, and this isn't about wanting to make Win32 Perl more like Unix. This is about wanting Win32 Perl to be like Perl on every other one of the 100+ platforms, from Plan 9 to Nokia mobile phones.

Windows Is No Special Flower.

The low level common Perl modules should be able to build and install portably on all platforms (or as many as possible), and from there normal Perl developers should not have to care which specific platform they are on if they are not doing anything exotic. It should Just Work.

Windows is not special, it just presents some challenges, much as every other platform has presented some challenges.

And I think the Perl community as a whole deserves at least the option to face those challenges themselves, rather than loading all the work and all the pressure, and all the cost onto one company.

It's not fair to ActiveState and it's not fair to the Perl community.
Re^5: [JOB] The Perl Foundation seeks Windows Developer
BrowserUk
created: 2006-04-03 11:51:34
perlmonks
prlmnks

Adam. Please do not see my questions and calls for consultation in this thread as dissention to the basic motives of your OP. Indeed I am enthused to be a participant in addressing the underlying causes of the problems you are trying to address.

Please see Re^9: JOB The Perl Foundation seeks Windows Developer for some explanation of my requests of further information/consultation before