Sorry, but I'm integrating my stuff into our test harness now. {groan}
I'm not exactly sure what you're getting at, but so far, I've been pretty much content with Test::Harness. If I had to test anything other than Perl, I would go either with the native test environment or, if that one doesn't work out, with the Test Anything Protocol, which specifies the output just like Test::Harness, so that Test::TAP::Model can generate pretty reports from it.
Is there one that doesn't suck? Any thoughts on a custom built test harness that is written in Java (barf)? Test harness gripes? How difficult is it for you to stop coding so that your test meets the harness seemlessly as the harness will allow? Clarification on that last one: you have to not code your own test harness since you already have one provided for you. Any gripes on limitations of your test harness? Dopey scenario sytax? Ridiculous work arounds you've had to do?
I work quite happily with Test::Harness and prove most of the time. I've got a few custom Test::Harness::Straps based runners for particular issues but generally I've found the Perl test harnesses to do what I need.
What problems are you having?
I happen to know of a department in a company that uses a test harness written in Java. Except for heavy stress tests, can you guess which gets more CPU time: the test or the harness? Do I even have to tell you? When I asked why we were using a CPU heavy/Java test harness rather than one written (and written well, ot course) in C, I was told (under the table) that long ago, it was a political decision! It would have hurt more feeling to choose the C one over the Java one!
OMG!
Luckily, this dept. now has a good Java programmer on staff, so it is slowly getting better. The original coder wrote the test harness because he "wanted to learn Java." Oh, boy!
When I asked why this dept was using a CPU heavy/Java test harness rather than one written (and written well, ot course) in C, I was told (under the table) that long ago, it was a political decision! It would have hurt more people's feelings to choose the C one over the Java one!
Well if the code being tested is Java I'd go the Java route too. Keeping the testing framework and the code being tested in the same language usually makes things much easier.
If I were you I'd just start things moving to one of the decent Java frameworks. TestNG is very nice, and the latest JUnit is pretty good too.
Again, I can't make major architectural decisions on the test harness. About the most I can do is request features if I can justify them. Although, in the next few years I might be able to make a case for those other Open Source harnesses you suggested.
As for writing the tests in Java just because the harness is writen in Java, well, it doesn't really matter. The harness-to-test interface is language independent. It has it's own syntax, and coding in Java gains no added benefit.
But thanks again for suggesting JUnit & TestNG. That may very well turn the corner for our dept in the years to come. I mean it. Sincerely.