Regression: detected by mauve

Project:JNode Core
Category:bug report

The following two successive results of the mauve based testsuite runs on 2009-05-03 demonstrate the appearence of a set of regressions that need to be investigated:
By comparing the summaries:
Run Date: May 3, 2009 1:09:39 PM GMT
Passed: 61305
Failed: 509
2. Summary:
Run Date: May 3, 2009 4:14:35 PM GMT
Passed: 60660
Failed: 532


Found part of it, though it only seems to directly account for some of them. classlib6, r5395

-    private static final 
-        AtomicReferenceFieldUpdater bufUpdater = 
-        AtomicReferenceFieldUpdater.newUpdater
-        (BufferedInputStream.class,  byte[].class, "buf");
+    private static final AtomicReferenceFieldUpdater bufUpdater;
+    static {
+        bufUpdater =
+                new PrivilegedAction() {
+                    @Override
+                    public AtomicReferenceFieldUpdater run() {
+                        return AtomicReferenceFieldUpdater.newUpdater(
+                                BufferedInputStream.class,  byte[].class, "buf");
+                    }
+                });
+    }

Some associated exceptions...

All of the java.text.BreakIterator tests fail with this exception

the java.text.CollationElementIterator tests don't specify why they failed.

the java.beans.Eventhandler tests have these NPEs

And then, the very definitive...

The purpose of the patch was to fix an issue with creating a BIS. You simply couldn't before. It would give you an access, exception. I had found the bug initially when trying to use a BIS within a command, and got a SecurityException thrown while instantiating the BIS class. (This was from deep in the mm during allocation when the VmType was being initialized for the first time)

This was discussed a bit more on IRC between myself and steve, the logs for that day are long, and the discussion is at the bottom...


There is more to worry about than just regressions. Add up the number of test and you'll see they have changed, 61205 + 509 = 61714, where as 60660 + 532 = 61192. That's a difference of 545 less passed, 23 more failed, and 522 less total test.

Usually what happens is a test fails part way through and reports X passed, Y failed, X+Y+Z total, so even the totals are wrong. This is what happened here, the columns say 11 passed, 1 failed, which should be 12 total, but it's reported as 11 passed, 0 failed, 11 total, I see that as a problem

I noticed this a while back, but have no clue as to where the problem lies, I'd assume mauve is partly to blame, as even a crashed test should not corrupt the results of the testsuite.

This is also related to the Regression test support for mauve, I would like to see this included in as part of the hourly testsuite run results