Regression: detected by mauve
Project: | JNode Core |
Component: | Code |
Category: | bug report |
Priority: | critical |
Assigned: | Unassigned |
Status: | active |
Jump to:
Description
The following two successive results of the mauve based testsuite runs on 2009-05-03 demonstrate the appearence of a set of regressions that need to be investigated:
1. http://testsuite.jnode.bucksch.org/2009-05-03-13-00/results/
2. http://testsuite.jnode.bucksch.org/2009-05-03-16-00/results/
By comparing the summaries:
1.Summary:
Run Date: May 3, 2009 1:09:39 PM GMT
Passed: 61305
Failed: 509
2. Summary:
Run Date: May 3, 2009 4:14:35 PM GMT
Passed: 60660
Failed: 532
- Login to post comments
#1
Found part of it, though it only seems to directly account for some of them. classlib6, r5395
Some associated exceptions...
All of the java.text.BreakIterator tests fail with this exception
http://testsuite.jnode.bucksch.org/2009-05-03-16-00/results/java/text/Br...
the java.text.CollationElementIterator tests don't specify why they failed.
the java.beans.Eventhandler tests have these NPEs
http://testsuite.jnode.bucksch.org/2009-05-03-16-00/results/java/beans/E...
http://testsuite.jnode.bucksch.org/2009-05-03-16-00/results/java/beans/E...
http://testsuite.jnode.bucksch.org/2009-05-03-16-00/results/java/beans/E...
http://testsuite.jnode.bucksch.org/2009-05-03-16-00/results/java/beans/E...
And then, the very definitive...
http://testsuite.jnode.bucksch.org/2009-05-03-16-00/results/java/io/Buff...
http://testsuite.jnode.bucksch.org/2009-05-03-16-00/results/java/io/Buff...
http://testsuite.jnode.bucksch.org/2009-05-03-16-00/results/java/io/Buff...
The purpose of the patch was to fix an issue with creating a BIS. You simply couldn't before. It would give you an access, exception. I had found the bug initially when trying to use a BIS within a command, and got a SecurityException thrown while instantiating the BIS class. (This was from deep in the mm during allocation when the VmType was being initialized for the first time)
Update:
This was discussed a bit more on IRC between myself and steve, the logs for that day are long, and the discussion is at the bottom...
http://echelog.matzon.dk/logs/browse/jnode.org/1241301600
#2
There is more to worry about than just regressions. Add up the number of test and you'll see they have changed, 61205 + 509 = 61714, where as 60660 + 532 = 61192. That's a difference of 545 less passed, 23 more failed, and 522 less total test.
Usually what happens is a test fails part way through and reports X passed, Y failed, X+Y+Z total, so even the totals are wrong. This is what happened here, the columns say 11 passed, 1 failed, which should be 12 total, but it's reported as 11 passed, 0 failed, 11 total, I see that as a problem
I noticed this a while back, but have no clue as to where the problem lies, I'd assume mauve is partly to blame, as even a crashed test should not corrupt the results of the testsuite.
This is also related to the Regression test support for mauve, I would like to see this included in as part of the hourly testsuite run results