JNode class library reorganization

Project:JNode All
Component:Miscellaneous
Category:task
Priority:normal
Assigned:lsantha
Status:active
Description

The JNode class library is being factored out to a separate SVN directory. This step is required on one hand for reducing the size of the overal source code that a JNode developer is exposed to and on the other hand for providing support for OpenJDK 7 while OpenJDK 6 based class library will be maintained for a while.

The main source tree of JNode is in the trunk svn directory while the OpenJDK 6 based JNode class library resides in the classlib6 directory (which is a sibling of trunk).
If someone intends to work on the class library it is recommanded to check out classlib6 in a separate directory.

Under the current circumstances from the classlib6 subproject we produced two files: classlib.pack.gz and classlib-src.jar.bz2. These are added to the all/lib directory of trunk. After an SVN update the build process will pick them up and convertem them into a format suitable for consuption by the main JNode project (classlib.jar and classlib-src.jar are created in the same directory).

Certain aspects of the classlib6 build proccess are not yet fully worked out. Also on in the main JNode project certain build optimizations are being considered.

Further progress of this task will be registered here. Upon completion this thread can be used to extend the JNode documentation.

#1

Submitted by lsantha on Wed, 04/29/2009 - 20:29.

The way the classlib.jar was handled inside jnode trunk has been improved and optimized.
classlib.jar is not unpacked any more. Its contents is processed when the classlib.jar changes or when a clean build is issued. The processing consists of building the plugins which contain the classlib classes in one step before building the other plugins. After this step the classlib plugins are reused until the next clean build or classlib.jar modification.
This change should also fix the situation where Eclipse (unlike the rest of tools and IDEs in use here) was not able to handle properly the situation after classlib.jar was unpacked. (The issue is described here in more detail: http://www.jnode.org/node/2995)

Eclipse users, please check if there are any more isues.

#2

Submitted by lsantha on Fri, 05/08/2009 - 19:36.

I've moved out the two classlib binaries from SVN to an FTP/download service that was donated to JNode earlier this year.

Who is working on trunk should only make SVN update and build JNode. You don't need to do anything special. It should all work 'out of the box'.

Who was working also on classlib6 is recommanded to remove the old classlib6 directory and check out classlib6 again from SVN. Please contact me for further information.

#3

Submitted by Stephen Crawley on Sat, 05/09/2009 - 03:57.

Please contact me for further information.

Please provide that further information here. It will save me having to post your emails in a followup comment.

Thanks.

#4

Submitted by lsantha on Sat, 05/09/2009 - 06:26.

For those who prefer to have the classlib6 project on their disk and do not want to get the classlib6 binaries over the net when someone else changes classlib6, they can add jnode.local.classlib=true to their jnode.properties in the JNode sourcetree and they should use quickdeploy in classlib6 to build and get the binaries locally.

For those who want to modify classlib6 and create and distribute new binaries: the proccess involves sensitive information which is not supposed to be posted here neither by myself nor by anyone else.
So please contact me.

#5

Submitted by Stephen Crawley on Sat, 05/09/2009 - 07:07.

Thank you for that clarification. I guess we'd be in this position (having to deal with sensitive access control information) no matter what solution we used for the problem of uploading classlib binaries. It is unfortunate though ...

Anyway it is NOW time that all of this was properly documented (minus the sensitive details of course) in the Documentation tree. The classlib6 reorg is finished now ... so it should be documented now. No more excuses Levente Smiling

#6

Submitted by cluster on Sat, 05/09/2009 - 10:07.

Just wanted to say, i really like this latest solution with the auto-ftp download.

Building core now only requires ~400mb of ram, and most other projects don't use over 300mb, so this is a whole lot nicer for those that aren't sporting 4gb+ ram. The only big ram user now is from assemble-plugins onward to building the iso, ram usage is around 6-800mb. Still much better than 1.5gb+ it used to be!

Also, my build times are much better as a result, old build times used to be 12-15 minutes for a clean build.

Download + Unpack + Build
7m 14s

Build only
4m 48s

#7

Submitted by lsantha on Sat, 05/09/2009 - 15:40.

Thank you for the information.
One of the main points of this reorganization was to make the job the majority of JNode programmers easier. I'm glad that to see it reaching this goal.
I will continue to investigate ways of making the JNode development a more productive activity.

#8

Submitted by Stephen Crawley on Sat, 05/09/2009 - 15:42.

For the record, in addition to the steps described in #4, I found that you needs to set "jnode.root.dir" in the jnode.properties file in the Classlib6 tree to point to your JNode tree.

We really need proper documentation for this Levente.

A suggestion to make classlib builds faster would be to rearrange the tasks so that quickdeploy doesn't go through the process of creating the pack and gz files, but simply created the jar files and copied them. The pack file in particular takes a long time to create ... even on a 4Gb machine with a preloaded FS buffer cache.

#9

Submitted by lsantha on Sat, 05/09/2009 - 15:47.

Yep, you are right... (even though I made the build system report to you what is missing and where)

#10

Submitted by Stephen Crawley on Sat, 05/09/2009 - 15:55.

Yea ... it did tell me ... after spending ~5 minutes building the pack and gz files Smiling

#11

Submitted by cluster on Sat, 05/09/2009 - 15:58.

A suggestion to make classlib builds faster would be to rearrange the tasks so that quickdeploy doesn't go through the process of creating the pack and gz files, but simply created the jar files and copied them

Better yet, if your on a unix machine, why not make it a hardlink, then you don't even have to copy it.

#12

Submitted by Stephen Crawley on Sat, 05/09/2009 - 16:05.

Re #11. I don't think that hardlinks are a good idea. They only save a second or so (on my machine) and they make the build.xml file development platform dependent.

#13

Submitted by cluster on Sat, 05/09/2009 - 16:09.

Oh i didnt mean put them in build.xml, i meant just do it in your own sandbox. Your right though, its probably not going to save you much, its just generally my first instinct to link instead of copy.

#14

Submitted by lsantha on Sat, 05/09/2009 - 16:14.

Indeed, we should keep (make) the development process completely platform independent. We shouldn't forget in any case that probably most Java developers are using windows out there which means that as soon as more developers will join JNode the majority of developers will be on windows.

On seeing more and more bash scripts in the source tree at more or less critical places I was thinking about using BeanShell for all our scripting needs. BeanShell is already in JNode. The startup scripts for a beanshell script then will be one-liners looking nearly the same both for unix and windows.