JNode needs a pre-build configuration tool

Project:JNode Core
Component:Miscellaneous
Category:feature request
Priority:normal
Assigned:Stephen Crawley
Status:closed
Description

JNode needs a tool to configure a freshly downloaded / checked out source distro. My vision is that this tool would interactively capture details of the user's build target platform; e.g. target m/c architecture (32 vs 64 bit), emulator selection and virtual hardware settings, choice of pre-built virtual hard disc images. The list is open ended.

We discussed this on IRC, and the high-level requirements / constraints for this tool that emerged from the discussion are:

  • Command line based configuration; not GUI.
  • Same configuration tool for Linux and Windows.
  • No new build platform dependencies; no Perl (etc), no cygwin (etc), no autoconf (etc).
  • The tool should be script-driven, for some definition of script.
  • The configuration work should be done using ant build files.

Please add any other requirements that you can think of as comments. We need to understand what we are trying to do before we get to the implementation details.

Also, if anyone knows of an existing, open-source Java-based solution ... maybe we can avoid reinventing the square wheel Smiling.

#1

I've been experimenting with building a simple JNode configurator using (just) Ant scripts. I've attached a first cut at an Ant script that knows about half a dozen configuration settings.

I'm not entirely happy with this approach. This Ant script is inflexible, limited (e.g. minimal validation) and very easy to get wrong. I could maybe improve things by writing some custom Ant tasks, or I could implement similar (but better) functionality as a free-standing Java app.

Oh yes ... to run the attached script you need Ant 1.7.0.

AttachmentSize
configure.xml_.txt6.74 KB

#2

Rather than continuing with Ant, I've reimplement the tool as a standalone Java command-line application. I've attached my latest version as a gzip'ed tar file. To try out the tool, untar it into a fresh directory somewhere (not in your JNode sandbox), compile using "ant compile", and run as follows:

    export CLASSPATH=classes:lib/nanoxml-2.2.3.jar
    java org.jnode.configure.Configure samplescript.xml

Note I've decided to use the latest version of nanoxml-java (not
nanoxml-lite) so that the parser can produce decent diagnostics for missing / incorrectly named elements and attributes in the script file.

AttachmentSize
configure.tar_.gz_.txt37 KB

#3

Here are some suggestions from Peter:

  1. Add some kind of "include" mechanism so that the script can be built up from different XML files; e.g. from the various projects.

    We need to understand the use-cases a bit to work out how "include" ought to behave. For instance, would we want to be able to have multiple projects contributing properties to one properties file?

  2. Add a mechanism for expanding properties into existing template files.

    This could be done using an ant task either at configure time or build time, but it could also make sense to do it in this tool.

  3. Bug: the vmsize parameter needs to be a multiple of 4.

    This can be done using a regex.

#4

Some ideas from me:

  1. Support more property file formats. For example, the Java 1.6 java.util.Properties API supports an XML property file format.
  2. Support getting defaults from a property file (e.g. like jnode.properties.dist) rather than wiring the defaults into the script file.
  3. Implement hooks for performing consistency checks via plug-in classes.

#5

Regarding the vmsize parameter. What do you think about an extra optional condition where you could add something like "(v > 256M) && (v < 4000M) && (v % 4 == 0)"?

Regarding point 2. The usecase I had in mind is to have some options in the vm core configurable, so that we can easily make different options for distributions, debugging,.. For example if we inline code, how deep do we inline and up to what size, if debug or trace of the GC is activated or not.

But even without that 2. addition I like the tool. Good work!

#6

Re: the vmsize parameter (as an example) and the extra optional condition. The problem is that the tool needs code to parse and evaluate the condition expressions. This is do-able, but it is easier to delegate checking to plug-in Java classes. Besides, at a pinch this example could be implemented using a (messy) regex.

I'm inclined to take the "simple but clunky" plug-in approach for now,

Thanks for the use-case too.

#7

JNODE BUILDER
=============

Being busy with personal things I haven't had time to try this out yet but from the comments so far it looks like this will be a tool that users might not find very easy to use.

Hereby I propose the creation of a new standalone graphical user interface based tool for JNode called: JNode Builder
This tool should have the following functions:

- creation of jnode plugin list where the user can select the set of desired plugins from the list of available plugins. Plugin manager can be reused here to cope with the plugin dependencies. If a plugin is selected its dependences will be selected too. The saving of such set of plugins, together with the main class of the plugin list and other runtime setting will result a plugin list file.

- support for various execution environments like vmware, qemu, bochs, vitrualbox etc. The user will select what type emulator, real or virtual hardware will be used and genearte the required configuration files, having the possibility to specify certain key settings like amount of RAM in that configuration.

- customization of the JNode build environment, including support for jnode.properties file, custom plugin list location, custom plugins and other settings

- enable the creation of plugins by the user, add required jars, specify exported/imported packages and support extension points. The effective implementation of this feature will need changes in the current build system as well in order to support the inclusion of prebuilt plugins. Currently adding a new plugin to jnode from an extarnal library is way more difficult than it should be. This feature will enable the user to start with an external jar and end with a prepackaged JNode plugin ready for inclusion into a JNode configuration or plugin list. Such prebuilt plugins will not be rebuilt each time we build JNode using ANT.

Such tool would support the user in the creation of highly customized JNode configurations and will save a lot of time for the users. I think many users are simply unable to think of custom JNode configurations and are not aware of the modularity and felxibility of JNode. Such tool would make those features readily available and usable via a user friendly, intuitive GUI. I think that would make a huge difference for JNode adoption!
Obviously the above list of features can be refined and extended at will as JNode Builder evolves.

Regards, Levente

#8

In my opinion the graphical frontend is not so important. Quite contrary I think a console tool has much benefits as it runs anywhere (also on headless clients) and we will be able to use it in JNode from the beginning. The xml-way of specifying the rulez makes it easy to add a graphical gui later on.

Regarding the plugin framework, I'm not sure if Stephens suggestion allready contained the same. But imho these should be two different set of tools. The normal user should not be confronted with stuff like the gc to be used or options about inlining,.. these are developer options. On the other hand the user must be able to install, delete and manage his software setup (i.e. the plugins). I do not see a direct connection between those and I think there shouldn't.
In my mind I compare it to linux as kernel config scripts make menuconfig or make xconfig. Different frontends to configure the kernel. For the softwaresetup you e.g. have apt, with consolefrontends like apt-get, apt-cache,.. or gui frontends like aptitude,...

#9

The command line operation and configuration files for the tool do not conflict with the GUI since GUI and command line can be two frontends for the same tool. The configuration file is needed for the GUI too since settings should be saved somewhere.

All the point of Steves initiative was making the job of those not familiar with JNode easier.
In this context the only really efective solution is the GUI.
Further more, in the Java community GUIs are much more widelely used and popular than command line tools. The Linux analogy is misleading here, not to mention that Linux never was a synonim of user friendliness until very recent time (if at all).
The GUI can also form the basis of a future plugin for your favorit IDE.

#10

Ok, I might have misinterpreted Stephens intention, perhaps he can clarify. But from our IRC discussion (see the initial request) we said: "Command line based configuration; not GUI."
For the usecases I named above I think Stephens current tool is great, easy to use and easy to extend.

I know that GUI tools are favored over command line tools and I see a need for that for the usecases you named which are imho still different ones. Managing the plugins installed should for sure be a nice GUI application. In this regard I named Linux because it shows the two different types of config systems I had in mind: Configuring the kernel, i.e. build time options and configuring your software setup.
Btw, the software installers for all linux distributions I know were far better for a very long time than anything comparable for any version of Windows. Is it setup.exe or install.exe, do you need uninstall.exe or can you remove it via "Software",... is a dll a shared library? Smiling So imho if we get something as userfriendly and easy to use as software like aptitude we can be very happy.

#11

Peter has accurately interpreted my intention here.

The primary use case for the tool I am building is JNode build configuration NOT a JNode installation. This is a tool for JNode developers ... people who want / need to build their own JNode installation media.

Now, I think it would be a good thing if the tool was sufficiently flexible that it could be adapted to be a JNode installer sometime in the future. And >>at that point<< it would be a good thing to support a GUI ... as an option. But I would prefer not to be distracted with this potential use-case right now.

[Also I agree 100% with Peter's point about headless installation. Installers, etc that require a display are a nightmare if your machine doesn't have one. Been there, don't want to go there again!]

Management of plugins to be started at boot time is an interesting use-case. Right now, it could make sense to support configuring the plugin lists using my tool. (There are some usability issues that need to be thought through ... like how to build the candidate plugins, and how to present them for user selection.)

But we also need to look into the future to a time where JNode is installed (once) onto a user's hard-drive, and routinely booted from the hard-drive. In that scenario, the plugin list would need to be a file on the hard drive that can be updated by something running on JNode. It is not clear my tool (or an installer-based tool) would be the right approach in this context.

[On re-reading Peter's last comment, I realise that he is saying the same thing as my 2 paragraphs above are saying.]

But I think we should not get distracted with things like JNode installers, configuring s/w within JNode, plugin development tools, and so on. These are all worthy projects, but IMO it is best to address them separately. I want to get something simple working to address the immediate needs of newbie JNode developers trying to build a bootable JNode image from a source distro / svn checkout.

#12

My suggetsion above is by NO MEANS and HAS NOTHING TO DO WITH an istaller.
As its name suggests: JNode BUILDER, it is a tool operating in the JNode pre-build/build environment (so in exactly the same domain as yours) but it's a wider scope and more feature reach tool than what was suggested in the beginning of this thread which supports a wider range of activities related to the build environemnt customization and the prepararion of a JNode boot image and plugin list.

I will probably reformulate it as a new feature request which if gets implemented can incorporate your tool.

#13

I have another potential usecase. It fits right into your template-expansion and probably you had that already in mind.
We could make some of the runtime properties configurable, too. E.g. for "jnode.interpreter" so that different developers can have different defaults.

Btw on a side note. It will probably be hard to decide which parts to rebuild once you change your properties. What was your thought on that? Just tell the user to clean rebuild JNode after he changed a property? At the moment if I change the memmgr in jnode.properties I have to do a clean rebuild. So it wouldn't be a change in "comfort" for configurable system properties.

#14

I've committed a first "release" version of the configuration tool to SVN. In addition to the features of the earlier versions this one supports:

  • improved handling of empty property values,
  • relative pathnames are now resolved relative to the script file's parent directory,
  • a script file import mechanism,
  • populating of initial values from a defaults file,
  • read / write / update of classic and XML property files,
  • output of properties via template file expansion for the following file types: classic and XML property files, arbitrary XML files, Java source code files and plain text, and
  • an API for plugins supporting other file types.

I plan to add support for configuring lists (e.g. JNode plugin lists), and a second plugin API for validating property values. But first I need to write the tool documentation and start working on the scripts for configuring the "jnode.properties" file and others.

I'll try to drop into the IRC channel later today to talk about various things.

#15

What's the status of this?
Will it be usable for the upcomming release?
How about an IRC meeting to discuss this?

#16

Some features are not yet implemented, but I think the tool is good enough. The script in all/conf-source knows about most of the properties listed in the main jnode.properties.dist file. You can give it a try by running:

  $ configure.sh all/conf-source/script.xml

(The script is currently set up to output to test.jnode.properties instead of jnode.properties ... to avoid clobbering the real config settings.)

Let me know what you think.

#17

Status:active» fixed

After the last set of changes (earlier this week) which make the tool "live", I'm calling it finished for now. If there is a need for enhancements, please open a new "Feature Request".

#18

Status:fixed» closed

Automatically closed -- issue fixed for two weeks with no activity.

#19

hi, I followed the procedure on http://www.jnode.org/node/2685, but i have this error :

Exception in thread "main" java.lang.NoClassDefFoundError: local/jnode/builder/build/classes:/media/Disque
Caused by: java.lang.ClassNotFoundException: local.jnode.builder.build.classes:.media.Disque
at java.net.URLClassLoader$1.run(URLClassLoader.java:217)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:205)
at java.lang.ClassLoader.loadClass(ClassLoader.java:323)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:294)
at java.lang.ClassLoader.loadClass(ClassLoader.java:268)
at java.lang.ClassLoader.loadClassInternal(ClassLoader.java:336)
Could not find the main class: local/jnode/builder/build/classes:/media/Disque. Program will exit.