Applications starting with A

Here is details about applications whose name starts with A

• Ant

website : http://ant.apache.org/
comments :

Applications starting with B

Here is details about applications whose name starts with B

• Beanshell

website : http://www.beanshell.org/
comments :

Applications starting with E

Here is details about applications whose name starts with E

• ecj (eclipse compiler for java)

website : http://www.eclipse.org ???
comments :

Applications starting with H

Here is details about applications whose name starts with H

• hsqldb

website : http://hsqldb.org/
comments :

Applications starting with J

Here is details about applications whose name starts with J

• JUnit

website : http://junit.org/
comments : only tested in console mode.

• javac

website : http://openjdk.java.net/
comments : the sun compiler for java. It works fine, but you can run into GC bugs when you compile your first program. The following "warm-up" sequence avoids this:

1. Run "gc".
2. Run "javac" with no arguments.
3. Run "gc" again.
4. Use "javac" to compile a (small) program.
• Jetty6

website : http://www.mortbay.org/
comments : it works partially

• JEdit

website : http://www.jedit.org/
comments : by using jedit.jar alone, I only see the splash screen. If I try the installer, it fails at 0% of progress with an "IOException" dialog box but no stacktrace.

• JChatIRC

website : http://jchatirc.sourceforge.net/
comments :

Applications starting with N

Here is details about applications whose name starts with N

• NanoHttpd

website : http://elonen.iki.fi/code/nanohttpd/
comments :

Applications starting with R

Here is details about applications whose name starts with R

• Rhino

website : http://www.mozilla.org/rhino/
comments :

Applications starting with T

Here is details about applications whose name starts with T

• tomcat

website : http://tomcat.apache.org/
comments :

The 20 minute guided tour.

This is a quick guide to get started with JNode. It will help you to download a JNode boot image, and explain how to use it. It also will give you get you started with exploring JNode's capabilities and give you some tips on how to use the JNode user interfaces.

To start with, you need to download a JNode boot image. Go to this page and click on the link for the latest version. This will take you to a page with the downloadable files for the version. The page also has a link to a page listing the JNode hardware requirements.

At this point, you have two choices. You can create a bootable CD ROM and then run JNode on real hardware by booting your PC from the CD ROM drive. Alternatively, you can run JNode on virtual PC using VMWare.

To run JNode on real hardware:

1. Download the "gzip compressed ISO image" from the JNode download page.
2. Uncompress the ISO image using "gunzip".
3. Use your favorite CD burning software to burn the ISO image onto a blank CD ROM.
4. Shutdown your PC.
5. Put the JNode boot CD into the CD drive
6. Boot from the CD, following the PC manufacturer's instructions.

To run JNode from VMWare:

1. Download a free copy of VMware-Player. (You can also use the free VMware-Server which allows to modify the VM parameters and so on, or you buy one of the more advanced VMWare products.)
2. Install VMWare following the instructions provided.
3. Download the "gzip compressed ISO image" and the "vmx" file from the JNode download page.
4. Uncompress the ISO image using "gunzip", and make sure that the image is in the same directory as the "vmx" file.
5. Launch VMWare, browse to find the JNode "vmx" file, and launch it as described in the VMWare user guide.

When you start up JNode, the first thing you will see after the BIOS messages is the Grub bootloader menu allowing you to select between various JNode configurations. If you have at 500MB or more of RAM (or 500MB assigned to the VM if you are using VMware), we recommend the "JNode (all plugins)" configuration. This allows you to run the GUI. Otherwise, we recommend the "JNode (default)" or "JNode (minimal shell)" configurations. (For more information on the available JNode configurations, ...).

Assuming that you choose one of the recommended configurations, JNode will go through the bootstrap sequence, and start up a text console running a command shell, allowing you to issue commands. The initial command will look like this:

JNode />

Try a couple of commands to get you started:

JNode /> dir

will list the JNode root directory,

JNode /> alias

will list the commands available to you, and

JNode /> help <command>

will show you a command's online help and usage informatiom.

There are a few more useful things to see:

• Entering ALT+F7 (press the ALT and F7 keys at the same time) will switch to the logger console. Entering ALT+F1 switches you back to the shell console.
• Entering SHIFT+UP-ARROW and SHIFT+DOWN-ARROW scroll the current console backwards and forwards.
• Entering TAB performs command name and argument completion.
• Entering UP-ARROW and DOWN-ARROW allows you to choose commands from the command history.

The JNode completion mechanism is more sophisticated than the analogs in typical Linux and Windows shells. In addition to performing command name and file name completion, it can do completion of command options and context sensitive argument completion. For example, if you want to set up your network card using the "dhcp" command, you don't need to go hunting for the name of the JNode network device. Instead, enter the following:

JNode /> dhcp eth<TAB>

The completer will show a list of known ethernet devices allowing you to select the appropriate one. In this case, there is typically one name only, so this will be added to the command string.

For more information on using the shell, please refer to the JNode Shell page,

I bet you are bored with text consoles by now, and you are eager to see the JNode GUI. You can start it as follows:

JNode /> gc
JNode /> startawt

The GUI is intended to be intuitive, so give it a go. It currently includes a "Text Console" app for entering commands, and a couple of games. If you have problems with the GUI, ALT+F12 should kill the GUI and drop you back to the text console.

By the way, you can switch the font rendering method used by the GUI before you run "startawt", as follows:

JNode /> set jnode.font.renderer ttf|bdf

If you have questions or you just want to talk to us, please consider joining our IRC channel (#JNode.org@irc.oftc.net). We're all very friendly and try to help everyone *g*

If you find a bug, we would really appreciate you posting a bug report via our bug tracker. You can also enter support and feature requests there.

Feel free to continue trying out JNode. If you have the time and the skills, please consider joining us to make it better.

Booting from the network

2 options are available here

Booting from USB memory stick

This guide shows you how to boot JNode from an USB memory stick.
You'll need a Windows machine to build on and a clean USB memory stick (it may be wiped!).

Step 1: Build a JNode iso image (e.g. build cd-x86-lite)
Step 2: Download XBoot.
Step 3: Run XBoot with Administrator rights
Step 4: Open file: select the ISO created in step 1. Choose "Add grub4dos using iso emulation".
Step 5: Click "Create USB"

XBoot will now install a bootloader (choose the default) and prepare the USB memory stick.
After then, eject the memory stick and give it a try.

When it boots, you'll first have to choose JNode from the menu. Then the familiar JNode Grub boot menu appears.

Eclipse Howto

This chapter explains how to use the Eclipse 3.2 IDE with JNode.

Starting

JNode contains several Eclipse projects within a single SVN module. To checkout and import these projects into Eclipse, execute the following steps:

1. Checkout the jnode module from SVN using any SVN tool you like.
   Look at the sourceforge SVN page for more details.
2. Start Eclipse
3. Select File - Import. You wil get this screen.

   

4. Select "Existing project into workspace".
5. Enter the root directory of the imported jnode CVS module in this screen.

   

   The listed projects will appear when the root directory has been selected.

6. Select Finish
7. You will end up with all projects into your Eclipse workspace like this:

   

Building

You can build JNode within Eclipse by using the build.xml Ant file found in the JNode-All project. However, due to the memory requirements of the build process, it is better to run the build from the commandline using build.bat (on windows) or build.sh (on unix).

Running in Bochs

Running JNode in Bochs does not seem to work out of the box yet. It fails on setting CPU register CR4 into 4Mb paging mode.
A compile setting that enables 4Mb pages is known to solve this problem. To enable this settings, running configure with the --enable-4meg-pages argument or add #define BX_SUPPORT_4MEG_PAGES 1 to config.h.

Running in KVM

Setup

If you have a CPU with hardware virtualization support you can run JNode in kvm wich is much faster than vmware-[player|server] (at least for me). You need a CPU that either supports Intel's IVT (aka Vanderpool) or AMD's AMD-V (aka Pacifica).

With

egrep '^flags.*(vmx|svm)' /proc/cpuinfo"

you can easily check if your CPU supports VT or not. If you receive output your CPU is supported, else it is not. If your CPU is supported also check that VT is enabled in your system bios.

Load the kvm modules matching your CPU, either "modprobe kvm_intel" or "modprobe kvm_amd", install kvm user tools and setup permissions so users may run kvm (Have a look at a HOWTO for your distro for details: Ubuntu, Gentoo).

Once you have setup everything you can start kvm from the commandline (I think there are GUI frontends too, but I'm using the command line). You should be carefull though, acpi in JNode seems to kill kvm, so allways disable acpi. I also had to deactivate the kvm-irqchip as it trashed JNode. The command that works for me is:

kvm -m 768 -cdrom jnode-x86-lite.iso -no-acpi -no-kvm-irqchip -serial stdout -net none -std-vga

The "-serial" switch is optional but I need it for kdb (kernel debugger). If you want to use the Vesa mode of JNode you should also use "-std-vga", overwise you will not have a vesa mode. Set the memory whatever you like (768MB is my default).

Things that still need to be tested:

Test gdb with kvm. This should be more fun than using qemu

Running in parallels

I found an only way to run JNode with parallels.

In Options->Emulation flags, there is a parameter called Acceleration level that takes 3 values :
- disabled : JNode will work but that's very slow
- normal : JNode won't boot (freeze at "Detected 1 processor")
- high : JNode won't boot (freeze at "Detected 1 processor")

Running in VBox

You can now run JNode on virtual box too. ACPI is not working but you'll get a prompt and can use JNode.

TODO: Test network, usb,...

Running in Virtual PC

This page will discripe how to run JNode in Virtual PC

At the current state JNode doesn't run in Virtual PC.

Running in VMWare

Basic Procedure

The JNode build process creates a VMWare compatible ".vmx" file that allows you to run JNode using recent releases of VMWare Player.

Assuming that you build JNode using the "cd-x86-lite" target, the build will create an ISO format CDROM image called "jnode-x86-lite.iso" in the "all/build/cdroms/" directory. In the same directory, the build also generates a file called "jnode-x86-lite.iso.vmx" for use with VMWare. To boot JNode from this ".iso" file, simply run the player as follows:

  $ vmplayer all/build/cdroms/jnode-x86-lite.iso.vmx

Altering the VMWare virtual machine configuration

By default, the generated ".vmx" file configures a virtual machine with a virtual CDROM for the ISO, a bridged virtual ethernet and a virtual serial port that is connected to a "debugger.txt" file. If you want to configure the VMWare virtual machine differently, the simplest option is to buy VMWare Workstation and use its graphical interfaces to configure and run JNode. (Copy the generated ".vmx" file to a different location, and use that as the starting point.)

If you don't want to pay for VMWare Workstation, you can achieve the same effect by changing the ".vmx" file by hand. However, changes you make that way will be clobbered next time you run the "build" script. The solution is to do the following:

This procedure assumes some changes in a patch that is waiting to be committed.

1. Create a file containing the VMX properties that you want to add or replace. Put this file somewhere that won't be clobbered by "build clean".
2. Copy the "jnode.properties.dist" file to "jnode.properties".
3. Open the "jnode.properies" file in a text editor.
4. Find the line that starts "#vmware.vmx.overrides=".
5. Remove the "#" and replace the characters after the "=" with the override file's pathname.
6. Save the file.
7. Run "build cd-x86-lite" (or whatever you normally use to build a CDROM image).

This should create the "jnode-x86-lite.iso.vmx" file with the VMX settings from your file as overrides to the default settings.

Unfortunately, VMWare have not released any documentation for the myriad VMX settings. The best 3rd-party documentation that I can find is the sanbarrow.com website. There are also various "builder" applications around, but they don't look all that good.

VMWare disks and Boot device order
If you add a VMWare virtual (or real) disk drive, the VMWare virtual machine's BIOS will try to boot from that drive. Unless you have set up the drive to be bootable, this won't work. The easy way to fix this is to change VMWare's BIOS settings to alter the boot device order.

1. Run vmplayer as above.
2. Quickly click the VMWare window to give it keyboard focus and hit [F2]. (You have small window to do this ... just a second or two!).
3. In the BIOS settings screen, select the "Booting" panel and move the "cdrom" to the top; i.e. the first device to be tried.
4. Save the BIOS settings to the virtual NVRAM and exit the BIOS settings editor to continue the boot.

By default the NVRAM settings are stored in the "JNode.nvram" file in "all/build/cdroms" directory, and will be clobbered when you run "build clean". If this is a problem, create a VMX override (see above) with a "nvram" entry that uses a different location for the file.

Running on a PC

To run JNode on a PC using the bootable CDROM, your PC must comply with the following specifications:

• Pentium or above CPU
  128Mb RAM or more (i'm not sure what the minimum is)
• Bootable CDROM drive that supports El-Torito harddisk emulation
• VGA compatible video card
• Keyboard

The GRUB Boot Menu

The first JNode related information you will see after booting from a JNode CDROM image is the GRUB bootloader page. The GNU GRUB bootloader is responsible for selecting a JNode configuration from a menu, loading the corresponding kernel image and parameters into RAM and causing the JNode kernel to start executing.

When GRUB starts, it displays the bootloader page and pauses for a few seconds. If you do nothing, GRUB will automatically load and start the default configuration. Pressing any key at this point will interrupt the automatic boot sequence and allow you to select a specific configuration. You can use the UP-ARROW and DOWN-ARROW to choose a JNode configuration, then hit ENTER to boot it up.

There are a number of JNode configurations in the menu:

• The configurations that include "MP" in the description enable JNode's multiprocessor support on a capable machine. Since JNode's multiprocessor support is not fully functional, you should probably avoid these configurations for now.
• The configurations the include "GUI" in the description will start up the JNode GUI. Configurations without this tag will start up the JNode command shell on a text console.
• The other difference between the configurations is in the sets of plugins that they load. In general, loading more plugins will cause JNode to use more RAM.

It is currently not a good idea to boot JNode straight to the GUI. If want to run the GUI, it is best to boot the one of the non-GUI configurations; typically "JNode (all plugins)". Then from the text console, run the following commands:

JNode /> gc

JNode /> startawt

acpi

acpi

alias

alias

arp

arp

basename

basename

beep

beep

bindkeys

bindkeys

bootp

bootp

bsh

bsh

bzip2

bzip2

cat

cat

cd

cd

class

class

classpath

classpath

clear

clear

compile

compile

console

console

cpuid

cpuid

date

date

del

del

device

device

df

df

dhcp

dhcp

dir

dir

dirname

dirname

disasm

disasm

echo

echo

edit

edit

eject

eject

env

env

exit

exit

gc

gc

grep

grep

gzip

gzip

halt

reboot

help

help

hexdump

hexdump

history

history

ifconfig

ifconfig

java

java

kdb

kdb

leed, levi

leed & levi

loadkeys

loadkeys

locale

locale

log4j

console

ls

ls

lsirq

lsirq

man

man

memory

locale

mkdir

mkdir

mount

mount

namespace

locale

netstat

netstat

onheap

onheap

page

page

ping

ping

plugin

plugin

propset

propset

pwd

pwd

ramdisk

ramdisk

reboot

reboot

remoteout

remoteout

resolver

resolver

route

route

rpcinfo

rpcinfo

run

run

#!<interpreter>

startawt

startawt

syntax

syntax

tar

tar

tcpinout

tcpinout

thread

thread

time

time

touch

cd

unzip

unzip

utest

utest

vminfo

vminfo

wc

wc

zip

zip

Testing remote programs in shell

If you want to test some java application, but don't want to recompile JNode completly every time you change your application, you can use the classpath command.

Set up your network, if you don't know how, read the FAQ.

Now you have to setup a webserver or tftp server on your remote mashine, where you place your .class or .jar files.
With the classpath command you can now add a remote path. E.g. "classpath add http://192.168.0.1/path/to/classes/". Using "classpath" without arguments shows you the list of added paths. To start your application simply type the class file's name.

For more info read the original forum topic from Ewout, read more about shell commands or have a look at the following example:

On your PC:
Install a Webserver (e.g. Apache) and start it up. Let's say it has 192.168.0.1 as its IP. Now create a HelloWorld.java, compile it and place the HelloWorld.class in a directory of your Webserver, for me that is "/var/www/localhost/htdocs/jnode/".

Inside JNode:
Type the following lines inside JNode. You just have to replace IP addesses and network device by values matching your configuration.

ifconfig eth-pci(0,17,0) 192.168.0.6
route add 192.168.0.1 eth-pci(0,17,0)
classpath add http://192.168.0.1/jnode/

now that a new classpath is added you can run your HelloWorld App by simply typing

HelloWorld

Changelog

Changes from JNode 0.2.8 to current SVN trunc version

Features
========
progress with OpenJDK integration
class library updated to OpenJDK6 b13
JNode now builds  with both Java 6 SE and OpenJDK6+IcedTea
javac source level and target level raised to 1.6
introduced mauve based regression testing
improved isolate support
added isolate invoker
added Russian keyboard support
improved NTFS support
added HFS+ formatter
progress with Bjorne shell
improved modal dialogs
console & shell improvements
a large number of bug fixes and improvements in the overall system
aiming better Java compatibility, stability and performance
real world applications starting to work: Jetty + recent Servlet/JSP examples, 
   PHP with Jetty + Quercus, JEdit, Groovy


Contributors to this release
============================
Levente Sántha
Fabien Duminy
Peter Barth
Martin Husted Hartvig
Stephen Crawley
Fabien Lesire
Daniel Noll
Tim Sparg
Stephen Meslin-Weber
Sergey Mashkov
Ben Bucksch

J2SDK 5.0 feature support

This page gives an overview of the support for J2SDK 5.0 features.
It reflects the status of the SVN trunk.

Feature	Status	Can be used
Generics	Supported
Generics in collection framework	Supported
Enhanced for loop	Supported
Autoboxing/unboxing	Supported
Typesafe enums	Supported
Varargs	Supported
Static import	Supported
Metadata (annotations)	Supported
Covariant return types	Supported

SVN Usage (Deprecated : we have moved to GitHub)

This page is deprecated since we have moved to GitHub

This is a slight overview of SVN and how to use it. First of all there are three ways to access SVN: svn, svn+ssh and via https. Sourceforge uses WebDAV, that means you can also browse the repository online with your favorite browser, just click on this link.

Subversion uses three toplevel directories named trunk, branches and tags. Trunk can be compared to CVS Head, branches and tags are self-explanatory, I think

To checkout the source simply type:
svn co https://jnode.svn.sourceforge.net/svnroot/jnode/trunk/ jnode
which creates a new directory called jnode (Mind the space between "trunk/" and "jnode"!) . In this directory all stuff of the repository in /jnode/trunk will be copied to jnode/ on your local computer.

svn up|add|commit as expected.

New to subversion is copy, move and delete. If you copy or move a file, the history is also copied or moved, if you e.g. delete a directory it will not show up anymore if you do a new checkout.

If you want to make a branch from a version currently in trunk you can simply copy the content from trunk/ to branches/, e.g. by:
svn copy https://jnode.svn.sourceforge.net/svnroot/jnode/trunk/ https://jnode.svn.sourceforge.net/svnroot/jnode/branches/my-big-change-branch/

I think that's the most important for the moment, for more information have a look at the SVN Handbook located here.

Btw, for using SVN within eclipse you have to install subclipse located here.

Using Git (at GitHub)

The URLs for official git repository at GitHub are listed below :

Site: https://github.com/jnode/jnode
Https: https://github.com/jnode/jnode.git
SSH: git@github.com:jnode/jnode.git

For those who know what they are doing already and simply want push access, refer to the page on setting up push access. For those that are unfamiliar with git, there are a few git pages below that explain some of the common tasks of setting up and using git. This of course is not meant to be a replacement for the git manual.

Git Manual: http://www.kernel.org/pub/software/scm/git/docs/user-manual.html
Git Crash Course for SVN users: http://git.or.cz/course/svn.html

ssh-keygen

git config remote.origin.url git@github.com:[user]/jnode.git

cd ~/git
git clone git@github.com:jnode/jnode.git jnode

git rebase origin master

git rebase master [branch]

git push origin [branch]

git config user.name [name]
git config user.email [email]

The Configure tool

The Configure tool is a Java application that is designed to be run in the build environment to capture and record JNode's build-time configuration settings. The first generation of this tool is a simple command-line application that asks the user questions according to an XML "script" file and captures and checks the responses, and records them in property files and other kinds of file.

The Configuration tool supports the following features:

• Property types are specified in terms of regexes or value enumerations.
• Properties are specified in terms of a property name and type, with an optional default value.
• Property sets are collections of properties associated with files:
  • They are typically loaded from the file, updated by the tool and written back to the file.
  • Properties can be expanded into templates XML and Java source files as well as classic Java properties files.
  • A "FileAdapter" API allows new file formats to be added as plugin classes.
• Property values are captured in "screens" consists of a sequence of "items" which define the questions that are presented to the user.
  • Each "item" consists of a property name and a multi-line "text" that explains the property to the user.
  • A screen can be made conditional, with a "guard" property that determines whether or not the properties in the screen are captured.
• A configuration script file can "import" other script files, allowing the configuration process to modularised. All relative pathnames in scripts are resolved relative to the script file that specifies them.

The configuration tool is launched using the "configure.sh" script:

   $ ./configure.sh

When run with no command arguments as above, the script launches the tool using the configuration script at "all/conf-source/script.xml". The full command-line syntax is as follows:

   ./configure.sh
   ./configure.sh --help
   ./configure.sh [--verbose] [--debug] <script-file>

The command creates and/or updates various configuration files, depending on what the script says. Before a file is updated, a backup copy is created by renaming the existing file with a ".bak" suffix.

Configuration script files

The Configure tool uses a "script" to tell it what configuration options to capture, how to capture them and where to put them. Here is a simple example illustrating the basic structure of a script file:

<configureScript>
  <type name="integer.type" pattern="[0-9]+"/>
  <type name="yesno.type">
    <alt value="yes"/>
    <alt value="no"/>
  </type>

  <propFile name="test.properties">
    <property name="prop1" type="integer.type"
              description="Enter an integer"
              default="0"/>
    <property name="prop2" type="yesno.type"
              description="Do you want to?"
              default="no"/>
  </propFile>

  <screen title="Testing set 1">
    <item property="prop1"/>
    <item property="prop2"/>
  </screen>
</configureScript>

The main elements of a script are "types", "property sets" and "screens". Lets describe these in that order.

A "type" element introduces a property type which defines a set of allowed values for properties specified later in the script file. A property type's value set can be defined using a regular expression (pattern) or by listing the value set. For more details refer to the "Specifying property types" page.

A "propFile" element introduces a property set consisting of the properties to be written to a given property file. Each property in the property set is specified in terms of a property name and a previously defined type, together with a (one line) description and an optional default value. For more details refer to the "Specifying property files" page.

A "screen" element defines the dialog sequence that is used to request configuration properties from the user. The screen consists of a list of properties, together with (multi-line) explanations to be displayed to the user. For more details refer to the "Specifying property screens" page.

Finally, the "Advanced features" page describes the control properties and the import mechanism.

  <type name="integer.type" pattern="[0-9]+"/>
  <type name="yesno.type">
    <alt value="yes"/>
    <alt value="no"/>
  </type>

  <type name="optinteger.type" 
        pattern="[0-9]*" emptyToken="none"/>

  <type name="yesno.type">
    <alt token="oui" value="yes"/>
    <alt token="non" value="no"/>
  </type>

  <propFile fileName="jnode.properties">
    <property name="jnode.vm.size"
                 type="integer.type"
                 description="Enter VM size in Mbytes"
                 default="512"/>
    <property name="jnode.vdisk.enabled" type="yesNo.type"
              description="Configure a virtual disk"
              default="no"/>
  </propFile<

# This file contains some interesting properties

# The following property is interesting
interesting=@interesting@

# The following property is not at all interesting
boring=@boring@

    at_sequence ::= '@' name [ '/' modifiers ] '@'
    name        ::= ... # any valid property name
    modifiers   ::= ... # one or more modifier chars

  <screen title="Main JNode Build Settings">
    <item property="jnode.virt.platform">
The JNode build can generate config files for use with
various virtualization products.
    </item>
    <item property="expert.mode">
Some JNode build settings should only be used by experts.
    </item>
  </screen>

  <screen title="Virtualization Platform Settings"
          guardProp="jnode.virt.platform" valueIsNot="none">
    <item property="jnode.vm.size">
You can specify the memory size for the virtual PC.  
We recommended a memory size of least 512 Mbytes.
    </item>
    <item property="jnode.virtual.disk">
Select a disk image to be mounted as a virtual hard drive.
    </item>
  </screen>

Configuration files

The primary JNode build configuration file is the "jnode.properties" file in the project root directory.

Other important configuration files are the plugin lists. These specify the list plugins that make up the JNode boot image and the lists that are available for demand loading in various the Grub boot configurations.

Build & development environment

This chapter details the environment needed to setup a JNode development environment.

Sub-Projects

JNode has been divided into several sub-projects in order to keep it "accessible". These sub-projects are:

Each sub-project has the same directory structure:

Eclipse

JNode is usually developed in Eclipse. (It can be done without)
The various sub-projects must be imported into eclipse. Since they reference each other, it is advisably to import them in the following order:

1. core
2. shell
3. fs
4. gui
5. net
6. builder
7. distr
8. all
9. sound
10. textui
11. cli

For a more details please have a look at this Howto.

IntelliJ IDEA

JetBrains Inc has donated a Open Source License for Intellij IDEA to the dedicated developers working on JNode.

Developers can get a license by contacting Martin.
Setup of the sub-projects is done with using the modules feature like with Eclipse.

One should increase the max memory used in the bin/idea.exe.vmoptions or bin/idea.sh.vmoptions file, edit the -Xmx line to about 350mb. IntelliJ can be downloaded at http://www.jetbrains.com/idea/download/ Use at least version 5.1.1. Note that this version can import Eclipse projects.

Requirements for building under Windows

1. Make sure that you have a Sun JDK for Java 1.6.0 at or near the most recent patch level. (Some older patch levels are known to cause obscure problems with JNode builds.)
2. Make sure that the pathname for the root directory your JNode tree contains no spaces. (Spaces in the pathname are likely to break the build.)
3. Create a "bin" directory to holds some utilities; see below.
4. Use the "System" control panel to add the "bin" directory to your windows %PATH%.
5. Download the "nasm" assembler from http://nasm.sourceforge.net. (Make sure that you get the Win32 version not the DOS32 version!)
6. Open the downloaded ZIP file, and copy the "nasm.exe" file to your "bin" directory. Then rename it to "nasmw.exe".

Now, can start a Windows command prompt, change directory to the JNode root, and build JNode as explained the next section.

Requirements for building under Linux

1. Make sure that you have a Sun JDK for Java 1.6.0 at or near the most recent patch level. (Some older patch levels are known to cause obscure problems with JNode builds.)
2. Make sure that the 'nasm' assembler is installed. If not, use "System>Add/Remove Software" (or your system's equivalent) to install it.

Building

Running "build.sh" or "build.bat" with no arguments to list the available build targets. Then choose the target that best matches your target environment / platform.

Alternatively, from within Eclipse, execute the "all" target of all/build.xml. Building in Eclipse is not advised for Eclipse version 2.x because of the amount of memory the build process takes. From Eclipse 3.x make sure to use Ant in an external process.

A JNode build will typically generate in the following files:

Some builds also generate an ISO image which you can burn to disk, and then use to boot into JNode from a CD / DVD drive.

Building on the Mac OSX / Intel platform

(These instructions were contributed by "jarrah".)

I've successfully built jnode on MacOS X from the trunk and the 2.6 sources. Here's what I needed to do:

Using OSX and PowerPC for JNode development and testing

Using OSX and PPC for JNode development and testing

What we want is:
1. CVS tool
2. IDE for development
3. A way to build JNode
4. A way to boot JNode for testing

First of all we need to install the XCode tools from apple. Usually it is shipped with your OSX, look in /Applications/Installers/. If it is not there you, you can download it from apple’s site.

1. CVS tool
Well cvs is already in the OSX installation. There are some GUI tools to make the use of cvs easier. SmartCVS is a good one, which you can use it in your windows/PC computer, or linux etc.

2. IDE
Eclipse.

3. How to build JNode with a ppc machine (not FOR, WITH ppc)
Good for us, JNode build process is based on apache ant, which as a java tool runs everywhere. The only problem is the native assembly parts of JNode. For them JNode build process uses nasm and yasm.

So the only thing we need is to build them for ppc and use them. They will still make x86 binaries as they are written to do.

First of all we have to get the nasm and yasm sources. The first one is on
http://nasm.sourceforge.net
and the other is on
http://www.tortall.net/projects/yasm/

After that we unzip them and start the compile.

NASM
Open a terminal window and go inside the directory with the nasm sources

Run ./configure to create the Makefile for nasm

If everything is ok you now are ready to compile nasm. Just run ‘’make nasm‘’. Maybe there will be a problem if you try to compile all the nasm tools by running ‘’make’’ (I had), but you dont need them. Nasm is enought.

Now copy nasm in your path. /usr/bin is a good place.

YASM
The same as for nasm open a terminal window and go to the directory with yasm sources.

Run ‘’./configure’’

Run ‘’make’’

Now you can either copy yasm to /usr/bin or run ‘’make install’’ which will install the yasm tools under /usr/local/bin.

That’s all with nasm and yasm. You are ready to build JNode. You may have problems using the buildl.sh script, but you can always run the build command manually ‘’java -Xmx512M -Xms128M -jar core/lib/ant-launcher.jar -lib core/lib/ -lib /usr/lib/java/lib -f all/build.xml cd-x86’’

4. Booting JNode
Well there is only one way to do that. Emulation.

There is VirtualPC for OSX, which is pretty good and fast. To use it just create a new virtual PC and start it. When the virtual PC is started right click on the CD-Rom icon at the bottom of the window (hmm I know there is no right click on macs I assume you know to press ctrl+click). Now tell the VirtualPC to use the JNode iso image as cdrom drive and boot from it. There you are!

I think there is also qemu for ppc. I have not ever used it, so I don’t know how you can configure it.

Directory structure

The JNode sources are divided into the following groups:

• all
  Contains the global libraries, the (Ant) build files and some configuration files. This group does not contain java sources.
• builder
  Contains the java source code used to build JNode. This includes several Ant tasks, but also code used to link Elf files and to write the JNode bootimage.
• core
  Contains the JNode virtual machine code (both java and assembler), the classpath java library sources and the core of the JNode operating system, including the plugin manager, the driver framework, the resource manager and the security manager. This is by far the largest and most complex group.
• distr
  Contains the first parts of the JNode distribution. This includes an installation program and various applications.
• fs
  Contains the file system framework, the various file system implementation and the block drivers such as the IDE driver, harddisk driver, CD-ROM etc.
• gui
  Contains the JNode gui implementation. This includes the graphics drivers, the AWT peer implementation, font renderers and the JNode desktop.
• net
  Contains the JNode network layer. This includes the network drivers, the network framework, the TCP/IP stack and the connection between the network layer and the java.net package.
• shell
  Contains the JNode command shell and several system shell commands.
• textui
  Contains a copy of the charva text based AWT implementation.
• cli
  Contains the bulk of JNode's commands.

Every group is a directory below the root of the JNode CVS archive. Every group contains one or more standard directories.

• build
  This directory is created during the build and contains the intermediate build results.
• descriptors
  This directory contains the plugin descriptors of the plugins defined in this group.
• lib
  This directory contains libraries (jar files) required only by this group. An exception is the All group, for which the lib directory contains libraries used by all groups.
• src
  This directory contains the source files. Below this directory there are one or more source directories (source folders in Eclipse) containing the actual source trees.

JNode coding DOs and DONTs

This page lists some tips on how to write good JNode code.

Please add other tips as required.

Avoid using System.in, System.out, System.err

Where possible, avoid using these three variables. The problem is that they are global to the current isolate, and are not necessarily connected to the place that you expect them to be.

In a user-level command should use streams provided by the Command API; e.g. by calling the 'getInput()' method from within the command's 'execute' method. Device drivers, services and so on that do not have access to these streams should use log4j logging.

Avoid cluttering up the console with obscure or unnecessary logging

If a message is important enough to be written to the console, it should be self explanatory. If it is unimportant, it should be logged using log4j at an appropriate level. We really do not need to see console messages left over from someone's attempts to debug something 12 months ago ...

Avoid using Unsafe.debug(...) methods

The org.jnode.vm.Unsafe.debug(...) methods write to the VGA screen and (when kernel debug is enabled) to the serial port. This is ugly, and should be reserved for important early boot sequence logging, VM and GC debugging, and other situations where log4j logging cannot be used.

Don't call 'Throwable.printStackTrace' and friends

Commands should allow unexpected exceptions to propagate to the shell level where they are handled according to the user's setting of the 'jnode.debug' property. (The alternative of adding a "--debug" flag to each command, is a bad idea. It is a lot of work and will tend to lead to inconsistencies of implementation; e.g. commands that don't implement "--debug", send "--debug" output to unexpected places, overload other functionality on the flag, etcetera.)

Services, etc should make appropriate log4j calls, passing the offending Throwable as an argument.

Do use JNode's Command and syntax.* APIs for commands

Commands that are implemented using the Command and syntax.* APIs support completion and help, and are more likely to behave "normally"; e.g. with respect to stream redirection. There are lots of examples in the codebase.

If the APIs don't do what you want, raise an issue. Bear in mind that some requests may be deemed to be "to hard", or to application specific.

Do include a 'main' entry point.

It is a good idea to include a legacy "public static void main(String[])" entry point in each JNode command. This is currently only used by the old "default" command invoker, but in the future it may be used to run the command in a classic JVM.

Don't use '\n\r' or '\r\n" in output.

The line separator on JNode is '\n', like on UNIX / Linux. If you expect your command to only run on JNode, it is not unreasonable to hardwire '\n' in output messages, etc. But if you want your command to be portable, it should use 'System.getProperty("line.separator")', or one of the PrintWriter / PrintStream's 'println' methods.

JNode Java style rules

All code that is developed as part of the JNode project must conform to the style set out in Sun's "Java Style Guidelines" (JSG) with variations and exceptions listed below. Javadocs are also important, so please try to make an effort to make them accurate and comprehensive.

Note that we use CheckStyle 4.4 as our arbiter for Java style correctness. Run "./build.sh checkstyle" to check your code style before checking it in or submitting it as a patch. UPDATE: And also, please run "./build.sh javadoc" to make sure that you haven't introduced any new javadoc warnings.

No TAB characters.

No TAB characters (HT or VT) are allowed in JNode Java source code.
Whitespace TABs should be replaced with the requisite number of spaces or newlines. Non-whitespace TABs (i.e. in Java strings) should be replaced with "\t" or "\f" escape sequences.

Use 4 space indentation.

Two or three characters is too little, eight is too much.

Maximum line width 120.

The JSG recommends 80 characters, but most people use tools that can cope with much wider.

Put following keywords on same line as }

For example:

    try {
        if (condition) {
            something();
        } else {
            somethingElse();
        }
    } catch (Exception ex) {
        return 42;
    }

Note that the else is on the same line as the preceding }, as is the catch.

Indent labels by -4.

For example:

public void loopy() {
    int i;
LOOP:
    for (i = 100; i < 1000000; i++) {
        if (isPrime(i) && isPrime(i + 3) && isPrime(i + 5) {
            break LOOP;
        }
    }
}

No empty { } blocks

A { } block with no code should contain a comment to say why the block is empty. For example,

    try {
        myStream.close();
    } catch (IOException ex) {
        // we can safely ignore this
    }

No marker comments

It is generally accepted that marker comments (like the following) add little to the readability of a program. In fact, most programmers think they are an eyesore and a waste of space.

    //*******************************************************
    //
    // Start of private methods
    //
    //*******************************************************

Avoid copying javadoc

Instead of copying the javadoc from the parent class or method, use the {@inheritDoc} tag and if needed add some specific javadoc.

    /**
     * {@inheritDoc}
     */    
    public void myMethod(String param1, int param2) {
    }

Give your references in the javadoc

When you are implementing a class from a reference document, add a link in the javadoc.

Packages

The java classes of JNode are organized using the following package structure.
Note that not all packages are listed, but only the most important. For a full list, refer to the javadoc documentation.

All packages start with org.jnode.

Common packages

• org.jnode.boot
  Contains the first classes that run once JNode is booted. These classes initialize the virtual machine and start the operating system.
• org.jnode.plugin
  Contains the interfaces of the plugin manager.
• org.jnode.util
  Contains frequently used utility classes.
• org.jnode.protocol
  Contains the protocol handlers for the various (URL) protocols implemented by JNode. Every protocol maps onto a package below this package, e.g. the plugin protocol handler is implemented in org.jnode.protocol.plugin.

JNode virtual machine

• org.jnode.vm
  Contains the core classes of the JNode virtual machine.
• org.jnode.vm.classmgr
  Contains the internal classes that represent java classes, methods & field. It also contains the classfile decoder.
• org.jnode.vm.compiler
  Contains the base classes for the native code compilers that convert java bytecodes into native code for a specific platform.
• org.jnode.vm.memmgr
  Contains the java heap manager, including the object allocator and the garbage collector.
• org.jnode.vm.<arch>
  For every architecture that is supported by JNode a seperate package exists, that contains the architecture dependent classes, including classes for threads and processors and classes for the native code compilation.

JNode operating system

• org.jnode.driver
  Contains the driver framework.
  All drivers and driver API's have a seperate package below this package. Drivers of a similar type are grouped, e.g. all video drivers have a package below org.jnode.driver.video.

• org.jnode.system
  Contains the interfaces for the various low level resources in the system, such as memory regions, I/O port regions, DMA access.
• org.jnode.fs
  Contains the filesystem framework.
  All file systems have a seperate package below this package, e.g. the EXT2 filesystem implementation is contained in the org.jnode.fs.ext2 package and its sub-packages.
• org.jnode.net
  Contains the network layer.
  All network protocols have a seperate package below this package, e.g. the IPv4 protocol and its sub-protocols is contained in the org.jnode.net.ipv4 package and its sub-packages.
• org.jnode.shell
  Contains the command shell.
  All system commands are grouped in packages below this package.

Special packages

There are some packages that do not comply to the rule that all packages start with org.jnode. These are:

• java.*, javax.*
  Contains the classpath implementation of the standard java libraries.

• gnu.*
  Contains implementation classes of the the classpath library.
• org.vmmagic.pragma
  Contains exception classes and interfaces that have special meaning to the virtual machine and especially the native code compilers. These classes are mostly shared with the Jikes RVM
• org.vmmagic.unboxed
  Contains non-normal classes that are used as pointers to raw memory, object references and architecture dependent integers (words). These classes have a special meaning to the virtual machine and especially the native code compilers and should never be instantiated or used without a good knowledge of their meaning. These classes are mostly shared with the Jikes RVM

Source file requirements

Header

All java source files must contain the standard JNode header found in <jnode>/all/template/header.txt.

Do not add extra information to the header, since this header is updated automatically, at which time these extra pieces of information are lots.

Add any extra information about the class to the classes javadoc comment. If you make significant contribution to a class, feel free to add yourself as an @author. However, adding a personal copyright notice is "bad form", and unnecessary from the standpoint of copyright law. (If you are not comfortable with this, please don't contribute code to the project.)

Encoding

All Java source files and other text-based files in the JNode project must be US-ASCII encoded. This means that extended characters in Java code must be encoded in the '\uxxxx' form. Lines should end with an ASCII linefeed (LF) character, not CR LF or LF CR, and hard tab (HT) characters should not be used.

If there is a pressing need to break these rules in some configuration or regression data file, we can make an exception. However, it is advisable to highlight the use of "nasty" characters (e.g. as comments in the file) so that someone doesn't accidentally "fix" them.

How to add a plugin to JNode

This page will describe how to add a java program to JNode as plugin, so that it can be called via its alias.

First of all you need to set up Eclipse (or your favorit IDE) as described in the readme, so that JNode builds without errors and you can use it (e.g. use JNode in VMWare).

There are different ways of extending JNode with a plugin.
A plugin can contain a class that extends Plugin and (or) normal java programs.
Every plugin is described by a descriptor.

For our example we will develop a plugin that contains a normal java program.

We need a name for our plugin : we will use sample, wich is also the packagename of our plugin.
It belongs to one of the JNodes subprojects in our case we will use the ordername sample in the shell subproject.

Every java-file for our plugin has to be in (or in subfolders):



\shell\src\shell\org\jnode\shell\sample

(for me it is d:\jnode\shell\src\shell\org\jnode\shell\sample)



Now we will write a small HelloWorld.java wich will be one of our plugin programs.
Here is the source of the file HelloWorld.java :

package org.jnode.shell.sample;

public class HelloWorld{

        public static void main(String[] args){

            System.out.println(“HelloWorld – trickkiste@gmx.de“);

        }

}

thats ok, but it will not be build until we create a descriptor and add our plugin to the JNode full-plugin-list.xml.



The plugin descriptor (org.jnode.shell.sample.xml stored in the descriptors folder of the shell subproject) and looks like this :

<?xml version="1.0" encoding="UTF-8"? >

<!DOCTYPE plugin SYSTEM "jnode.dtd">


<plugin id="org.jnode.shell.sample"

            name="Sample Plugin"

            version="0.2"

            license-name="lgpl"

            provider-name="Trickkiste">

<requires>

            <import plugin="org.jnode.shell"/>

</requires>

<runtime>

            <library name="jnode-shell.jar">

                        <export name="org.jnode.shell.sample.*"/>

            </library>

</runtime>

<extension point="org.jnode.shell.aliases">

            <alias name="HelloWorld" class="org.jnode.shell.sample.HelloWorld"/>

</extension>

</plugin>

Now we need to add our Plugin to the JNode full-plugin-list.xml, this file is located in jnode\all\conf your entry should look like this :

[...]

            <plugin id="org.jnode.util"/>

            <plugin id="org.jnode.vm"/>

            <plugin id="org.jnode.vm.core"/>

            <plugin id="org.jnode.shell.sample"/>

</plugin-list>

thats it, you can now build JNode and test your HelloWorld plugin by typing HelloWorld.

What we can do now is add „normal“ programs to JNode via its provided Pluginstructure.

Arguments - The Basics

In JNode's command line interface, the Argument types are the Command programmers main tool for interacting with the user. The Argument provides support to the syntax mechanism to accept parameters, or reject malformed parameters, issuing a useful error message. The argument also supplies completion support, allowing a command to provide specific completions on specific domains.

Organization

At the moment, Arguments are mostly grouped into the shell project under the org.jnode.shell.syntax package. For the time being they will remain here. There is an effort being made to 'untangle' the syntax/argument APIs so this designation is subject to change in the future.

New arguments that are created should be placed into the cli project under the org.jnode.command.argument package if their only use is by the commands under the cli project.

How it works

Every command that accepts an option will require Arguments to capture the options and their associated values. The syntax parser makes use of an argument 'label' to map a syntax node to a specific argument. The parser then asks the Argument to 'accept' the given value. The argument may reject the token if it doesn't not satisfy it's requirements, and provide a suitable error message as to why. If it accepts the token, then it will be captured by the argument for later use by it's command.

Arguments also provide the ability to 'complete' a partial token. In some situations completions are not possible or do not make sense, but in many situations completions can be very helpful and save on typing, reduce errors, and even provide a little help if there are alot of options. The more characters there are in the token, the narrower the list of completions becomes. If the argument supplies only a single completion, this completion will be filled in for the user. This is a very powerful capability that can be used to great effect!

Using arguments

Before writing a command, it is important to consult the various specifications that many commands may have. Once you have an idea of the arguments you will need for the command, and you have a syntax put together, you can begin by adding your arguments to the command.

Along with the label that was discussed earlier, commands also take a set of flags. A set of flags is supplied by the Argument class, but individual Argument types may also supply their own specific flags. At the end of this document will be a list of known flags and their purpose, but for now we will discuss the common Argument flags.

SINGLE and MULTIPLE

By default arguments are 'SINGLE'. This means that the argument may only contain one value. In order to change this, and allow the argument to capture multiple values, you must set the MULTIPLE flag.

OPTIONAL and MANDATORY

By default arguments are 'OPTIONAL'. This means that if the argument is not populated with any values, it will not be considered an error. In order to have the parser fail if the argument is not populated with at least one value, set the MANDATORY flag.

EXISTING and NONEXISTENT

These flags are not used by all arguments and may be used to alter the behavior of 'accept' and 'complete' depending on the argument, and their values. As an example, the FileArgument by default will accept most any string that denotes a legal file name. In order to force it to only accept tokens that denote an existing file on the file system than set the EXISTING flag. In order to force it to only accept tokens that denote a file that does not exist already, set the NONEXISTENT flag.

Most arguments have overloaded constructors that allow you to not set any flags. If no such constructor exists, then feel free to create one! Optionally, it is safe to provide '0'(zero) for the flags parameter to mean no flags.

Once you have created the arguments that your command will need, you need to 'register' the arguments. This needs to be done in the Command constructor. Each argument needs to be passed to the registerArguments(Argument...) method. Once this is done, your arguments are ready to be populated the syntax parser.

(Note: Arguments that have been registered, but do not have a matching syntax node with its label will not cause an error at runtime. But they do make trouble for the 'help' command. For this reason it is recommended to not register arguments that have not yet been mapped in the syntax.)

Using arguments

When your command enters at the execute() method, the arguments will be populated with any values that were capture from the command line. For the most part, you will only need to be concerned with three methods supplied by Argument.

public boolean isSet()

If the argument has accepted and captured a token, then this method will return true. Commands should always check this method before querying for the captured values. If you query an argument for its values when it has none, the behavior is undefined and the return value (or possible exception) is unspecified and subject to change without notice. (The one case where this is not totally true is when an argument has the MANDATORY flag, as in this case this will _always_ return true. Though it is still considered 'good practice' to check this method before querying for values)

public V getValue()

This method returns the single value of an argument that was registered as SINGLE. If the argument has the MULTIPLE flag set, this method should not be used as it will throw an exception if there is more than one value captured by the argument. If there are no values captured, this currently returns null, but as noted earlier, this may not always be the case, and should not be relied upon.

public V[] getValues()

This method returns the captured values as an array. Calling this method when there the SINGLE flag is set is perfectly acceptable. Though it is usually more convenient to use the getValue() method.

Thats about it for arguments. Simple huh? Arguments are designed to allow for rapid development of commands and as such provide a nice simple interface for using arguments 'out of the box' so to speak. But the real power of arguments are their ability to be extended and manipulated in many ways so as to provide a more feature filled command line interface.

Basic argument types

Here are a list of the more common argument types, along with a short description on their purpose, features and usage.

AliasArgument

An argument that accepts an 'alias'. It provides completion against those aliases that have been registered via plugin descriptors, and may also include the bjorne-style aliases if the bjorne interpreter is in use. (I'm not sure if it currently does, if not it should!)

FileArgument

An argument that accepts a java.io.File. As used in an example above, this argument is affected by the EXISTING and NONEXISTENT flags of Argument. FileArgument also currently provides two of its own flags that may be set. ALLOW_DODGY_NAMES is used to override the 'accept' and 'complete' features to allow filenames that begin with a '-'(hyphen). Normally FileArgument would consider such a filename to be an error, and reject such a token. There is also the HYPHEN_IS_SPECIAL flag, which allows a single '-' to be accepted. The purpose for this is to allow '-' to exist amongst a list of files, denoting stardnard input or output should be used instead. This feature is subject to change (pending some 'better way' of handling this).

FlagArgument

This is likely to be the most used argument of all. It is used to denote a option that has no associated argument. This argument does not actually capture a token, instead it holds a single value of true if it has been found. This means that you can use isSet() to map its value to a local/instance boolean value. In some cases, a command may wish to allow a flag to be used multiple times to add advanced meaning. The command can use getValues().length in such a case to determine the number of times it has been specified.

IntegerArgument / LongArgument / DecimalArgument(TODO)

Allows an integer value to be captured on the command line. These arguments do not provide very helpful completion, as their domain of completions is generally too large. Their main purpose is to parse valid integers, rejecting those that are malformed.

StringArgument

This is one of the most 'accepting' arguments, as it will accept any token that is given to it. It also provides no completion. If your command really needs an unbounded String, then this is the right argument to use. This argument should be extended for cases where you want to accept a string, but the domain of acceptable strings is limited, and you wish to reject those tokens not within that domain and also possibly provide completion for the argument.

URLArgument

Similar in some respects to FileArgument, the URLArgument accepts valid tokens that represent a URL. This argument also respects the EXISTING and NONEXISTENT flags. Completion for parts of a url (the scheme for example), may be able to complete, but actual URL completion of domain names and the like may be nearly impossible. It should also be noted that the EXISTING and NONEXISTENT flags will likely cause a DNS lookup to be performed,

Syntax - Defining Commands

The syntax of a command is the definition of options, symbols and arguments that are accepted by commands. Each command defines its own syntax, allowing customization of flags and parameters, as well as defining order. The syntax is constructed using several different mechanisms, which when combined, allow for a great deal of control in restricting what is acceptable in the command line for a given command.

How it works

When you define a new command, you must give define a syntax bundle within a syntax extension point. When the plugin is loaded, the syntax bundle is parsed from the descriptor and loaded into the syntax manager. When the bundle is needed, when completing or when preparing for execution, the bundle is retrieved. Because a syntax bundle is immutable, it can be cached completely, and used concurrently.

Also, the help system uses the syntax to create usage statements and to map short & long flags to the description from an argument.

The puzzle pieces

See this document page for a concise description of the various syntax elements.

When setting out to define the syntax for a command, it is helpful to layout the synopsis and options that the command will need. The synopsis of a command can be used to define separate modes of operation. The syntax block itself is an implied <alternatives>, which means if parsing one fails, the next will be tried. To give an example of how breaking down a command into multiple synopsis can be helpful, we'll setup the syntax for a hypothetical 'config' command that allows listing, setting and clearing of some system configurations.

First, our synopsis...

config
    Lists all known configuration options and their values
config -l 
    Displays the value of the specified option
config -s  
    Sets the chosen option to the desired value
config -c 
    Clears the value of the chosen option

And our syntax...

<syntax alias="config">
  <empty />
  <option argLabel="list" shortName="l">
  <sequence>
    <option argLabel="set" shortName="s">
    <argument argLabel="value">
  </sequence>
  <option argLabel="clear" shortName="c">
</syntax>

To be continued...

Utility classes

The cli project contains a few utility classes to make implementation of common features across multiple commands easier. Because it is recommended that these classes be used when possible, they are quite well documented, and provide fairly specific information on their behavior, and how to use them. A brief outline will be provided here, along with links to the actual javadoc page.

AbstractDirectoryWalker

ADW is _the_ tool for doing recursive directory searches. It provides a Visitor pattern interface, with a set of specific callbacks for the implementor to use. It has many options for controlling what it returns, and with the right configuration, can be made to do very specific searching.

Control

The walker is mainly controlled by FileFilter instances. Multiple filters can be supplied, providing an implied '&&' between each filter. If any of the filters reject the file, then the extending class will not be asked to handle the file. This can be used to create very precise searches by combining multiple boolean filters with specific filter types.

The walker also provides the ability to filter files and directories based on a depth. When the minimum depth is set, files and directories below a given level will not be handled. The directories that are passed to walk() are considered to be at level 0. Therefore setting a min-depth of 0 will not pass those directories to the callbacks. When the maximum depth is set, directories that are at the maximum depth level will not be recursed into. They will however still be passed to the callbacks, pending acceptance by the filter set. Therefore setting a value of 0 to the max level may return the initial directories supplied to walk(), but it will not recurse into them.

Note: Boolean filters are not yet implemented, but they are on the short list.

Extending the walker

Although you can extend the walker to a class of it's own, the recommended design pattern is to implement the walker as a non-static inner class, or an anonymous inner class. This design gives the implemented callbacks of the walker access to the inner structure of the command it's used in. When the walker runs it will pass accepted files and directories to the appropriate callback methods. The walker also has callbacks for specific events, including the beginning and end of a walk, as well as when a SecurityException is encountered when attempting to access a file or directory.

public abstract void handleFile(File)

Tells the implementing class that a regular file has been found and accepted.

public abstract void handleDir(File)

Tells the implementing class that a directory has been found and accepted.

public void handleSpecialFile(File)

Tells the implementing class that a file has been found that is neither a directory or a regular file.

protected void handleRestrictedFile(File)

Tells the implementing class that it has found a file that triggered a SecurityException. By default, this method throws an IOException. This will cause walking to completly halt, which is likely undesired, and so it is highly recommended to override this method to provide suitable error message, and optionally continue walking.

protected void handleStartingDir(File)

Tells the implementing class that it is about to start walking the file system from the given file. This is triggered before the file itself is actually resolved. So the caller has a chance to do some initialization, like possibly changing the current working directory to make a relative path resolve with a different prefix path.

protected void lastAction(boolean)

Tells the implementing class that walking has finished. If the walker stopped walking because it was requested to do so, then the boolean parameter will be true. Otherwise if the walker finished normally, it will be false.

Kernel debugger

A very simple kernel debugger has been added to the JNode nano-kernel. This debugger is able to send all data outputted to the console (using Unsafe.debug) to another computer via a null-modem cable connected to COM1.

From the other computer you can give simple commands to the debugger, such as dump the processor thread queues and print the current thread.

The kernel debugger can be enabled by adding " kdb" to the grub kernel command line, or by activating it in JNode using a newly added command: "kdb".

Ewout

Using "remoteout" to record console / logger output

The remoteout command allows you to send a copy of console output and logger output to a remote TCP or UDP receiver. This allows you to capture console output for bug reports, and in the cases where JNode is crashing.

Before you run the command, you need to set up a receiver application on the remote host to accept and record the output. More details (including a brief note on the JNode RemoteReceiver application) may be found in the remoteout command page. Please read the Bugs section as well!

Boot and startup

During the boot process of JNode, the kernel image is loaded by Grub and booted. After the bootstrapper code, we're running plain java code. The fist code executed is in org.jnode.boot.Main#vmMain() which initializes the JVM and starts the plugin system.

Driver framework

The basic device driver design involves 3 components:

• Device: a representation of the actual hardware device
• Driver: a software driver able to control a Device
• DeviceAPI: a programming interface for a Device, usually implemented by the Driver.

There is a DeviceManager where all devices are registered. It delegates to DeviceToDriverMapper instances to find a suitable driver for a given device. Instances of this mapper interface use e.g. the PCI id of a device (in case of PCIDevice) to find a suitable driver. This is configurable via a configuration file.

For a device to operate there are the following resources available:

• Hardware interrupts. A driver can register an IRQHandler which is called on its own (normal java-) thread. The native kernel signals a hardware interrupt by incrementing a counter for that interrupts, after which the thread scheduler dispatches such events to the correct threads of the IRQHandler's.
• DMA channels. A driver can claim a DMA channel.
  This channel can be setup, enabled and disabled.
• IO port access. An Unsafe class has native methods for this. A device must first claim a range of IO ports before it can gain access to it.
• Memory access. A device can claim a range of the the memory addressspace. A MemoryResource is given to the device. The device can use the methods of the MemoryResource to actually access the memory.

Filesystem framework

The filesystem support in JNode is split up into a generic part and a filesystem specific part. The role of the generic part is:

1. Keep track of all mounted filesystems.
2. Map between path names are filesystem entries.
3. Share filesystem entries between various threads/processes.

The role of the filesystem specific part is:

1. Store and retrieve files.
2. Store and retrieve directories.

We should be more specific about what a filesystem is. JNode makes a distinction the a FileSystemType and a FileSystem. A FileSystemType has a name, can detect filesystems of its own type on a device and can create FileSystem instances for a specific device (usually a disk). A FileSystem implements storing/retrieving files and directories.

To access files in JNode, use the regular classes in the java.io package. They are connected to the JNode filesystem implementation. A direct connection to the filesystem implementation is not allowed.

FrameBuffer devices

This chapter details the FrameBuffer device design and the interfaces involved in the design.

FrameBufferAPI

All framebuffer devices must implement this API.

FrameBufferConfiguration

TODO write me.

Surface

TODO write me.

HardwareCursorAPI

TODO write me.

DisplayDataChannelAPI

TODO write me.

Network devices

This chapter details the design of network devices and describe the interfaces involved.

NetDeviceAPI

Every network device must implement this API.

The API contains methods to get the hardware address of the device, send data through the device and get/set protocol address information.

When a network deivce receives data, it must deliver that data to the NetworkLayerManager. The AbstractNetworkDriver class (which is usually the baseclass for all network drivers) contains a helper method (onReceive) for this purpose.

Network protocols

This chapter will detail the interfaces involved in the network protocol layer.

NetworkLayer

This interface must be implemented by all network protocol handlers.

TransportLayer

This interface must be implemented by OSI transport layer protocols.

LinkLayer

This interface must be implemented by OSI link layer protocols.

Registration

To register a network layer, the network layer class must be specified in an extension of the "org.jnode.net.networkLayers" extension point.
This is usually done in the descriptor of the plugin that holds the network layer.

Architecture specifics

This chapter contains the specific technical operating system details about the various architectures that JNode is operating on.

The new command line syntax mechanism

In the classical Java world, a command line application is launched by calling the "main" entry point method on a nominated class, passing the user's command arguments as an array of Strings. The command is responsible for working out which arguments represent options, which represent parameters and so on. While there are (non-Sun) libraries to help with this task (like the Java version of GNU getOpts), they are rather primitive.

In JNode, we take a more sophisticated approach to the issue of command arguments. A native JNode command specifies its formal arguments and command line syntax. The task of matching actual command line arguments is performed by JNode library classes. This approach offers a number of advantages over the classical Java approach:

• The application programmer has less work to do.
• The user sees more uniform command syntax.
• Diagnostics for incorrect command arguments can be more uniform.

In addition, this approach allows us to do some things at the Shell level that are difficult with (for example) UNIX style shells.

• The JNode shell does intelligent command line completion based on a command's declared syntax and argument types. For example, if the syntax requires a device name at the cursor position when the user hits TAB, the JNode shell will complete against the device namespace.
• The JNode help command uses a command's declared syntax to produce accurate "usage" and parameter type descriptions. These can be augmented by descriptions embedded in the syntax, or in separate files.
• In the new version of the JNode syntax mechanisms, command syntaxes are specified in XML separate from the Java source code. Users can tailor the command syntax, like UNIX aliases only better. This can be used to support portable scripting; e.g. Unix-like command syntaxes could be used with a POSIX shell compatible interpreter to run Unix shell scripts.

As the above suggests, there are two versions of JNode command syntax and associated mechanisms; i.e parsing, completion, help and so on. In the first version (the "old" mechanisms) the application class declares a static Argument object for each formal parameter, and creates a static "Help.Info" data structure containing Syntax objects that reference the Arguments. The command line parser and completer traverse the data structures, binding values to the Arguments.

The problems with the "old" mechanisms include:

• Use of statics to hold the Argument and Help.Info objects makes JNode commands non-reentrant, leading to unpredictable results when a command is executed in two threads.
• The Syntax, Argument and associated classes were never properly documented, making them hard to maintain and hard to use.
• There were numerous bugs and implementation issues; e.g. Unix-style named options didn't work, completion didn't work properly with alternative syntaxes, and so on.
• Command syntaxes could not be tailored, as described above.

The second version (the "new" mechanisms) are a ground-up redesign and reimplementation:

• Argument objects are created by the command class constructor, and registered to form an ArgumentBundle. Thus, command syntax is not an impediment to making command classes re-entrant.
• Syntax objects are created from XML that is defined in the command's plugin descriptor, and that can be overridden from the JNode shell using the "syntax" command.
• The "new" Syntax classes are much richer than the "old" versions. Each Syntax class has a "prepare" method that emits a simple BNF-like grammar; i.e. the MuSyntax classes. This grammar is used by the MuParser which performs n-level backtracking, and supports "normal" and "completion" modes. (Completion mode parsing works by capturing completions at the appropriate point and then initialing backtracking to find other alternatives.)

 cat 
 cat -u | -urls <url> ... |
 cat <file> ... 

 cat d.txt

 cat a.txt b.txt c.txt

 cat a.txt b.txt c.txt > d.txt

   39   <extension point="org.jnode.shell.syntaxes">
   40     <syntax alias="cat">
   41       <empty description="copy standard input to standard output"/>
   42       <sequence description="fetch and concatenate urls to standard output">
   43         <option argLabel="urls" shortName="u" longName="urls"/>
   44         <repeat minCount="1">
   45           <argument argLabel="url"/>
   46         </repeat>
   47       </sequence>
   48       <repeat minCount="1" description="concatenate files to standard output">
   49         <argument argLabel="file"/>
   50       </repeat>
   51     </syntax>

   54     private final FileArgument ARG_FILE =
   55         new FileArgument("file", Argument.OPTIONAL | Argument.MULTIPLE,
   56                 "the files to be concatenated");

   58     private final URLArgument ARG_URL =
   59         new URLArgument("url", Argument.OPTIONAL | Argument.MULTIPLE,
   60                 "the urls to be concatenated");

   62     private final FlagArgument FLAG_URLS =
   63         new FlagArgument("urls", Argument.OPTIONAL, "If set, arguments will be urls");

   67     public CatCommand() {
   68         super("Concatenate the contents of files, urls or standard input to standard output");
   69         registerArguments(ARG_FILE, ARG_URL, FLAG_URLS);
   70     }

   79     public void execute() throws IOException {
   80         this.err = getError().getPrintWriter();
   81         OutputStream out = getOutput().getOutputStream();
   82         File[] files = ARG_FILE.getValues();
   83         URL[] urls = ARG_URL.getValues();
   84 
   85         boolean ok = true;
   86         if (urls != null && urls.length > 0) {
   87             for (URL url : urls) {
   ...
  107         } else if (files != null && files.length > 0) {
  108             for (File file : files) {
   ...
  127         } else {
  128             process(getInput().getInputStream(), out);
  129         }
  130         out.flush();
  131         if (!ok) {
  132             exit(1);
  133         }
  134     }

 cat ( ( -u | -urls <url> ) | <file> ) ...

 cat ( <url> | <file> ) ...

    public static void main(String[] args) throws Exception {
        new XxxClass().execute(args);
    }

    <argument argLabel="foo">
    <argument label="foo" description="this controls the command's fooing" argLabel="foo">

    <empty description="dir with no arguments lists the current directory">

    <option argLabel="filename" shortName="f">
    <option argLabel="filename" longName="file">
    <option argLabel="filename" shortName="f" longName="file">

    <verb symbol="subcommand1" argLabel="someArg">

    <sequence description="the input and output files">
        <argument argLabel="input"/>
        <argument argLabel="output"/>
    </sequence>

    <alternatives description="specify an input or output file">
        <option shortName="i" argLabel="input"/>
        <option shortName="o" argLabel="output"/>
    </alternatives>

    <repeat description="zero or more files">
        <argument argLabel="file"/>
    </repeat>

    <repeat minCount="1" description="one or more files">
        <argument argLabel="file"/>
    </repeat>

    <repeat maxCount="5" eager="true" 
              description="as many files as possible, up to 5">
        <argument argLabel="file"/>
    </repeat>

    <optional description="nothing, or an input file and an output file">
        <argument argLabel="input"/>
        <argument argLabel="output"/>
    </optional>

    <optional eager="true"
                 description="an input file and an output file, or nothing">
        <argument argLabel="input"/>
        <argument argLabel="output"/>
    </optional>

    <powerSet description="any number of inputs and outputs">
        <option argLabel="input" shortName="i"/>
        <option argLabel="output" shortName="o"/>
    </powerSet>

    <optionSet description="flags and value options">
        <option argLabel="flagOne" shortName="1"/>
        <option argLabel="flagTwo" shortName="2"/>
        <option argLabel="argThree" shortName="3"/>
    </optionSet>

    <syntax alias="cpuid">
        <empty description="output the computer's id">
    </syntax>

    <syntax alias="dir">
        <empty description="list the current directory"/>
        <argument argLabel="directory" description="list the given directory"/>
    </syntax>

    <alternatives>
        <argument argLabel="number">
        <argument argLabel="file">
    </alternatives>

Using Command Line Completion (old syntax mechanism)

Note: this page describes the old syntax mechanism which is currently being phased out. Please refer to the parent menu for the pages on the new syntax mechanism.

The JNode Command Line Completion is one of the central aspects of the shell. JNode makes use of a sophisticated object model to declare command line arguments. This also provides for a standard way to extract a help-document that can be viewed by the user in different ways. Additionally, the very same object model can be used to access the arguments in a convenient manner, instead of doing the 133735th command line parsing implementation in computer history.

The following terms play an important role in this architecture:

• Parameter
  A key that can be passed to a programm. Typically, this are command line switches like "-h" or "--help", or indicators for the type of the assigned argument.
• Argument
  A value that can be passed to a program. This can be filenames, free texts, integers or whatever type of arguments the program needs.
• Syntax
  A program can define multiple Syntaxes, which provide for structurally different tasks it can handle. A Syntax is defined as a collection of mandatory and optional Parameters.

Black-box command tests with TestHarness

Overview

This page gives some guidelines for specifying "black-box tests" to be run using the TestHarness class; see Running black-box tests".

A typical black-box test runs a JNode command or script with specified inputs, and tests that its outputs match the outputs set down by the test specification. Examples set specification may be found in the "Shell" and "CLI" projects in the respective "src/test" tree; look for files named "*-tests.xml".

Syntax for test specifications

Let's start with a simple example. This test runs "ExprCommand" command class with the arguments "1 + 1", and checks that it writes "2" to standard output and sets the return code to "0".

<testSpec title="expr 1 + 1" command="org.jnode.shell.command.posix.ExprCommand"
             runMode="AS_ALIAS" rc="0">
  <arg>1</arg>
  <arg>+</arg>
  <arg>1</arg>
  <output>2
</output>
</testSpec>

Notes:

1. The odd indentation of the closing "output" tag is is not a typo. This element is specifying that the output should consists of a "2" followed by a newline. If the closing tag was indented, the test would "expect" a couple of extra space characters after the newline, and this would cause a spurious test failure.
2. Any literal '<', '>' and '&' characters in the XML file must be suitably "escaped"; e.g. using XML character entities;

An "testSpec" element and its nested elements specifies a single test. The elements and attributes are as follows:

"title" (mandatory attribute)

gives a title for the test to identify it in test reports.

"command" (mandatory attribute)

gives the command name or class name to be used for the command / script.

"runMode" (optional attribute)

says whether the test involves running a command alias or class ("AS_ALIAS") in the same way as the JNode CommandShell would do, running a script ("AS_SCRIPT"), or executing a class via its 'main' entry point ("AS_CLASS"). The default for "runMode" is "AS_ALIAS".

"rc" (optional attribute)

gives the expected return code for the command. The default value is "0". Note that the return code cannot be checked when the "runMode" is "AS_CLASS".

"trapException" (optional attribute)

if present, this is the fully qualified classname of an exception. If the test throws this exception or a subtype, the exception will be trapped, and the harness will proceed to check the test's post-conditions.".

"arg" (optional repeated elements)

these elements gives the "command line" arguments for the command. If they are omitted, no arguments are passed.

"script" (conditional element)

if "runMode" is "AS_SCRIPT", this element should contain the text of the script to be executed. The first line should probably be "#!<interpreter-name>".

"input" (optional element)

this gives the character sequence that will be available as the input stream for the command. If this element is absent, the command will be given an empty input stream.

"output" (optional element)

this gives the expected standard output contents for the command. If this element is absent, nothing is expected to be written to standard output.

"error" (optional element)

this gives the expected standard error contents for the command. If this element is absent, nothing is expected to be written to standard error.

"file" (optional repeating element)

this gives an input or output file for the test, as described below.

Syntax for "file" elements

A "file" element specifies an input or output file for a test. The attributes and content
are as follows:

"name" (mandatory attribute)

gives the file name. This must be relative, and will be resolved relative to the test's temporary directory.

"input" (optional attribute)

if "true", the element's contents will be written to a file, then made available for the test or harness to read, If "false", the element's contents will be checked against the contents of the file after the test has run.

"directory" (optional attribute)

if "true", the file denotes a directory to be created or checked. In this case, the "file" element should have no content.

Script expansion

Before a script is executed, it is written to a temporary directory. Any @TEMP_DIR@ sequence in the script will be replaced with the name of the directory where input files are created and where output files are expected to appear.

Syntax for test sets

While the test harness can handle XML files containing a single <testSpec> element, it is more convenient to assemble multiple tests into a test set. Here is a simple example:

<testSet title="expr tests"">
  <include setName="../somewhere/more-tests.xml"/>
  <testSpec title="expr 1 + 1" ...>
...
  </testSpec>
  <testSpec title="expr 2 * 2" ...>
...
  </testSpec>
</testSet>

The "include" element declares that the tests in another test set should be run as part of this one. If the "setName" is relative, it will be resolved relative to this testSet's parent directory. The "testSpec" elements specify tests that are part of this test set.

Plugin handling

As a general rule, JNode command classes amd aliases are defined in plugins. When the test harness is run, it needs to know which plugins need to be loaded or the equivalent if we are running on the development platform. This is done using "plugin" elements; for example:

  ...
  <plugin id="org.jnode.shell.bjorne"
             class="org.jnode.test.shell.bjorne.BjornePseudoPlugin"/>
  <plugin id="org.jnode.shell.command.posix"/>
  ...

These elements may be child elements of both "testSpec" or "testSet" elements. A given plugin may be specified in more than one place, though if a plugin is specified differently in different places, the results are undefined. If a "plugin" element is in a "testSpec", the plugin will be loaded before the test is run. If a "plugin" element is in a "testSet", the plugin will be loaded before any test in the set, as well as any test that are "included".

The "plugin" element has the following attributes:

"id" (mandatory attribute)

gives the identifier of the plugin to be loaded.

"version" optional attribute)

gives the version string for the plugin to be loaded. This defaults to JNode's default plugin version string.

"class" optional attribute)

gives the fully qualified class name for a "pseudo-plugin" class; see below.

When the test harness is run on JNode, a "plugin" element causes the relevant Plugin to be loaded via the JNode plugin manager, using the supplied plugin id and the supplied (or default) version string.

When the test harness is run outside of JNode, the Emu is used to provide a minimal set of services. Currently, this does not include a plugin manager, so JNode plugins cannot be loaded in the normal way. Instead, a "plugin" element triggers the following:

1. The plugin descriptor file is located and read to extract any aliases and command syntaxes. This information is added to Emu's alias and syntax managers.
2. If the "plugin" element includes a "class" attribute, the corresponding class is loaded and the default constructor is called. This provides a "hook" for doing some initialization that would normally be done by the real Plugin. For example, the "plugin" in the bjorne tests use the "BjornePseudoPlugin" class to registers the interpreter with the shell services. (This is normally done by "BjornePlugin".)

Arrays

Arrays are allocated just like normal java objects. The number of elements of the array is stored as the first (int) instance variable of the object. The actual data is located just after this length field.

Bytecodes that work on arrays are do the index checking. E.g. on the X86 this is implemented using the bound instruction.

Classes & Objects

Each class is represented by an internal structure of class information, method information and field information. All this information is stored in normal java objects.

Every object is located somewhere in the heap. It consists of an object header and space for instance variables.

Exceptions

At the start of each method invocation a frame for that method is created on the stack. This frame contains references to the calling frame and contains a magic number that is used to differentiate between compiled code invocations and interpreted invocations.
When an exception is thrown, the exception table of the current method is inspected. When an exception handler is found, the calculation stack is cleaned and code executing continues at the handler address.
When no suitable exception handler is found in the current method, the stackframe of the current method is destroyed and the process continues at the calling method.

When stacktrace of an exception is created from the frames of each method invocation. A class called VmStackFrame has exactly the same layout as a frame on the stack and is used to enumerate all method-invcocations.

Garbage collection

JNode uses a simple mark&sweep collector. You can read on the differences, used terms and some general implementation details at wikipedia. In these terms JNode uses a non-moving stop-the-world conservative garbage collector.

About the JNode memory manager you should know the following: There is org.jnode.vm.memmgr.def.VmBootHeap. This class manages all objects that got allocated during the bootimage creation. VmDefaultHeap contains objects allocated during runtime. Each Object on the heap has a header that contains some extra information about the heap. There's information about the object's type, a reference to a monitor (if present) and the object's color (see wikipedia). JNode objects can have one of 4 different colors and one extra finalization bit. The values are defined in org.jnode.vm.classmgr.ObjectFlags.

At the beginning of a gc cycle all objects are either WHITE (i.e. not visited/newly allocated) or YELLOW (this object is awaiting finalization).

The main entry point for the gc is org.jnode.vm.memmgr.def.GCManager#gc() which triggers the gc run. As you can see one of the first things in gc() is a call to "helper.stopThreadsAtSafePoint();" which stops all threads except the garbace collector. The collection then is divided into 3 phases: markHeap, sweep and cleanup. The two optional verify calls at the beginning and end are used for debugging, to watch if the heap is consistent.

The mark phase now has to mark all reachable objects. For that JNode uses org.jnode.vm.memmgr.def.GCStack (the so called mark stack) using a breadth-first search (BFS) on the reference graph. At the beginning all roots get marked where roots are all references in static variables or any reference on any Thread's stack. Using the visitor pattern the Method org.jnode.vm.memmgr.def.GCMarkVisitor#visit get called for each object. If the object is BLACK (object was visited before and all its children got visited. Mind: This does not mean that the children's children got visited!) we simply return and continue with the next reference in the 'root set'. If the object is GREY (object got visited before, but not all children) or in the 'root set' the object gets pushed on the mark stack and mark() gets called.
Let's make another step down and examine the mark() method. It first pops an Object of the mark stack and trys to get the object type. For all children (either references in Object arrays or fields of Objects) the processChild gets called and each WHITE (not visited yet) object gets modified to be GREY. After that the object gets pushed on the mark stack. It is important to understand at that point that the mark stack might overflow! If that happens the mark stack simply discards the object to push and remembers the overflow. Back at the mark method we know one thing for sure: All children of the current object are marked GREY (or even BLACK from a previous mark()) and this is even true if the mark stack had an overflow. After examining the object's Monitor and TIB it can be turned BLACK.
Back at GCManager#markHeap() we're either finished with marking the object or the mark stack had an overflow. In the case it had an overflow we have to repeat the mark phase. Since many objects now are allready BLACK it is less likly the stack will overflow again but there's one important point to consider: All roots got marked BLACK but as said above not all children's children need to be BLACK and might be GREY or even WHITE. That's why we have to walk all heaps too in the second iteration.
At the end of the mark phase all objects are either BLACK (reachable) or WHITE (not reachable) so the WHITE ones can be removed.

The sweep again walks the heap (this time without the 'root set' as they do not contain garbage by definition) and again visits each object via org.jnode.vm.memmgr.def.GCSweepVisitor#visit. As WHITE objects are not reachable anymore it first tests if the object got previously finalized. If it was it will be freed, if not and the object has a Finalizer it will be marked YELLOW (Awaiting finalization) else it will be freed too. If the object is neither WHITE nor YELLOW it will be marked WHITE for the next gc cycle.

The cleanup phase at the end sets all objects in the bootHeap to WHITE (as they will not be swept above) as they might be BLACK and afterwards calls defragment() for every heap.

Some other thoughts regarding the JNode implementation include:

It should be also noted that JNode does not know about the stack's details. I.e. if the mark phase visits all objects of a Thread's stack it never knows for a value if it is a reference or a simple int,float,.. value. This is why the JNode garbage collector can be called conservative. Every value on the stack might be a reference pointing to a valid object. So even if it is a float on the stack, as we don't know for sure we have to visit the object and run a mark() cycle. This means on the one hand that we might mark memory as reachable that in reality is garbage on the other hand it means that we might point to YELLOW objects from the stack. As YELLOW objects are awaiting finalization (and except the case the finalizer will reactivate the object) they are garbage and so they can not be in the 'root set' (except the case where we have a random value on the stack that we incorrectly consider to be a reference). This is also the reason for the current "workaround" in GCMarkVisitor#visit() where YELLOW objects in the 'root set' trigger error messages instead of killing JNode.

There is some primilary code for WriteBarrier support in JNode. This is a start to make the gc concurrent. If the WriteBarrier is enabled during build time, the JNode JIT will include some special code into the compiled native code. For each bytecode that sets a reference to any field or local the writebarrier gets called and the object gets marked GREY. So the gc will know that the heap changed during mark. It is very tricky to do all that with proper synchronization and the current code still has bugs, which is the reason why it's not activated yet.

Java Security

This chapter covers the Java security implemented in JNode. This involves the security manager, access controller and privileged actions.
It does not involve user management.

The Java security in JNode is an implementation of the standard Java security API. This means that permissions are checked against an AccessControlContext which contains ProtectionDomain's. See the Security Architecture for more information.

In JNode the security manager is always on. This ensures that permissions are always checked.
The security manager (or better the AccessController) executes the security policy implemented by JNodePolicy. This policy is an implementation of the standard java.security.Policy class.
This policy contains some static permissions (mainly for access to certain system properties) and handles dynamic (plugin) permissions.

The dynamic permissions are plugin based. Every plugin may request certain permissions. The Policy implementation decides if these permissions are granted to the plugin.

To request permissions for a plugin, add an extension to the plugin-descriptor on connected to the "org.jnode.security.permission" extension-point.
This extension has the following structure:

Multiple permission's can be added to a single extension.

If you need specific permissions, make sure to run that code in a PrivilegedAction. Besides you're own actions, the following standard PrivilegedAction's are available:

Multi-threading

Multithreading in JNode involves the scheduling of multiple java.lang.Thread instances between 1 or more physical processors. (In reality, multiprocessor support is not yet stable). The current implementation uses the yieldpoint scheduling model as described below.

Yieldpoint scheduling

Yieldpoint scheduling means that every thread checks at certain points (called "yieldpoints") in the native code to see if it should let other threads run. The native code compiler adds yieldpoints into the native code stream at the beginning and end of a method, at backward jumps, and at method invocations. The yieldpoint code checks to see if the "yield" flag has been set for the current thread, and if is has, it issues a yield (software-)interrupt. The kernel takes over and schedules a new thread.

The "yield" flag can be set by a timer interrupt, or by the (kernel) software itself, e.g. to perform an explicit yield or in case of locking synchronization methods.

The scheduler invoked by the (native code) kernel is implemented in the VmProcessor class. This class (one instance for every processor) contains a list of threads ready to run, a list of sleeping threads and a current thread. On a reschedule, the current thread is appended to the end of the ready to run thread-list. Then the sleep list is inspected first for threads that should wake-up. These threads are added to the ready to run thread-list. After that the first thread in the ready to run thread-list is removed and used as current thread. The reschedule method returns and the (native code) kernel does the actual thread switching.

The scheduler itself runs in the context of the kernel and should not be interrupted. A special flag is set to prevent yieldpoints in the scheduler methods themselves from triggering reentrant yieldpoint interrupts. The flag is only cleared when the reschedule is complete

Why use yieldpoint scheduling?

JNode uses yield point scheduling to simplify the implementation of the garbage collector and to reduce the space needed to hold GC descriptors.

When the JNode garbage collector runs, it needs to find all "live" object references so that it can work out which objects are not garbage. A bit later, it needs to update any references for objects that have been moved in memory. Most object references live either in other objects in the heap, or in local variables and parameters held on one of the thread stacks. However, when a thread is interrupted, the contents of the hardware registers are saved in a "register save" area, an this may include object references.

The garbage collector is able to find these reference because the native compiler creates descriptors giving the offsets of references. For each class, there is a descriptor giving the offsets of its reference attributes and statics in their respective frames. For each method or constructor, another descriptor gives the corresponding stack frame layout. But we still have to deal with the saved registers.

If we allowed a JNode thread to be interrupted at any point, the native compiler would need to create descriptors all possible saved register sets. In theory, we might need a different descriptor corresponding to every bytecode. By using yield points, we can guarantee that "yields" only occur at a fixed places, thereby reducing the number of descriptors that that need to be kept.

However, the obvious downside of yieldpoints is the performance penalty of repeatedly testing the "yield" flag, especially when executing a tight loop.

Thread priorities

Thread can have different priorities, ranging from Thread.LOW_PRIORITY to Thread.HIGH_PRIORITY. In JNode these priorities are implemented via the ready to run thread-list. This list is (almost) always sorted on priority, which means that the threads with the highest priority comes first.

There is one exception on this rule, which is in the case of busy-waiting in the synchronization system. Claiming access to a monitor (internals) involves a busy-waiting loop with an explicit yield. This yield ignores the thread priority to avoid starvation of lower-priority threads, which will lead to an endless waiting time for the high priority thread.

Classes involved

The following classes are involved in the scheduling system. All of these classes are in the org.jnode.vm package.

• VmProcessor
• VmThread contains the internal (JNode specific) data for a single thread. This class is extended for each specific platform

Native code compilation

All methods are compiled before being executed. At first, the method is "compiled" to a stub that calls the most basic compiler and then invokes the compiled code.

Better compilers are invoked when the VM detects that a method is invoked often. These compilers perform more optimizations.

Object allocation

All new statements used to allocate new objects are forwarded to a HeapManager. This class allocates & initializes the object. The objects are allocated from one of several heaps. Each heap contains objects of various sizes. Allocation is currently as simple as finding the next free space that is large enough to fit all instance variables of the new object and claiming it.

An object is blanked on allocation, so all instance variables are initialized to their default (null) values. Finally the object header is initialized, and the object is returned.

To directly manipulate memory at a given address, a class called Unsafe is used. This class contains native methods to get/set the various java types.

Synchronization

Synchronization involves the implementation of synchronized methods and blocks and the wait, notify, notifyAll method of java.lang.Object.

Both items are implemented using the classes Monitor and MonitorManager.

Lightweight locks

JNode implement a lightweight locking mechanism for synchronized methods and blocks. For this purpose a lockword is added to the header of each object. Depending on the state of the object on which a thread wants to synchronize a different route it taken.

This is in principle how the various states are handled.

1. The object is not locked: the lockword it set to a merge of the id of this thread and a lockcount of '1'.
2. The object is locked by this thread: the lockcount part of the lockword is incremented.
3. The object is locked by another thread: an inflated lock is installed for the object and this thread is added to the waiting list of the inflated lock.

All manipulation of the lockword is performed using atomic instructions prefixed with multiprocessor LOCK flags.


When the lockcount part of the lockword is full, an inflated lock is also installed.


Once an object has an inflated lock installed, this inflated lock will always be used.

Wait, notify

Wait and notify(all) requires that the current thread is owner of the object on which wait/notify are invoked. The wait/notify implementation will install an inflated lock on the object if the object does not already have an inflated lock installed.

Development teams

Development in the JNode project is done in development teams.
These teams have been introduced to:

• Focus development efforts
• Shorten learning curve for new developers
• Delegate project control

Overall coordination of the JNode project remains in the hands of the
projects founder: Ewout Prangsma.

How to join the development team

We always welcome new dedicated developers.

Following content is outdated since we have moved to GitHub.

If you want to join the development team, contact one of the developers who is working on an issue you want to contribute to, or contact the project admin for more information.

You will be asked to submit your first patches via email, before you'll be granted access to the SVN repository.

Student Projects

In order to imply students in JNode, I will expose here some projects.

Git repository
    The gitorious project is located here : http://gitorious.org/jnode.

    It's compound of:

• svn-mirror, which is a mirror of the subversion repository
• a set a clones of svn-mirror, one for each student project

Contact
    If you are interested, you can contact me (Fabien DUMINY) :

• with my jnode.org contact form (in english or french)
• with my blog contact form :
  • en français
  • in english

Remarks

• Since it's a draft, that document may evolve
• In order to have some warranty of real work, I have contacted some schools to imply students in these tasks. These students will be served first and in the order of their arrival

Isolated java heaps

Isolating memory allocation will cause some problems that need to be dealed with.

Problems
When an object is allocated in one isolate, and this isolate is closed, the object is also removed. But what if this object is registered somewhere in another isolate?
Answer: A problem. Objects may not cross isolate bounderies.

A new networking framework

Networking framework

The goals of this proposal:
- Flexibility, more capabilities
- Simplicity, better Object-Oriented design

The following are the 3 basic types of entities within this framework:
- Network devices
- Network filters
- Applications.

The network devices are the device drivers of the Network interfaces or virtual network drivers (like the local loopback). We assume that a device driver like that takes a packet stored in a memory buffer and write it on the media (wire, air or anything else). Also it receives a packet from the media and writes it to a memory buffer. The device drivers that do some job of the networking system, such as checksum calculation or crypto work, may be treated also as Network filters (they will be discussed later).

At the opposite hand are the applications. This is any software that creates packets and injects them, or receives a packet from the networking subsystem. For example: a ping command, or dns server software, and also the java SocketImpl system.

Both Network device drivers and applications are the endpoints of this model. From them the packets are coming in and out of the networking subsystem of the OS. Between them we have the Filters.

The filters take packets from any component and forward them to other components. A subcategory of them, are the protocol stacks. The filters are the components that are used to transform the packets or do other things with these packets. For example a TCP/IP stack is a filter that parses the incoming byte arrays (RawPacket from the device driver) to TCP/IP packets, or encapsulates data into TCP/IP packets to be sent later over another protocol. There is more that the TCP/IP stack will do internally, but it’s a matter of the TCP/IP stack implementation and not of this framework proposal.

These filters have one or more PacketStreams. Most components may have two packet streams. Any packet stream implements two interfaces, the PacketListener and the PacketFeeder. Any PacketListener may be registered to a feeder to receive packets from him. This way we can have Chains of packet streams, some of them may split (?)*. Also a filter may be just a PacketFeeder or PacketListener. For example a component that captures packets directly to the filesystem, a counter for some specific packets, a traffic generator, etc (but they may not be treated as endpoints).

For performance reasons we can use a “listening under criteria” system. And only when these criteria are matched the feeder will send the packet to this listener. We can have an interface ListenerCriteria with a simple listener specific method that reads some bytes or anything else from the incoming packet and return true or false. This method will be called from the packet feeder, before he sends the packet to the listener. For example an IPListenerCriteria will check the ethertype bytes if the packet feeder is EthernetLayer. Or another ListenerCriteria implementer may check the class of a packet to see if it is instance of the ICMPPacket. Listening under criteria will be a way to filter packets that will be passed from stream to stream.

The PacketFeeder’s will have a registry, where they store the currently registered listeners to them. For this registry I suggest to use a List of bindings, where every binding of this list will have as key a PacketCriteria type instance and as value a list of the PacketListeners that are listening under these criteria.

For performance reasons, a packet feeder may have two or more different registries of listeners, one for the high priority listeners and one for the others or one registry for the protocol to protocol submission and one for the others (and if they exist).

To avoid deadlocks between components and performance degradation, when a feeder passes a packet to a listener, the listener may use incoming queue and have its own thread to handle the packet. Except if the packet handling that he would do is something very quick.

Another issue, is how all this relations between the components will be managed. A command or a set of commands will be useful. This is a part of Ifconfig mainly.

The result of all these will be a web of networking components, where everyone can communicate to everyone. Think the possibilities, I have found many already.

This is an abstract of what I am thinking. The details is better to be discussed here.

Pavlos Georgiadis

Packet representation

Packet representation

The goal of this proposal is mainly to speed up the packet handling and to provide the developers a simpler and more Object Oriented packet representation. It aims to remove the current SocketBuffer and replace it with Packets.

Currently the packets are represented with the SocketBuffer class. The SocketBuffer is something like a dynamic buffer (to be more accurate it is more like a list of dynamic buffers). When a program wants to send a network packet, it creates the protocol headers, and then the headers are inserted in this buffer list with the packet payload (if there is a payload). Finally the NIC driver copies all the bytes of the packet into another fixed array to be used from the driver.
When we send a packet we move all the packet data 2 times (from the headers to the SocketBuffer and from it to the final fixed array). When we receive a packet, the SocketBuffer acts like a fixed array, which provides us with some useful methods to read data from him.

What I suggest is to represent the packets as…Packets. All the protocol packets are the same thing. They have a header and a payload (some of them have a tail too). Every packet is able to know its size and how to handle its data (including the payload).

So let’s say we have the interface Packet.

What we need from the Packet implementers:
- Set and Get methods for the data of the packets (class specific and wont be in the Packet interface)
- A way to store the data in an object and to have a method that will return the entire packet as a byte[] (when we send the packet).
- Methods that will parse a byte array to create the Packet object (when we receive a packet).

Any packet is represented with a class that implements the Packet interface. For example IPv4Packet, ICMPPacket, TCPPacket etc. Every component of the networking can access the data of this packet with the appropriate set and get methods (If a program uses a packet knows it’s specific assessors).

To accomplish the second and the third we need the following methods for the interface Packet:

public void getData(byte[] b);
public void getData(byte[] b, int offset);
public void getSize();

The getData(byte[] b) writes the packet in the given byte array. If the length of this array is not enough we can throw an exception. The getData(byte[] b, int offset) writes a packet to the array b, starting from the offset position. The getSize method will return the length of this packet, including the length of its payload.

These methods will be called mainly from the network drivers. This is the point where the packet is converted from object to a memory buffer and vice versa.

When the Ethernet protocol sends a packet to the driver the driver will call the getSize() of the Ethernet packet to determine how big will be the array that will store the entire packet. The Ethernet packet getSize() method will return for example 14+payload.getSize(). Remember that the payload is also a packet, let’s say an IP packet that may return for example 20+payload.getSize(). As the driver has determine the length of the entire packet, it will create the memory buffer b and it will call the getData(byte[] b) to the Ethernet packet, which will write the first 14 bytes and also it will internally call payload.getData(b, 14). This way we move the data only once, from the objects to the final byte array.

A common class that implements the Packet interface is the RawPacket, which is a packet that maps a byte array, or a portion of this array as packet.

The Raw packet will be mainly used:
- To store the data payload of a packet (for example the data payload of a TCP packet)
- To map a received packet before it will be parsed from the networking components.

A practical example for the second:

When a packet is received from a NIC, the driver will create a RawPacket with the array that stores the currently received frame. Later this RawPacket will be send for example to the Ethernet protocol, which will parse the first 14 bytes to its attributes and create another (or modify the same) RawPacket that will map the same byte[] from the position 15 to the end. This RawPacket will be the payload of this Ethernet packet, that later will be send to the IPv4 stack for example and so on.

Pavlos Georgiadis