Prevayler, Prevalence layer for Java - integration in JNode

Thanks to everyone for the sugestions, they were very helpful, though perhaps not in they way that they were intended.

I consider component architectures to be my forte, and a first found it hard going to design a oodbms. My first atempt was among the ugliest things I have ever writen. It had a complicated locking system, very ineficient persistence mechinism, and clumsy symantics. Over all it was uneligent and dificult, not something you would expect from java or a first class os.

After reading the prevalence comment above I commented that SoftReference caching would make an oodb more efficient. After that more comments came in with suggestions. As I was reading the documentation for these systems it came to me that an oodbms at its core is not that very diferent from a component architecture.

With this enlightened point of view, I began to redesign the JNodeFileSystem (wich I renamed JNodeFS). This is what i did:

1) I added SoftReference caching as was sugested. This makes every aspect of the system enormously faster. (The JNodeFileSystem was deserializing the object from harddrive at every call. Ah!)

2) I added a store/update style persistence mechinism. The stored object state must be kept curent with the live object state, in case of a system crash. Instead of serializing the entire object to drive at every update, only store the changed part. A store saves the entire object. A update saves only the modified part. Many updates may follow a store, but a store replaces the old store, and all of the updates. This proccess keeps the persistent state identical with the live state, while minimizing repetative data storage.

//This is the cool part
3)I combined dynamic subclassing as in EJB CMP, and a proxy system like RMI, to produce a dynamic class proxy. The system works like this, a client adds a pre-existing object to the JNodeFS. The jnfs caches the object and examins the class of the object and writes a subclass implementation that overwrites all of the public methods of the original object. This class is compiled and instanciated with a key to the original cached object. Finally this new class proxy object is returned to the client.

The overloaded methods of the class proxy object cominicate to the jnfs which method was called and with what arguments. The jnfs uses the key from the class proxy to look up the original object, and then uses the supplied data to invoke the method dynamically and then returns the result.

All arguments are wrapped and unwrapped properly and exceptions are handeld accuratly.

Because the proxy object extends the supplied object's class, it can be handled just like the supplied object. For example:

Hashtable t = (Hashtable) jnfs.putObject(new Hashtable()); //get proxy
t.put("two","one"); // Use normal api
t.put("date",new Date());
System.out.println( t.get("date"));
t.remove("date");
t.add("Cool","Huh?");

This looks just like a normal Hashtable interaction, but it is all done through proxy. The persistent state of the object is updated every time a method is called on the proxy object. If the entire state of the object had to be writen, then that could get expensive, but only the method name, args, and signiture are saved in an update.

There are some syntactic requirements for an object to be elegible for JnodeFS persistence. Most of them are trivial.

Object can not:
Be final or abstract.
Object must:
Be instance of Serializable, or Externalizable, and provide an empty (no-arg) constructor.

These are the enforced requirements. For the system to work properly the object must be deterministic, and all of the methods that need to be exposed in the proxy must be public.

If it is not deterministic (if it does not do the same thing, provided it has the same args) then the live and persistent states will lose synchronization. If proxy methods are not public, they will not be read by reflection, will not be put in the proxy class, and will not relay invocation properly.

The object cache is built on a stack. When ever an object is looked up (this happens at every proxy invocation) the retreved object is moved from its place to the top of the stack. The stack is seached from top down, so objects that are used often are found quickly. The cache will not excede 90% of the vm memory, and since it is based on SoftReferences, if the vm suddenly needs a lot of space it can just clear cache entries.

Once an object has been added to the jnfs it can be retrieved agian using queries. A Query is based intirely on reflection, and comparative equality (.equals()). For example to look up the Hashtable we just made you could do this:

Query q = new Query(Hashtable.class); //Class of object to search
Method m = Hashtable.class.getMethod("get",new Class[]{Object.class});
q.addConstraint(new QueryMethod(m, new Object[]{"two"},true,"one");

// this calls .get("two").equals("one");
jnfs.executeQuery(q);

executeQuery() returns a collection of keys to object that matched the constraints.

The jnfs cross references objects by interface and super class recursively, so we could just as easily searched for Map.class, or Dictionary.class.

In sumary the JnodeFS alows for the transparent persistence of objects in an extreamly efficient manor.

The JnodeFS is a major improvement over my first design.

What do you think?

Alex

I think it's an interesting idea because everything in JAVA is an Object (but special case for primitive types). So, why not file themself !

This give me other ideas :
- the virtual memory (ie memory on disk also called swap file) can contain unused Objects (at a given time). Perhaps, we could think this memory as a folder and all files in it as "swapped Objects".

- We could extend the previous idea to a "suspend to disk" functionality

- The cache disk could be a Collection of Objects

- A ".jpg" file (for example) could be thought as JPGFile that extends (or implements) ImageFile. And the JNode GUI can operate on Objects and their Classes instead of Files.

IfF you need a simple indexed object store, you might want to have a look at JISP: http://www.coyotegulch.com/jisp/ to see if it helps.

Hi,
if you need some Help about Prevayler (but I don't know it very much), contact me. At the moment, Prevayler hasn't some features:
having a mix of Prevayler and an OODBMS could be a good thing.

Have you also tried to use Ozone-Db, a OODBMS OpenSource, from www.ozone-db.org ?

Only one question:
what will happen to the JNode System if the File System OODBMS hang ? All the rest of the System will be able to keep stable ? And can I configure the FileSystem Daemon to use x MB of Ram, like the SGA in Oracle ? In this manner we could tune that Service depending to the Pc and the use we want (for ex. if I use the System as a FileServer I could give to your Daemon more memory ...).

Excuse me but I haven't understand:
this will be the only FileSystem supported in JNode, or is an Extended FileSystem ?

Have you seen that in Microsoft LongHorn (the Next Release of Windows XP) they will implement a similar FileSystem ... I think that's a good idea. Good work.

Bye,
Sandro