Thoughts on more powerful filesystem sematics
Submitted by sanity on Fri, 05/28/2004 - 12:27.
I have been somewhat interested in ideas around more powerful filesystems, supporting more powerful set-theory functions than provided by common hierarchial approaches (which essentially focus on the member_of relationship between objects), perhaps even a filesystem that approaches the power of a relational database (although tailored towards the storage of Java objects and programatic manipulation of those objects - ie. not a simple SQL database 
JNode seems like an incredible opportunity to explore "next generation" operating system concepts such as this, and I was wondering whether there has been any thought around this?
If anyone is interested on some further reading, here is a description of the BeOS filesystem which applied many of these concepts:
- Login to post comments
Some network-engine backend?
Hi,
in my old company, we implemented a network engine much similar to
http://www.tm4j.org/
to handle a powerful modelling of a network of file nodes, security nodes, different views on the network strukture via a query language, metadata to these nodes as node attributes, distributed storage etc.etc.
I think it owuld be worth to explore these sematics when you talk about a powerful backend to JNode!
What fo you think?
/peter
Advanced FS
Hi,
Some of us have discussed providing advanced FS support, this being made possible by the power and flexiblity of Java's oop design.
Although several people joined in the discussion, as far as I know, I am the only person to produce a product. I do not fault these individuals, they have all been working steadily on more relevant issues.
However, I have been alowed the privilage to work on more long range projects, and amoung them have been a type of transparent oodb, and an object atribute system.
I have read your reference artical and find my systems to be similar, though perhaps not as advanced.
My transparent oodb manages the persistent state of an object while allowing access to that object via an objective proxy (not interface proxy) this allows for the client to manipulate the object using that object's api. This is transparent access.
The oodb has a query function that is accomplished by interface implementation. This being apposed to a sub language like sql, or the construction of a complex query object. Here's an example.
new PersistenceQuery()
{
public Class getType()
{
return Document.class;
}
public boolean fitsConstraints(Object o)
{
Document d = (Document)o; //don't need isInstance()
int i = d.getText(0,d.getLength()).indexOf("War and Peace");
return (i != -1) ? true : false;
} //end fitsConstrancts()
}; //end class
This query will produce soft references to every object of type Document that contains the string "War and Peace". As you can see, queries are made in normal java code.
The transparent oodb is mostly fail safe. It uses a store\update system to achive effiecient serialization rates, along with protecting against data loss. There are still a few places where currupt files need to be deleted after a falure, but it is fairly reliable.
This system is used to achieve type attributes (ie. getter's and setter's et. al.).
I have also made a system for aplying textural attributes to objects. This system is built on the transparent oodb. Simply, it pairs objects with collections of Strings. The objects can be regained by querrying for any combinatin of the Strings. The system is of course more refined by the use of special classes like Attribute, AttributeList and an AttributeManager, which hide all of the work of storing and searhcing.
I have also writen a nice little gui to save and open objects in the attribute system. It works fine until you try to get the sellected objects. The method blocks until the user confirms the sellection with the save or open button, like getFiles() in FileDialog. However when my getSelections() is called the gui stops drawing. ???
I am very interested in seeing these types of abilities added to jnode, because I belive in the power they provide.
What do you think of what I have told you? Would it help to see my code?
Alex
Interesting
I am certainly glad that I am not the first person to consider this, but I am not quite sure I fully understand your proposal.
Is it not the case that if we are to find which objects fit the constraint specified, then an exhaustive search of all objects would be required?
Details
There are several pertenent factors that narrow the field. The first of which is that we are talking about objects that have been specifically stored in the peristence mechinism; not the set of all live objects in the vm.
Next is the type of the object being saught. getType() in PersistenceQuery must be implemented to return a the desired type, this can be any Class except Object, Serializable, or Externalizable. These exceptions are made firstly because the types do not represent actual data, and because every object in the store would match this critierion.
Anyway, the getType() is used to narrow the field to only the instances of that class.
Finaly there is the ability to restrict the number of objects returned from the search. This is done by overriding getResultLength() in PeristenceQuery to return the maximum number of objects the seach can yeild.
There is also the possibilty of narrowing the search by the date that the saught objects were last accessed or modified. This has not been implemented yet, because it would not have been useful to my imediate needs.
I do not mean to insinuate that my systems are perfect or even the complete answere to the question of extended fs support. But it seems to me that they are a good start.
What do you think?
Alex
Question
Hi.
I have several quetions about your oodb.
As I understood, every type of object can be
stored. Are there any limitations?
How can the system determine the type of an
object without having access to the class
signature (e.g. a root-user creates a new class,
saves an instance with world-read-access and the
signature - or the class file - is saved with
access restrictions)?
If any type of instance can be stored, how have you
handled something like this:
class C {
C c;
public static void main(String[] args) {
C c1 = new C();
C c2 = new C();
c1.c = c2;
c2.c = c1;
}
}
How do you prevent the system from writing recursively
every referenced object (and in this case infinitely many)?
How do you load them from disk?
I'm sorry for some stupid questions but I read the thread
about filesystems and thought about a purely oodb as a
filesystem also and hacked down some code.
I'd like to particpate in your work if possible - if
I do not put you off with my questions
best regards
Benjamin
Answer
>As I understood, every type of object can be
>stored. Are there any limitations?
Because the system will use either an ObjectStream or an XMLEncoder an object can be persisted whether it is Serializeable or not. The only structuaral limitations are that the subject class must define a public null (empty) constructor, and that no methods in the class may define more than five arguments.
The first requirement is unpreventable, the proxy object that is created does not need any state from its super class, so we use a null constructor.
The second requirement comes from the fact that java byte code handles differently methods with more than five arguments. This difference is encounter by the asm ClassVisitor (asm.objectweb.org) that is used to create the proxy classes. With some careful (and long) work this requriement could be overcome.
>How can the system determine the type of an
>object without having access to the class
>signature (e.g. a root-user creates a new class,
>saves an instance with world-read-access and the
>signature - or the class file - is saved with
>access restrictions)?
At the moment there is no concept of a world-read-access for the db. When software needs to exchange data with software it should do it directly not by bouncing it off of the storage medium.
The only reason for multiple applications to have access to a singe datum is when that datum represents user-data. The user wants to be able to access his data from many different applications.
In my systems this proplem of global access is solved by using the attribute system that sits on the oodb. When data is saved using the attribute system, only the attribute system can access that data. But data can also be loaded from the atribute system. This makes for a user-driven global access method.
>If any type of instance can be stored, how have you
>handled something like this:
>class C {
>C c;
>public static void main(String[] args) {
>C c1 = new C();
>C c2 = new C();
>c1.c = c2;
>c2.c = c1;
>}
>}
>How do you prevent the system from writing recursively
>every referenced object (and in this case infinitely many)?
>How do you load them from disk?
Data is persisted and recovered using either an io.Object*Stream or a beans.XML(De|En)coder. Both of these (sets) of classes handle duplicate objects in the reference tree by making a special backreference into the already-written stream. Thus circular refferences are handled transparently.
>I'd like to particpate in your work if possible - if
>I do not put you off with my questions
That is excellent. Please contact me here.
Hmmm
I agree that there are some simple ways to narrow the search, however even after doing that we could still be left with too many objects for an exhaustive search to be satisfactory. There should be some method of indexing as with a database. I suggest taking a look at JISP to get a feel for one way this could be achieved.