Reformation
Submitted by jalynch on Mon, 06/14/2004 - 13:23.
I would like to thank every one for your advice and references lately.
I am currently working on several ends of my projects in order to make them more flexible and compatible.
I experimented with my transparent persister and replaced io.Serialization with beans.XMLEncoding. The best thing about xml is very resilient to class definition changes. However XMLEncoding is unfamiliar in this area, requires the formation of bean properties (get & set), and is inificient at managing some types of data (particularly byte[]). So I have widened my project to support both serialization, and xml encoding. A mode recomendation is passed to the persister to indicate which to use. If no mode is specified (or if auto is specified) the object will be serailized if it is Serializable and encoded if it is not.
I have also undertaken to remove the transparent persister from the plugin manager and make it a plugin. This has the benifit of largely reducing my contribution to core size. My plugin manager will manage its own data from now on.
Next I will remove the networking funtion from the plugin manager. This will also greatly reduce core size (because the networking is using jGroups). It place of the direct networking, I will create a hook feature that will allow installed plugins to register as PluginDetectors for the PluginManager (hook themselves to the manager by class name and version). When the manager cannot find a class localy, it will lookup the hooks within itself (with the classname/version) instanciate the resultent class and call locatePlugin(String classname,Version v). This method will return an InputStream to a plugin containing the wanted class, or null if it cannot be found.
This system will allow for multiple hooks to be added to the manager, for increased flexibility. I will redesign my peer net as a plugin that will hook itself to the manager. This feature will also allow me to correct an outstanding bug in my ObjectProxyFactory. (a class loading problem).
I will also add a much needed startup signal to the PluginManager. plugins will need to register their ServiceAgents with the addStartupListener(). On the revers, any plugin that has been loaded, and declares a ServiceAgent will recieve a shutdown signal.
Finally is the ConfigurationTool. I have recieved two excellent references from Zsombor, one for a jmx implentation (actuall manager) and one for a viewing console. I am investigating the licences now. Both are very nice and both will require some work to port to my PluginManager (which is to jNode).
What do you think?
Alex
- Login to post comments
PluginManager
I have now completed the third major version of my plugin manager.
This new version has no dependecies on external systems, and is thus aproximately one sixth the size of my last version.
These are its features:
1) Dynamic Install, Uninstall, and Update of plugins.
2) A plugin may register to recieve signals for Install, Unistall, Startup, Shutdown and Update.
3) Support for multiple versions of a plugin (with inteligent Update signal)
4) Support for PluginLocator hooks, which are used for lookup-time detection and installation of plugins.
Before I give details for these features I need to explane the structure of a plugin. The plugin is a jar of classes. The classes must be in a package directory (you can't have lib infront of org.jnode...). The jar must also contain an xml descriptor and a reference to it in the Manifest, for example
Descriptor: jar/Path/To/XML/Descriptor
The xml descriptor conforms to this dtd:
!ELEMENT plugin (dependencies*)
!ATTLIST plugin
title CDATA #REQUIRED
provider CDATA #REQUIRED
version CDATA #REQUIRED
service-agent CDATA #IMPLIED
!ELEMENT dependencies (dependency+)
!ELEMENT dependency EMPTY
!ATTLIST dependency
class-name CDATA #REQUIRED
version CDATA #REQUIRED
First notice that dependecies are declared by class name and version. These values are used to lookup a plugin at run time. It is not neccessary to add a dependency element for every class the plugin depends on. If in the course of class loading a class that a plugin depends on cannot be found in the plugin, and that dependency is not declared in the descriptor, then the class will be sought in the plugin manager by name alone. The fist class of that name (of any version) will be returned.
Thus you see that you only need to specify a dependecy in the descriptor if you need a specific version of a class.
Note that it will soon be possible to declare dependecies on packages, like:
dependency class-name="org.jnode.somepackage.*" version="4"
The next important thing to notice about the xml descriptor is the service-agent attribute of the plugin element. service-agent is implied and need only be declared by plugins interested in receiving service signals. The value of service-agent should be the fully qualified name of an subclass of ServiceAgent.
ServiceAgent is abstract and declares the following protected abstract void methods:
installService();
uninstallService();
updateService(ServiceAgent[] oldAgents);
startup();
shutdown();
Any plugin that defines a service-agent will recive the following signals:
installService() is called ater the plugin has been installed. startup() will not be called at this time by the plugin manager, but the implementation could easily call it.
uninstallService() is called before the plugin has been uninstalled. shutdown() will not be called at this time by the plugin manager, but the implementation could easily call it.
updateService(ServiceAgent[] oldVersions) is called after the plugin has been installed, if and only if there are older versions of that plugin already installed on the system. The implementation may take any action at this time including unistalling the old versions. install() will not be called at this time by the plugin manager, but the implentation could easily call it.
The final two signals are conditional.
shutdown() In order for a plugin to recieve a shutdown signal, the plugin must have been loaded and its classes still in use by the system. If this is the case at shutdown time then the plugin will recieve the shutdown() signal.
startup() In order for a plugin to recieve a startup() signal, it must specifically register as a startupListener via the addStartupListener method in the plugin manager. A listeners validity is checked by addStartupListener and if good the listener information is persisted in a vesion safe reflective form. (More information on VS reflection later.) Any plugin registered as a startup listener will recieve a startup() signal at system startup. It is recomended that a plugin register as a startup listener in install() and deregister in uninstall().
That is the descripton of most of the funtion of the plugin manger.
The only thing left to descuss is version safe reflection. Normal java reflection that tries to use two versions of a class (from two different plugins) will most likely fail. This is because the classes were loaded by diferent loaders and will not be equal(). All normal reflection checks Class equallity before proceding.
This of course could only occure accross plugins. The internall class-state of a plugin remains consistent throught its lifetime.
A situation where things could go wrong is this: plugin A depends on version 1 of plugin B, but plugin F depends on version 2 of plugin B. If either plugin A or F tried to perform reflection on a Class instance from the other (via method param) an exception would be thrown.
Another issue is persisting reflection objects like Class, Method, and Field. I don't know what part of these classes are serailized but it won't be what version of plugin they came from. Infact they are probable serialized with just names, which means that the objects may be reconstructed but there is no certainty of which version they are.
I will begin work on VS wrapper classes to correct these problems.
So, what do you think,
Alex
Plugins Dependencies for Services Startup/Shutdown
Hi Alex,
I've read your post, and I think you are doing a really very good job.
But do you are also able to give to any Plugin a list of dependencies for a correct startup/shutdown of a Plugin ? In Win2000 I can do this for ex. when I need a service that requires another service (application server needs the db service instance before), and the reverse in shutdown of the service. And to have a dependency with other things, like the network interface (to force a reload of the service after some modify to the config, for ex.) or to the xinetd daemon ?
An, is it possible to put a Service in a non-auto-startup/disabled status (in Win2000 it's manual instead of automatic) ?
Bye,
Sandro
Dependencies
Thank you for your support.
>But do you are also able to give to any Plugin a list of >dependencies for a correct startup/shutdown of a Plugin ?
Specifically defining statup and shutdown procedures is not possible, nor should it be neccessary. Plugins that register to recieve the startup signal are loaded at startup time. Any plugins that they depend on by class definition are loaded as they are referenced. Plugins only stay loaded while they are strongly referenced by an active thread. Once they are weakly reachable they fall to the garbage collector. After this happens the plugin can again be safely loaded.
Thus startup occurs by a chain of references, and unloading occurs as plugins are no longer needed.
A shutdown signal is given to every plugin that is still strongly referenced at shutdown time. A plugin reciving shutdown should not dismantle or finalize itself. Niether can a plugin unload itself. All that should be done is for the plugin to stop (safely) any internal threads that would keep it and other plugins that it references strongly reachable. If all of the plugins stop their threads then no plugin will be strongly reachable (that is they will only be reachable by other plugins that are not strongly reachable, and so on) this produces the unloading of every active plugin. The memory held by those plugins may be cleared by the gc, though that may not be neccessary if there is no other shutdown code.
>And to have a dependency with other things, like the network >interface (to force a reload of the service after some modify to the >config, for ex.) or to the xinetd daemon ?
Plugin configuration should be done in a oop way. Method calls should change the state of the pluign object, and those changes should be persisted. This means that the plugin does not need to be reloaded after being configured.
One idea that has been stated for this configuration system, is to allow a plugin to specify a specal class that represents its configurable properties. An instance of that class would be managed by the plugin manager (or associated core system) for the plugin.
If the plugin referenced that configurable object (dynamically not just at startup) then any changes to its state would be reflected in the operation of the plugin.
>An, is it possible to put a Service in a non-auto-startup/disabled >status (in Win2000 it's manual instead of automatic) ?
This is considered the default setup. A pluign is only loaded when it is needed, unless it has registered for the startup event.
What do you think?
Alex
Good
Hi Alex,
what you say is good.
At the moment I know JNode only a little ... what I was saying probably is more related to Services, but if it's handled like you do with Plug-ins should be ok.
Bye,
Sandro
Class Versions
Hi Alex,
I would like to eliminate the class version. That can only be acheived by introducing some restrictions / rules that prevent new classes become incompatible to the old one, so here is some brainstorming about getting rid of handling multiple class versions:
I.e. it's perfectly possible for a new class version to add methods. But it can't remove methods. (Changing methods is like removing the old method and adding a new one in one step.) Why can't methods be removed? Because other classes might use them. So we must make shure that no other classes are still using the methods we want to remove. If this is the case we don't need to provide the ability to handle multiple versions of a component / class / whatsoever and just use the latest version.
Ok, but how do we ensure that no other classes use the methods we want to remove? We deprecate them (and include the version when they will be removed in the deprecation tag, of course). The every user of the class will know the method will be removed.
Please comment!
Regards,
Sebastian
Versioning
Honestly, I dont think this is possible. Not having to worry about versions at all would ba a major benifit to a developer and would also make the pm implentation easier, however I dont think this can be done.
Firstly I dont think that the deprication modle is feasible. Code sources are not examined and byte code is not recompiled so a deprication warning would never be exposed on a client system. The only party that would see the deprication would be the developer of a plugin that depends on the depricated method. But that would only occur if the developer wanted to make a new version.
If the developer never tried to make a new version the original version would be forever dependent on the depricated method, and thus we could never safely update the class of that method.
More than this is the idea of plugin versions. Suppose a developer creates a plugin that provides a graphical spreadsheet program. Next suppose that, later, the developer creates a new version of the plugin with advances features and bug fixes. Finaly suppose, that the user of a jnode system wants both version of the spreadsheet installed because, for some purposes, he prefers the old functionality.
How could we install both plugins? If there was no version concept in the system we could not, we would have to choose between the two. And there again may arise the problem of version dependencies.
In summary, I do not think that it is possible to eliminate versioning. It is possible however to minimize its appearence to both the developer and the user. This must be done as much as is possible.
Respectfully,
Alex
Re: Versioning
Hi Alex,
You are right saying when developers never create a new version of their code, they will never notice deprecated methods. But I think most applications and libraries are not simply released in version 1.0 and don't get touched anymore. Normally applications are developed for a rather long time. Also the longer an application exists the more stable the interfaces will become, which will result in less deprecations.
In order to notify developers on deprecations the same communication channels can be used as to notify developers on new features: the mailinglists.
Also I can't imagine a user installing two versions of a spreadsheet or any other application. If a new version of an application omits important features users will probably retain the old version. On the other hand these conflicts should be avoided by communication between users and developers.
Where multiple version will be used most often is in libraries. But they can and do use deprecations to remain compatible to a number of older versions (e.g. the J2SE libraries do so, to a certain extent).
Just some contra again
Maybe we can all collect some use cases to see where multiple versions might be useful / required and where they can be replaced by careful but costlier interface maintenance.
Regards.
Sebastian
Extensions & extensionpoints
Hi Alex,
The current pluginmanager in jnode provides for extension and extensionpoints, which are heavily used by the OS.
Do you provide these also?
Ewout
Extensions
Extensions and extensionpoints are not directly supported in my manager as I do not view this behavor as important.
That being said, If you will explain to me its importance to you and mabey some examples from jNode my oppinion may be changed.
The e/ep behavor could be implemented by any given plugin, or better, a plugin could be developed that provides a Version Safe implentation of this behavor for others to use. Further still you may show that e/ep should be added to the core of my manager.
Let me know and we will discuss it,
Alex
Extensions & Extensionpoints
The reason they are important, it that they are the entire foundation of the JNode OS. Extension points are the places where functionality from different plugins come together. E.g. the driver framework uses them to "register" drivers for specific devices.
So this is not something that can be easily changed, and also something i don't want to change that easily.
Ewout
Submit
Ok. I will submit to The Admin. I will now begin the forth phase of my development. I will modify my manager so that it will fit into jnode, and use all of jnodes existing plugins. I will integrate jnode's security protocal to my work and alter my descriptors to match the existing format. I hope to report soon.
Alex