JNode and Apache Geronimo GSHELL

The GSHELL API is an independently developed API for running commands in Java.

http://cwiki.apache.org/GSHELL/

WANTED: Universal API for all kinds of commands.

Please allow me to explain what I really want. I want one and only one API for all kinds of commands. Why? Because I want to invoke a shell script that blends all kinds of commands together, blurs the distinction between many different SPIs.

What exactly is an API here? An API is only the part that enables you to invoke a command. Actually, it does not necessarily include a specific command interface.

In a more universal model, JNode Command API becomes a service provider interface. A command implements the Command interface of the SPI, not the API. Any command that implements the JNode command interface is compatible with the universal model.

With equal status, GShell Command API becomes a service provider interface. Any command that implements the GShell command interface is compatible with the universal model.

Therefore, anyone who writes for GShell is writing for JNode. Anyone who writes for JNode is writing for GShell. (Although they many not realise it.)

A universal command API is given a command line. It uses the JNode Command API to invoke a JNode command. It uses the GShell Command API to invoke a GShell command. As a user, I get a very large collection of commands. The API does not care if a command is JNode or GShell.

What prevents the pursuit of such a friendly API? Unfortunately, it is difficult, if not impossible, to compile and jar the JNode Command API as a separate and distinct product. This is a necessary first step to building JNode Command API as an implementation of a more universal service provider interface.

The build process for JNode seems to be one step, one giant compile. Is it possible to build the classpath/openjdk separately? Is it possible to build JNode into a collection of smaller, less tightly coupled, independent jars?

JNode is going to be self-hosting some day, isn't it? In order for JNode to be self-hosting, it is difficult, if not impossible, to build the entire operating system when only one part changes.

I am still struggling to build JNode Command API on Microsoft Windows. And yet, building GShell Command API was a walk in the park. While the actual Java code in JNode is far superior, I'm stuck using GShell because its build process is simpler.

OSGi 4.2 is trying to standardise a command API

Looks like OSGi is going down this path of standardising a Command API for use by all OSGi vendors for shell commands. So Felix and any other OSGi projects will merge together into a single API finally. GShell might follow since their situation is kind of similar. JNode is still an OS... but one could argue that an OSGi container is practically an OS. Eye-wink

Details are in the draft: http://www.osgi.org/download/osgi-4.2-early-draft.pdf

Basically what they've done is:
* Command methods are normal Java methods and return values.
* Convertor services can convert String <-> Object for different parameter types.
* The shell language is some weird cross between BeanShell and TCL.
* No shell redirection, use "cat" and "tac" instead.
* Commands use System.* streams and the container manages which thread is writing where, and has a service for setting streams per thread.
* Any existing service object can contain commands.
* Normal main(String[]) still works as a command.

The way they've done the convertor stuff seems okay, but notably with their stuff there is no way to say that the parameter after --host is a hostname. :-/

I'm not sure what we should do about this ...

IMO, it is a good thing that someone is (finally) trying to produce a standard in this area, even if the standard is going to be fairly pedestrian.

In theory, we could try to join the standardization process. In practice we'd have marginal influence unless we managed to scrape together $20,000 a year for full OSGi membership. (And even if we did, I can say from experience that folks who don't represent big companies don't really have much influence. Been there and done that in both OMG and JCP expert groups.)

The best we could do if we wanted OSGi to pay attention to our work would be to try to identify key people on the relevant OSGi expert groups and show them JNode. But realistically, I don't think it would work. Another alternative might be for (say) Daniel's company to sponsor us to join OSGi. But $20,000 plus travel expenses is a lot to ask for, and I cannot imagine how one would make a business case.

In addition to the difficulty of getting our ideas in front of the right people, I think it is probably too late. They already have an early draft on public release, and it is unlikely that they will make a major change in direction at this stage.

So all in all, I think that the best thing we can do is to 1) wait for the OSGi specifications to mature, 2) see if they are any good, 3) see if they get significant uptake, and then 4) decide if we want to implement them in JNode.

In the meantime, it should be "business as usual" with implementing and enhancing what we already have in the JNode codebase. We've already come a long way in the shell / command area and there are lots more interesting ideas to pursue.

GShell is not an operating system

GShell is and has always driven to be a framework to allow Java developers to easily create rich command-line applications. Its not much more than that. Its also not intended to be a universal command-line interface, as IMO Java's public static void main(String[]) seems to handle that quite well. GShell however provides developers some benefits to share code, make IoC driven commands, i18n aware with simple command-line parsing all integrated. Over the months I've been trying to strip down the deps to make it more accessible, but really if folks are using GShell, please ping the development list and let me know, I'm more than happy to take on input and alter the system if it makes sense.

But IMO... jnode and gshell are not really comparable at all. yes, the gshell docs are lacking, because its basically me, and I'm using gshell to facilitate Apache Geronimo needs. What is lacking is community to diversify the project for all of the use I've not considered at all.

All that said, I've yet to look at JNode's command API... I might look it over. But I have to also point out a big difference... the license of JNode. So even if there was some consensus on a command API, ASL-based projects like GShell can't use anything GPL or LGPL (unfortunately). This has caused a lot of problems, in the past an now in the present. I'm on the fence with this, as I under stand some of the reasons to go GNU, but practically GNU tends to limit reach, limit use and prevents organizations to use GNU-license products based solely on the fact that its GNU. Many organizations _solve_ this probjem with a dual license, which usually amounts to... this is free software unless you are commercial, then you have to use our commercial license, which usually means go pay someone money. Hey we all like to make money, but BSD/ASL licenses... are quite different.

I write a lot of software, I want anyone to use it... whenever however, whatever they do... no strings... take my work, change it, do whatever, sell it, blah, I don't care. Its licensed under Apache...

I'm not sitting here scheming on how to make money... I just like to write software, I enjoy working on GShell because I've dreamed of something like this for years and years.

Anyways, I'll shutup in a second, I just happened to google for "gshell" today and see who else was using my software for fun... and well, I'm tickled that I found this, though wondering why I didn't hear about it on our lists. Oh well... no worries.

Happy holidays to all.

BTW, nice of you to cap GShell properly Eye-wink

Cheers,

--jason

Where are variables for a JNode command?

GSHELL provides full support for passing key-value pairs to a command. Its "variables" are something like environment variables in POSIX. This mechanism enables a calling program to pass key-value pairs to a called program. It also enables a called program to pass key-value pairs to a calling command.

The Isolates API provides a simple mechanism for passing key-value pairs to a command. Its properties are something like environment variables, too. This mechanism enables a callind program to pass key-value pairs to a called program.

Where are variables for a JNode command? Key-value pairs are maintained by a shell. The set command sets a key-value pair. How are these variables passed to a command?

Please make yourself contactable

I wanted to send you a message, but you haven't made yourself contactable.

Sorry for the delay.

I have followed your instructions ( http://www.jnode.org/node/2668 ) to make myself contactable.

Could you please explain how to view a contact page of a member?

Click on the user's link and

Click on the user's link and then select the "Contact" tab to get a page to send a message.

You can use "Search" to find a user's link if you can remember their name. Alternatively, the link should be visible on various book, forum or comment nodes they have created.

Using parts of JNode

I believe that the Apache Geronimo project considered their options and have no other choice than to develop yet another independent API for Java commands. Why? Because the good work that has already been done on other projects, such as JNode, cannot be easily reused in a real world application.

First, a Command API must be the best of breed. It must be crisp, suited to one and only one purpose. It must be clean, solve a problem simply and directly. It must be unencumbered, developed independently from all other APIs. It must be better than all of the other Command API alternatives.

Second, a Command API must be use-able in real world applications. It must be used in real world applications in order to be improved to meet the needs of real world applications. It must be stable, improve-able without threat of breaking other people's classes. It must be documented, distributed, maintained and supported.

Lastly, a Command API is a critical part of an operating environment. A platform-independent Command API is a critical part of a real world application strategy. A Command API that only works inside of JNode is severely limited, no matter how technically advanced it might be.

Should JNode depend upon GSHELL in the same way that it depends upon the Isolates and other APIs? Is it possible for JNode to provide a GSHELL compatibility layer--so that any GSHELL command can run without modification in JNode?

Noted ...

Judging from the website, I must say that JNode's shell and command infrastructure already appears to do a lot more than GSHELL in some areas, and appears to be more mature. Your implied criticisms of JNode will be addressed in the fullness of time. For example, the Command API will be ported to other platforms ... when I've addressed other more important issues.

There is a major distinction between the GSHELL APIs and the Isolate APIs and others like log4j. The isolates APIs are defined by a JSR and log4j is a defacto standard, but GSHELL is neither standards-based or defacto standards. So a decision to support GSHELL in JNode requires some hard evidence that this not just another little project that nobody is going to use. A recitation of claimed [*] properties of GSHELL is simply not good enough to convince me.

Of course, if you are volunteering to do the work to build JNode / GSHELL bridging code ... or whatever ... I'm sure that we can accommodate any software that you produce. And of course, if GSHELL does take off, we could always decide to do the work ourselves.

[* For example, your implication that GSHELL is better documented, maintained and supported than the JNode equivalent is NOT borne out by what I saw on the website.]

Bridging to Program API

I am working on building a bridge. I have a few hundred commands (programs) written for the Program API. I want to run all these commands on JNode.

Let me describe a bridge. A bridge is a mechanism to create compatibility where there was none. A Program-Command bridge enables me to create a Program and run it as if it were a Command. A wrapper class implements the Command interface and executes a Program.

Let me describe a command line interface (in Java). A command line interface is responsible for mapping a command name to a command class. When a command class is-a Command, there is no need for a wrapper. When a command class is-a Program, the program is wrapped and then invoked as if it were a Command.

Therefore, a few hundred, fully tested Programs become immediately available as Commands on JNode.

Interesting ...

What are these few hundred programs that you speak of? What do they do? Where can we read about them?

Do you propose to add them to the JNode codebase, or do you have some other idea as to how a JNode end-user would install them?

Are these your programs, and what is their current licensing? Can they be relicensed as LGPL?

(I expect that you already know that the JNode codbase is currently under an LGPL license. The consensus of the active developers is that we should move to a GPL license, but we haven't been sufficiently motivated to make this happen. For a start, since copyright to the JNode codebase is retained by the respective authors, we'd need to track down everyone and ask for their permission to change the license. If someone objects, we'd then have to figure out if we can replace their code.)

My favorite program is packageaudit-scan.

Today, my favorite program is packageaudit-scan. This program, for an example, generates a list of Java package names. Given a URI, it scans for .class, .java, .jar, .zip and .rpm files. When it finds a .java file, it parses the source file to extract a package name. When it finds a .class file, it uses the BCEL library to extract a package name. When it finds a .zip or .jar file, it (recursively) scans the ZIP archive-compatible archive for .class and .java files and extracts the package name. When it finds an .rpm file, it (recursively) scans it for .class, .java, .jar and .zip files.

And yes, the JNode project has been scanned.

http://www.gchii.org/packageaudit/search?option=package&search=%25jnode

We have scanned Fedora Core 9. We have scanned the entire JPackage Project. We have scanned the entire Maven 2 repository.

The Package Audit product is licensed under the GPL. For more information, see also http://cjos.sourceforge.net/html/PackageAuditPages.html .

"consensus in moving to GPL" ? ? ?

This comes as a serious surprise to me because I knew only about the consensus of keeping the current LGPL licensing.

Anyway, let's don't make this beginning of a new debate on licensing because we have better things to do.

It seems I was mistaken ...

It seems I misinterpreted previous JNode discussions on licensing ...

gchii - the take-away for you is that we can only accept the code you are talking about if it is licensed (or can be re-licensed) as LGPL. AND, this is not likely to change any time soon. I hope that doesn't affect you interest in working with JNode.

Regrettably, not what I meant to imply at all

I'm so sorry that I am unable to express myself clearly. I express an amount of sadness. No, I did not mean to imply that GSHELL is better than JNode. You are absolutely correct; I agree with you: GSHELL does not have better documentation, maintenance and support. I also agree that JNode already does a lot more than GSHELL. It is more mature.

I compare JNode to a highly idealized Command API that I would like to use. Of course, it does not yet exist. Given time, this is what the JNode Command/Shell API can be.

Well...

A lot of these problems came along because there was no Command API in Java itself. If Sun had given us some proper entry point into an application instead of what they gave us, which was:

  • static method
  • static references to err and out streams

And I would hope that if Sun were to do it all again, they would make the main entry point a plain old Java object with an execute(String[], Context) method on it.

I think things would be a lot happier everywhere if they had done it from the outset.

I'm about to drop a (large) mail on the Apache Felix users list about their own (yes, a new one) Command API, in the hope that we can reduce the number of these things... Felix is a little unique though as their shell doesn't (yet?) have standard input.