Some views from a programmer [1/3: requirements]

Daniel Latrémolière daniel.latremoliere at gmail.com
Wed Sep 16 12:58:07 EDT 2009


I would like to add my two cents in this discussion with my 
understanding of modularity and my concrete use cases (hoping they are 
general). Please note that this mail does not study how to achieve 
compatibility with current code, this is the subject of the next mail. 
For me, the first step is defining the objectives of modularity, which I 
can be express as follows.

Simplistically (time dimension omitted), I will tell modularity is 
allowing better understanding and development of complex software: I 
think this can mostly be currently achieved by good packages naming and 
JAR packaging. The only new feature I can found useful for me is a new 
access right "module" for all types, methods, fields to simplify and 
enforce API notion (hiding items not in API).

Rigorously (time dimension included), I think the master word of 
modularity is avoid disruption of incompatible versions of a library and 
enable a smoother path of evolution in software (hiding items not public 
can be seen as a part of this problem for simplifying the problem). The 
software evolution can be split in two parts: source and binary.


      Smooth source evolution

Smooth source evolution need to be able of removing old bad code. Please 
see an example of what I would like to be able of writing (example given 
on jdk libs for clarity):

| // first version of module jdk8
package java.lang;
class System {
    // same API than currently without gc() method.
}

// last version of jdk7 (adaptor to jdk8 for compatibility)
package java.lang;
import jdk8.java.lang.management.MemoryMXBean;
class System extends jdk8.java.lang.System {
    void gc() {
        ManagementFactory.getMemoryMXBean().gc();
    }
}
// NB: If you add requirement that System class is intrinsically 
singleton (no extends allowed),
// you will create a jdk7 adaptor class on jdk8, by having delegate 
methods for all preserved System methods. |

Then, source evolution need, for me, support addressing different 
versions of a module in the same code. By example, clean evolution need 
to remove deprecated API but providing at the same time an adaptor for 
the old version of module using the new version of module. For allowing 
this, it is needed to change practices for code evolution. Currently the 
real code is in deprecated code (for compatibility) and new code call 
old code. I would like from modularity to reverse this situation (for 
smoothing evolution) and put real code in non-deprecated methods while 
throwing really the deprecated code: this is my biggest interest in 
modularity.

By allowing the last version 1.x to call version 2.0 of a module, it is 
possible to have a clean code in version 2.0 because the life cycle of 
version of 1.x will begin with the true code and finish by being a 
simple adaptor on a cleaner module (2.0). If the module author allow the 
code of the adaptor to be distributed (more honestly: give it to the 
customers for not having to support the thrown API), the adaptor can 
even (with some user help) be inlined by IDE in user code for upgrading 
his code from using module 1.x to using module 2.0 (cf. annexe for some 
possible annotations useful for giving hints to IDE).

But, for allowing two different versions of a module in the same code, 
it is needed that, in each class, the declarations of imported class 
will need to be global (and not simply a class in module identifier) 
like "import jdk7.java.util.Map;" in classes (I have imported, in this 
example, a module named jdk7). A tool will need to be configured to 
resolve the module and declare it as the dependency with a specific name 
of a module: cf. infra discussion on Application management.

With this modularity, the performance evolution will have the same 
inversion than compatibility support. It will allow full speed for clean 
new code and slightly hampering old code (slower & fatter) having to use 
adaptors before migration to new library (inlining of adaptors). This is 
not a problem because old code has been written for old (less 
performing) computers, then performance evolution is sufficiently smooth.

Please note, that with this modularity, a part of evolution of the code 
will be only automatic inlining of an adapters classes. This will 
probably need less work for developer (faster to make), create less bugs 
(faster in test), then go faster in production, allowing smaller and 
better cycles in program evolution. I don't count how many Java 
programmers dislike having to develop for an old old old Java release, 
then this is clearly an interesting feature for unifying Java community 
by reducing the community split between versions of Java (small 
individual development being more up-to-date than big companies with 
heavy history of code).


      Smooth binary evolution

I think smooth binary evolution need to have a module system supporting 
loading and unloading each module separately and supporting different 
versions at the same time (given different modules can ask for different 
versions of a same module). I think, even if it is complex, a possible 
future need will be the replacing of a living module without disruption 
(no current need but it is useful to allow the future possibility, cf. 
annexe for some requirements for this possible feature).

I am not sure than average Java programmer understand ClassLoader, by I 
am sure they are understanding [and a large part of their customers] the 
difference between restarting an application and hot replacing only a 
module [and the corresponding quality of service] and the possibility of 
using modules asking different versions of the same module.

Given that in Java a Class can not be unloaded without its ClassLoader, 
it is needed to be able of having at least one ClassLoader per module. 
For the same reason, parent ClassLoader is probably a problem because 
unloading parent will probably unload childs. Given that I cannot give a 
definition of a ClassLoader if it is not the module loader, I will ask 
for one and only one ClassLoader per module. This provides a simple and 
currently working solution for allowing different identities for classes 
with the same qualified class name but coming from different modules.

My only asked feature would be to not block this possible evolution, 
then having one ClassLoader per module had to be a possibility for a 
real application management (but one global classloader can be allowed 
for a simple application manager like Classpath if useful). Another 
feature currently needed is to have a simple API for replacing a module 
with all others modules using it (not useful for Classpath application 
manager).


      Application management

I think it is an error to speak of "dependency management", because it 
is only one part of application lifecycle. I would prefer a term like 
"application management". By example, given an application, smooth 
binary evolution can (in the future) need to change a module in JVM 
without changes in declaration of dependencies (when developer decides 
it is time to upgrade a library). This is an application management, in 
the same sense than JMX.

You can see that in many complex software: the integration (global 
functional tests and packaging) is a project itself (e.g. Linux 
distributions). Here is another example of difference: an application 
provide a set of features and is, in many complex projects, time-based 
for releasing. A module provide a distinct set for functionalities and 
had to be mostly functionality-based for releasing (if the module has a 
clear objective and is not an aggregate of utilities, it will have 
steps, one for each improvement).

When you use a module with support of plug-ins, like Commons VFS, you 
cannot define absolutely in the code of Commons VFS or in your module 
the dependencies used, because it will depends of final use. Will you 
need Commons Net for FTP, or Commons HttpClient for HTTP? you don't know 
before the last step of integration and URL configuration in 
application! Last example, how would you tell that under Java5 you need 
JAXB API, but not under Java6? (and in this case, that is simpler 
because you are not overriding a JDK API)

When beginning to work with some sort of module repositories, I have 
hoped to have all dependencies automatically managed, but now I think 
that application management (repository of Ivy descriptors in my case) 
is not perfect but is clearly better and is adapted to my use cases, 
even if my modules descriptors has scopes reflecting mostly my 
development context. Then I think dependencies does not have to be 
managed by the module system. It is out of his capabilities, because 
dependency management is a global (all the application) problem and not 
a local (to the module) problem. In my experience, I have any number of 
scopes allowed (contrary to Maven [1]), but I see that these scopes 
reflects mostly my practices of development (uses of module) and not the 
internals of module.

Without dependency management, module system need only to expose 
methods, to the application manager, for giving it the code of a module 
and declaring the module who is the dependency named of another module 
Then the only need for module system is a module unique identifier 
(usually made using three Strings: organisation, module, version) and a 
local name for each dependency of a module. The names of dependencies 
are only anchors in module where the module management plug a real module.

There is no defined syntax for these three Strings used as module 
identifier: the syntax is a free task for an external application 
management tool with his own configuration repository - green pages - of 
descriptors like Ivy or another. From the module system point of view, 
this is only an identifier and only equality test is needed for binding 
a specific module as the named dependency of another module (following 
directive of application management tool).


      Conclusion

For me, modularity need to improve evolution of current programs and 
requires a "module" access right for hiding elements (simplification of 
problems). Source evolution need a change in identity of class: the 
qualified class name becomes a local to the module identifier and not a 
global identifier of class (like currently in binary where qualified 
class name is a local identifier relative to his ClassLoader instance). 
Binary evolution needs that dependencies are only named but not resolved 
in the context of module definition. Then the module declaration had to 
provide a list and dependencies given only by names (the anchors) to be 
resolved by application manager (integration step).

Modularity add an immediate need of an API for managing module 
dependencies and [not necessary in JDK] annotations for giving hints to 
IDE to inline code [see Annexe]. Modularity add probably a future need 
of improving Instrument for adding specific rights of modification for 
only one module and Management for managing resources used by each 
module. It will probably increase a future need of a secure API for heap 
visiting (like JVM TI but as Java classes, cf. annexe).

End of my use cases and needed features consequently: hoping to have not 
too many incoherences, having expressed a simple realistic target for 
the present and letting room for future possibilities of evolution. 
Thank you for reading this mail and please excuse me for all errors of 
English. The two following mails are only for detailing how to achieve a 
good compatibility with current Java source code and binaries and some 
annexes cited in the core of this mail. They can only be useful if you 
find interest in this mail.


      Bibliography and footnotes

[1]: 
http://maven.apache.org/guides/introduction/introduction-to-dependency-mechanism.html

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://cs.oswego.edu/pipermail/jsr294-modularity-observer/attachments/20090916/42f65cff/attachment.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: daniel_latremoliere.vcf
Type: text/x-vcard
Size: 199 bytes
Desc: not available
URL: <http://cs.oswego.edu/pipermail/jsr294-modularity-observer/attachments/20090916/42f65cff/attachment.vcf>


More information about the jsr294-modularity-observer mailing list