|
History
| Log In
|
|
|
|
[
Permlink
| « Hide
]
Walter Mundt - 21/Apr/04 04:24 PM
Here's a patch I posted to the mailing list awhile ago when JIRA was down to make the parser not blow up with some of the new generics and for-each syntax. It's pretty basic -- nested parameterized types break it, and there are a few other things it doesn't like much. Better than nothing, though.
Walter,
could you incorporate enum support? I guess those will appear soon in 1.5 sources .. Heiko yep, I want to move to hibernate 3 and java 1.5. Xdoclet code
generation is the only thing holding me back. will somebody apply this patch sometime in my lifetime ? :(
Yannick,
as I don't know how old you are or in what medical condition, I can't answer this one ... This patch will not make it into xdoclet 1.2.2, but will be incorporated afterwards. The patch is incomplete and needs more tweaking I am trying to decide if in future I should use the hibernate
code generation tools or go in the opposite direction and drive it all from my source code. Would you imagine that by January ( when I guess hibernate 3 will be stable ) that xdoclet will support java 1.5/5 and hibernate 3 syntax ? It would be usefull to know this as I could then decide to use hibernate to drive everything or to use xdoclet instead. --b I am sure, that there will be at least a snapshot of a release that supports JDK 5 until then.
But I don't think it is a good idea to either - hold this release until JDK 5 support is fully in - deliver the next release with support for some but not all new constructs in JDK 5. I can't wait until January. It's already been months since this issue was reported and JDK 1.5 throws warnings left and right if you use raw collections classes; rather annoying that. As a workaround, is there some way for me to tell hibernate.doclet to not process JDK 1.5 files? I tried excluding them from the <fileset> but the problem is that the files with xdoclet markups depend on classes that make use of JDK 1.5 and so xdoclet complains about missing dependencies.
Is there a workaround whereby xdoclet marked-up files can remain in JDK 1.4, while depending on other classes that make use of JDK 1.5 and xdoclet will not complain? I apologize if I seem to be pushing for this issue a bit too strongly, but if you guys can spend a few days getting a proper patch in place it would be much appreciated. Don't get me wrong. I did not say "wait until january".
Can't you use '-source 1.4' option to javac? Even if you could do "-source 1.4", which isn't supported in JDK 1.5 as far as I know, it would not help. Remember that xdoclet is parsing the java files, not the class files.
I wish I knew how to sort the open issues by votes because somehow I suspect that this issue has the most votes of all open issues. I know, eight votes is a joke :) but then again, our community is small. I personally view xdoclet as being mostly feature-complete. It would be nice to get it working, as is, under JDK 1.5 before adding new features. Lastly, I don't know what internal parser you guys use, but if it is something standard like ANTLR, I am sure the JDK 1.5 grammar is floating around (other projects require it) and perhaps we can just plug one in. Yes, here are two of them for starters. yet another java 1.5 grammar Michael Stahl Wed Aug 25, 2004 10:23 a recognizer and tree parser for java 1.5, developed independently from the one by michael studman http://www.antlr.org/grammar/1093454600181/java15-grammar.zip Michael Studman Sat Jul 24, 2004 16:51 An update to the Java 1.3/1.4 grammar (incorporating Cortex's generics additions) to support all Java 1.5 language changes. http://www.antlr.org/grammar/1090713067533/index.html I think if xdoclet does not keep up with the language people may turn to attributes instead. I personally prefer xdoclet because I don't really want all that stuff added to the generated class files but I guess it's each to their own. --b Actually I notice that you are using javacc rather than antlr,
no problem the javacc project page has an announcement that they have java 1.5 suppport. https://javacc.dev.java.net/ They don't appear to have a jjtree grammar though. Is this the problem ? If it is the case then perhaps we should be lobbying the javacc guys for java 1.5 grammar so your team can do it's thing. --b If someone would provide support for enums in addition to the existing patch, then this could be solved relatively soon.
I did have a look at it in the past, but this is not trivial. Just replacing the existing grammar with a 1.5 one doesn't help, as the "interfacing between the grammar and xdoclet" would be lost. Anyone who can't wait for xdoclet 1.x to add JDK 1.5/5.0 support might want to take a look at XDoclet 2, as that no longer uses xjavadoc. In fact, the way it's architected the parser is pluggable though the existing modules use QDox (qdox.codehaus.org) to do it. I don't know if QDox supports enums yet, but the latest CVS certainly includes a change for "Implement QDOX-53: Add support for parsing of JSR-14 generic types." The down side is there's fewer existing modules than for XDoclet 1.x.
Andrew,
I remember trying to information about XDoclet 2 a few weeks ago and I could not find anything. The link to XDoclet 2 on the main site was down last time I checked and it wasn't discussed anywhere else. So, where does one find out more about XDoclet 2? Does it have a Hibernate module among other things (this is the only module I am interested in at the moment)? Where does one download a copy of XDoclet 2? Thank you, Gili Here's the link to the xdoclet 2 site:
http://xdoclet.sf.net/wiki However, it's parser (qdox) has only very preliminary support for Java 1.5 features. I just checked out the qdox code from CVS, and although it has some support for generics, it's not complete. For example, it doesn't support the '?' placeholder, as in: public class Bar { private Class<? extends Date> klass; } The qdox parser will barf on that. Also, it seems to have *no* support for any other 1.5-specific features, including enums. For the one impatients of using jdk5 like me, here is a workaround I use to be able to get generics, and still have xdoclet running :
I add a small ant task that parse all my sources with a regex, removing all the generics, and all the declaration of the foreach loops. The resulting code isn't functional, but it's good enough to run xdoclet on it. Note, the regex works fine for me, but it might need to be improved for more complex cases. Right now, the most complex handled case is simething like : <String,byte[]> So, here is the ant task : <target name="remove-generics" depends="init"> <!-- Since XDoclet isn't already able to parse java 1.5 source code, we remove all references to generics and for each loops. --> <property name="generics.match.pattern" value="<[\w\[\]]+(<\w+>)?,?[\w\[\]]*(<\w+>)?>|(for\s?\(.+:.+\))"/> <echo message="Pattern used to remove generics : ${generics.match.pattern}"/> <copy toDir="${build.generate.dir}/java" overwrite="true"> <fileset dir="${src.main.dir}"> <include name="**/*.java"/> </fileset> </copy> <replaceregexp byline="true" flags="g"> <fileset dir="${build.generate.dir}/java"> <include name="**/*.java"/> </fileset> <regexp pattern="${generics.match.pattern}"/> <substitution expression=""/> </replaceregexp> </target> 1.5 is officially out now and I've run right into this problem. Is there any kind of workaround for this issue? Copying all my source files for every compile is not really an option as our project is way too big for that to be practical. The patch provided doesn't offer enough functionality. I use XDoclet for hibernate - surely someone has come up with a better solution?
I sure wish there was. My solution is to not use xdoclet with java 1.5 and hibernate. We maintain our mappings files manually. It's a pain in the ass, with lots of duplication. But there doesn't seem to be much of an alternative, if you require full java 1.5 parsing.
OK, here's a workaround idea (though I know nothing about XDoclet). There has got to be an InputStream or Reader involved somewhere in the process of XDoclet reading the Java source to parse it... so, if we Just had some Java code that could strip generics enough to make the XDoclet parser happy, and we could somehow incorporate this Java code into a InputStream/Reader filter (wrapper/decorator/whatever term you want to use), then it could possibly be a case of simply wrapping the InputStream/Reader before handing it off to the parser... So, the tricks would be:
1. Java code that can strip generics in the form of an InputStream/Reader filter. 2. Enough knowledge of XDoclet source to know how to interpose this filter before handing the stream off to XDoclet. This is basically the same general concept as the copy-all-your-source-and-regex-it idea above, except you don't have to copy all your source (it becomes more efficient as we can avoid invoking the OS and the slow disk IO). Unfortunately, I don't have time to accomplish either of these. :-( Stripping generics, no matter what the approach, is not a workable solution for our needs. We make full use of most of the new 1.5 features, including enums, annotations, for:loops, etc. Specifically, we have hibernate mappings for Enum classes. No amount of generics stripping will solve that problem. Our project needs full java 1.5 parsing support, or enough support to handle generics and enums and ignore anything else that doesn't involve class declarations.
Scott,
What you said makes no sense to me. Generics, enums and most of the other JDK 1.5 features are purely compiler syntactic suger. Hence, there is absolutely no benefit to having XDoclet "recognizer" them as enums, generics etc. If XDoclet sees "Set" instead of "Set<Integer>" that is absolutely fine and the resulting mappings will be valid. Adding JDK-1.5 recognition is nice to have because then this: @hibernate.collection-one-to-many class="someType" changes into: @hibernate.collection-one-to-many and XDoclet is smart enough to auto-detect the "class" by grabbing the generic type. Again, this is just syntactic suger and is not required. If you guys filter JDK 1.5 code into 1.4 before feeding it into XDoclet it'll make me (and others) very happy. You can use the regexp mentioned above as a starting point. So perhaps I overstated. I don't need a full JDK 1.5 parser. But *something* needs to be able to parse JDK 1.5 syntax and convert it into something XDoclet can understand. If that *something* is XDoclet itself (or whatever parser it's using today), then all the better. My point was that "stripping generics" is not good enough for me. I need enum parsing/conversion at least. For now, the incremental effort of making sure our manually-created mapping files are consistent with the classes is a far easier story to tell management then creating, testing, and verifying a suite of "jfront"-style regexes to convert 1.5 syntax into 1.4.
Ok, so far generics -> raw types and enums -> integer constants have been discussed. Are there any other entities that need to be converted for XDoclet to work?
If someone could comment on how easy/hard it would be to handle enum -> integer constants I think we can begin work on this. For me, enums -> integer contants is not sufficient. Our enums are persisted as full blown classes, with properties that map to columns in the DB. We're running a modified version of hibernate (patch submitted) that allows persisting classes that don't provide a default ctor for this very reason. So any solution has to treat JDK 1.5 enums for what they really are: java classes.
Scott,
Agreed, but my point is that XDoclet processes files one at a time. The only time you have to concern yourself with enums is when you're processing their .java file. In such a case, you simply have to swap "enum" for "class" and convert its values into "final int" values and you're done. In fact, I will go further and say that you could even strip the values altogether since XDoclet doesn't care about the actual class implementation, but rather about its commments. So when processing enums, we simply need to replace "enum" by "class" and strip away the values. I'm not sure what the specification says about enums, but from what I've seen, everything between the beginning of the class body up until the first semicolor seems to make up the values. Stripping them should be simple enough. If the specification turns out to be more complicated, we could always enhance our enum support in the future but this should be enough for a start. Hi,
simplest would be if someone would patch the attached grammar-patch to support enums Heiko Is there an open source utility anywhere that will convert the new 1.5 syntax to compatible 1.4 syntax? For example generics to their erasure forms, enums to their class form, etc... if so, it could be possible to interpose this utility (maybe via an inline InputStream/Reader filter) between the file and XDoclet. Of course, the best option is to modify the grammar, but I'm not sure how many people are familiar with how XDoclet grammar is setup, and I certainly don't have the time to figure it out. If there is an already existing project that can do such a conversion, then it may be a good temporary fix. In any case, the ultimate solution, as you mention, is to fix the grammar. Really, at this point I don't care how it gets done, so long as it works. :-)
Here is the fixed version of XJavadoc which can successfully deals with new java 1.5 language syntax.
I'm currently working on one big project (based on java 1.5) and I've been very disappointed with XDoclet inability to work with Tiger syntax, so I've permitted oneself to spend a few days in order to make serious effort in resolving of this scary problem. 1) The old java 1.2 grammar was completely replaced with new java 1.5 (was taken on javacc.dev.java.net) 2) then i've carefully merged XJavaDoc java building routines with new syntax in Java1.5-xjd.jjt (it was tiresome work :)) (in order to build java parser javacc-3.2 was used) 3) tested with all junit test in xjavadoc and with light java 1.5 test + tested in real world application with using java 1.5 & xdoclet & hibernate tags. files: /xjavadoc-1.1-j5.jar - complete binary build of xjavadoc with java 1.5 support (in order to use XDoclet with Tiger you should replace old xjavadoc.jar with xjavadoc-1.1-j5.jar) Affected sources: /afsrc//test/ - simple test routines /afsrc/Java1.5-xjd.jjt - javacc & xjavadoc parser template - the replacement of old - Java1.2-b.jjt /afsrc/Token.java - generated javacc Token.java - should be replaced with this file limitation: enums - are completelly ignored by xjavadoc (also in XDoclets enums is not sipported) Hope it helps you with xdoclet & java 1.5 Adamansky Anton. Anton,
this is cool. I don't understand your comment about Token.java. As this is normally generated on every run .. should the generated one always be replaced by the class supplied by you or is this only meant for initial bootstrapping? | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
Improvement
Resolved
Critical
Support Java 1.5 Syntax ?
