QDox
  1. QDox
  2. QDOX-140

Parser fails to translate Unicode escapes

          Details

          • Type: Bug Bug
          • Status: Closed Closed
          • Priority: Major Major
          • Resolution: Fixed
          • Affects Version/s: 1.6.3
          • Fix Version/s: 1.9.1
          • Component/s: Parser
          • Labels:
            None
          • Number of attachments :
            1

            Description

            The parser does currently not translate Unicode escapes as described in JLS, Lexical Structure.

            If Unicode escapes are used for identifiers, an exception is raised as shown by the attached source file:

            com.thoughtworks.qdox.parser.ParseException: syntax error @[17,37] in .../src/main/java/it/MyMojo.java
        at com.thoughtworks.qdox.parser.impl.Parser.yyerror(Parser.java:716)
        at com.thoughtworks.qdox.parser.impl.Parser.yyparse(Parser.java:826)
        at com.thoughtworks.qdox.parser.impl.Parser.parse(Parser.java:697)
        at com.thoughtworks.qdox.JavaDocBuilder.addSource(JavaDocBuilder.java:300)
        at com.thoughtworks.qdox.JavaDocBuilder.addSource(JavaDocBuilder.java:316)
        at com.thoughtworks.qdox.JavaDocBuilder.addSource(JavaDocBuilder.java:312)
        at com.thoughtworks.qdox.JavaDocBuilder$1.visitFile(JavaDocBuilder.java:369)
        at com.thoughtworks.qdox.directorywalker.DirectoryScanner.walk(DirectoryScanner.java:43)
        at com.thoughtworks.qdox.directorywalker.DirectoryScanner.walk(DirectoryScanner.java:34)
        at com.thoughtworks.qdox.directorywalker.DirectoryScanner.walk(DirectoryScanner.java:34)
        at com.thoughtworks.qdox.directorywalker.DirectoryScanner.scan(DirectoryScanner.java:52)
        at com.thoughtworks.qdox.JavaDocBuilder.addSourceTree(JavaDocBuilder.java:366)

            Unicode escapes in comments (more likely) don't make the parser fail but still need translation.

            1. Java Source File
              MyMojo.java
              0.5 kB
              Benjamin Bentmann

              Activity

              Hide
              Permalink
              Paul Hammant added a comment -

              I've added a test to LexerTest which shows this problem when it hits the JFlexLexer

              Show
              Paul Hammant added a comment - I've added a test to LexerTest which shows this problem when it hits the JFlexLexer
              Hide
              Permalink
              Robert Scholte added a comment -

              The problem seems to be caused by something else. The way java-files are read doesn't trigger the reader to convert them to chars. A backwardslash in the uncompiled java-file is just read as a backwardslash.
              Here's a document about this stuff http://www.xrce.xerox.com/competencies/content-analysis/tools/publis/javacc_unicode.pdf

              Show
              Robert Scholte added a comment - The problem seems to be caused by something else. The way java-files are read doesn't trigger the reader to convert them to chars. A backwardslash in the uncompiled java-file is just read as a backwardslash. Here's a document about this stuff http://www.xrce.xerox.com/competencies/content-analysis/tools/publis/javacc_unicode.pdf
              Hide
              Permalink
              Robert Scholte added a comment -

              r580 | rfscholte | 2009-03-10 23:05:36 CET

              qdox-140 bugfix: unicode support
              ----------------------------------------------------------------------------

              Show
              Robert Scholte added a comment - r580 | rfscholte | 2009-03-10 23:05:36 CET qdox-140 bugfix: unicode support ----------------------------------------------------------------------------
              Robert Scholte made changes -
              Field Original Value New Value
              Status Open [ 1 ] Closed [ 6 ]
              Fix Version/s 1.10 [ 15020 ]
              Resolution Fixed [ 1 ]
              Paul Hammant made changes -
              Status Closed [ 6 ] Reopened [ 4 ]
              Resolution Fixed [ 1 ]
              Paul Hammant made changes -
              Status Reopened [ 4 ] Closed [ 6 ]
              Fix Version/s 1.9.1 [ 15252 ]
              Fix Version/s 1.10 [ 15020 ]
              Resolution Fixed [ 1 ]

                People

                • Assignee:
                  Unassigned
                  Reporter:
                  Benjamin Bentmann
                • Votes:
                  0 Vote for this issue
                  Watchers:
                  0 Start watching this issue

                  Dates

                  • Created:
                    Updated:
                    Resolved: