1*16467b97STreehugger RobotANTLR 3 for Ruby 2*16467b97STreehugger Robot by Kyle Yetter ([email protected]) 3*16467b97STreehugger Robot http://antlr.ohboyohboyohboy.org 4*16467b97STreehugger Robot http://github.com/ohboyohboyohboy/antlr3 5*16467b97STreehugger Robot 6*16467b97STreehugger Robot== DESCRIPTION: 7*16467b97STreehugger Robot 8*16467b97STreehugger RobotFully-featured ANTLR 3 parser generation for Ruby. 9*16467b97STreehugger Robot 10*16467b97STreehugger RobotANTLR (ANother Tool for Language Recognition) is a tool that is used to generate 11*16467b97STreehugger Robotcode for performing a variety of language recognition tasks: lexing, parsing, 12*16467b97STreehugger Robotabstract syntax tree construction and manipulation, tree structure recognition, 13*16467b97STreehugger Robotand input translation. The tool operates similarly to other parser generators, 14*16467b97STreehugger Robottaking in a grammar specification written in the special ANTLR metalanguage and 15*16467b97STreehugger Robotproducing source code that implements the recognition functionality. 16*16467b97STreehugger Robot 17*16467b97STreehugger RobotWhile the tool itself is implemented in Java, it has an extensible design that 18*16467b97STreehugger Robotallows for code generation in other programming languages. To implement an 19*16467b97STreehugger RobotANTLR language target, a developer may supply a set of templates written in the 20*16467b97STreehugger RobotStringTemplate (http://www.stringtemplate.org) language. 21*16467b97STreehugger Robot 22*16467b97STreehugger RobotANTLR is currently distributed with a fairly limited Ruby target implementation. 23*16467b97STreehugger RobotWhile it does provide implementation for basic lexer and parser classes, the 24*16467b97STreehugger Robottarget does not provide any implementation for abstract syntax tree 25*16467b97STreehugger Robotconstruction, tree parser class generation, input translation, or a number of 26*16467b97STreehugger Robotthe other ANTLR features that give the program an edge over traditional code 27*16467b97STreehugger Robotgenerators. 28*16467b97STreehugger Robot 29*16467b97STreehugger RobotThis gem packages together a complete implementation of the majority of features 30*16467b97STreehugger RobotANTLR provides for other language targets, such as Java and Python. It contains: 31*16467b97STreehugger Robot 32*16467b97STreehugger Robot* A customized version of the latest ANTLR program, bundling all necessary 33*16467b97STreehugger Robot java code and templates for producing fully featured language recognition 34*16467b97STreehugger Robot in ruby code 35*16467b97STreehugger Robot 36*16467b97STreehugger Robot* a ruby run-time library that collects classes used throughout the code that 37*16467b97STreehugger Robot ANTLR generates 38*16467b97STreehugger Robot 39*16467b97STreehugger Robot* a wrapper script, `antlr4ruby', which executes the ANTLR command line tool 40*16467b97STreehugger Robot after ensuring the ANTLR jar is Java's class path 41*16467b97STreehugger Robot 42*16467b97STreehugger Robot== FEATURES 43*16467b97STreehugger Robot 44*16467b97STreehugger Robot1. generates ruby code capable of: 45*16467b97STreehugger Robot * lexing text input 46*16467b97STreehugger Robot * parsing lexical output and responding with arbitrary actions 47*16467b97STreehugger Robot * constructing Abstract Syntax Trees (ASTs) 48*16467b97STreehugger Robot * parsing AST structure and responding with arbitrary actions 49*16467b97STreehugger Robot * translating input source to some desired output format 50*16467b97STreehugger Robot 51*16467b97STreehugger Robot2. This package can serve as a powerful assistant when performing tasks 52*16467b97STreehugger Robot such as: 53*16467b97STreehugger Robot * code compilation 54*16467b97STreehugger Robot * source code highlighting and formatting 55*16467b97STreehugger Robot * domain-specific language implementation 56*16467b97STreehugger Robot * source code extraction and analysis 57*16467b97STreehugger Robot 58*16467b97STreehugger Robot== USAGE 59*16467b97STreehugger Robot 60*16467b97STreehugger Robot1. Write an ANTLR grammar specification for a language 61*16467b97STreehugger Robot 62*16467b97STreehugger Robot grammar SomeLanguage; 63*16467b97STreehugger Robot 64*16467b97STreehugger Robot options { 65*16467b97STreehugger Robot language = Ruby; // <- this option must be set to Ruby 66*16467b97STreehugger Robot output = AST; 67*16467b97STreehugger Robot } 68*16467b97STreehugger Robot 69*16467b97STreehugger Robot top: expr ( ',' expr )* 70*16467b97STreehugger Robot ; 71*16467b97STreehugger Robot 72*16467b97STreehugger Robot and so on... 73*16467b97STreehugger Robot 74*16467b97STreehugger Robot 75*16467b97STreehugger Robot2. Run the ANTLR tool with the antlr4ruby command to generate output: 76*16467b97STreehugger Robot 77*16467b97STreehugger Robot antlr4ruby SomeLanguage.g 78*16467b97STreehugger Robot # creates: 79*16467b97STreehugger Robot # SomeLanguageParser.rb 80*16467b97STreehugger Robot # SomeLanguageLexer.rb 81*16467b97STreehugger Robot # SomeLanguage.g 82*16467b97STreehugger Robot 83*16467b97STreehugger Robot3. Try out the results directly, if you like: 84*16467b97STreehugger Robot 85*16467b97STreehugger Robot # see how the lexer tokenizes some input 86*16467b97STreehugger Robot ruby SomeLanguageLexer.rb < path/to/source-code.xyz 87*16467b97STreehugger Robot 88*16467b97STreehugger Robot # check whether the parser successfully matches some input 89*16467b97STreehugger Robot ruby SomeLanguageParser.rb --rule=top < path/to/source-code.xyz 90*16467b97STreehugger Robot 91*16467b97STreehugger Robot-> Read up on the package documentation for more specific details 92*16467b97STreehugger Robot about loading the recognizers and using their class definitions 93*16467b97STreehugger Robot 94*16467b97STreehugger Robot== ISSUES 95*16467b97STreehugger Robot 96*16467b97STreehugger Robot* Currently, there are a few nuanced ways in which using the ruby output differs 97*16467b97STreehugger Robot from the conventions and examples covered in the ANTLR standard documentation. 98*16467b97STreehugger Robot I am still working on documenting these details. 99*16467b97STreehugger Robot 100*16467b97STreehugger Robot* So far, this has only been tested on Linux with ruby 1.8.7 and ruby 1.9.1. 101*16467b97STreehugger Robot I'm currently working on verifying behavior on other systems and with 102*16467b97STreehugger Robot slightly older versions of ruby. 103*16467b97STreehugger Robot 104*16467b97STreehugger Robot== LICENSE 105*16467b97STreehugger Robot 106*16467b97STreehugger Robot[The "BSD license"] 107*16467b97STreehugger RobotCopyright (c) 2009-2010 Kyle Yetter 108*16467b97STreehugger RobotAll rights reserved. 109*16467b97STreehugger Robot 110*16467b97STreehugger RobotRedistribution and use in source and binary forms, with or without 111*16467b97STreehugger Robotmodification, are permitted provided that the following conditions 112*16467b97STreehugger Robotare met: 113*16467b97STreehugger Robot 114*16467b97STreehugger Robot 1. Redistributions of source code must retain the above copyright 115*16467b97STreehugger Robot notice, this list of conditions and the following disclaimer. 116*16467b97STreehugger Robot 2. Redistributions in binary form must reproduce the above copyright 117*16467b97STreehugger Robot notice, this list of conditions and the following disclaimer in the 118*16467b97STreehugger Robot documentation and/or other materials provided with the distribution. 119*16467b97STreehugger Robot 3. The name of the author may not be used to endorse or promote products 120*16467b97STreehugger Robot derived from this software without specific prior written permission. 121*16467b97STreehugger Robot 122*16467b97STreehugger RobotTHIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESS OR 123*16467b97STreehugger RobotIMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES 124*16467b97STreehugger RobotOF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. 125*16467b97STreehugger RobotIN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY DIRECT, INDIRECT, 126*16467b97STreehugger RobotINCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT 127*16467b97STreehugger RobotNOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, 128*16467b97STreehugger RobotDATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY 129*16467b97STreehugger RobotTHEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT 130*16467b97STreehugger Robot(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF 131*16467b97STreehugger RobotTHIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. 132*16467b97STreehugger Robot 133