# class RDoc::Parser

A parser is simple a class that subclasses RDoc::Parser and implements scan to fill in an RDoc::TopLevel with parsed data.

The initialize method takes an RDoc::TopLevel to fill with parsed content, the name of the file to be parsed, the content of the file, an RDoc::Options object and an RDoc::Stats object to inform the user of parsed items. The scan method is then called to parse the file and must return the RDoc::TopLevel object. By calling super these items will be set for you.

In order to be used by RDoc the parser needs to register the file extensions it can parse. Use ::parse_files_matching to register extensions.

    require 'rdoc'

    class RDoc::Parser::Xyz < RDoc::Parser
      parse_files_matching /\.xyz$/

      def initialize top_level, file_name, content, options, stats
        super

        # extra initialization if needed
      end

      def scan
        # parse file and fill in @top_level
      end
    end

### Attributes

parsers[R]

An Array of arrays that maps file extension (or name) regular expressions to parser classes that will parse matching filenames.

Use ::parse_files_matching to register a parser's file extensions.

file_name[R]

The name of the file being parsed

### Public Class Methods

alias_extension(old_ext, new_ext) click to toggle source

Alias an extension to another extension. After this call, files ending “new_ext” will be parsed using the same parser as “old_ext”

    # File lib/rdoc/parser.rb, line 57
    def self.alias_extension(old_ext, new_ext)
      old_ext = old_ext.sub(/^\.(.*)/, '\1')
      new_ext = new_ext.sub(/^\.(.*)/, '\1')

      parser = can_parse_by_name "xxx.#{old_ext}"
      return false unless parser

      RDoc::Parser.parsers.unshift [/\.#{new_ext}$/, parser]

      true
    end

binary?(file) click to toggle source

Determines if the file is a “binary” file which basically means it has content that an RDoc parser shouldn't try to consume.

    # File lib/rdoc/parser.rb, line 73
    def self.binary?(file)
      return false if file =~ /\.(rdoc|txt)$/

      s = File.read(file, 1024) or return false

      return true if s[0, 2] == Marshal.dump('')[0, 2] or s.index("\x00")

      mode = 'r:utf-8' # default source encoding has been chagened to utf-8
      s.sub!(/\A#!.*\n/, '')     # assume shebang line isn't longer than 1024.
      encoding = s[/^\s*\#\s*(?:-\*-\s*)?(?:en)?coding:\s*([^\s;]+?)(?:-\*-|[\s;])/, 1]
      mode = "rb:#{encoding}" if encoding
      s = File.open(file, mode) {|f| f.gets(nil, 1024)}

      not s.valid_encoding?
    end

can_parse(file_name) click to toggle source

Return a parser that can handle a particular extension

    # File lib/rdoc/parser.rb, line 131
    def self.can_parse file_name
      parser = can_parse_by_name file_name

      # HACK Selenium hides a jar file using a .txt extension
      return if parser == RDoc::Parser::Simple and zip? file_name

      parser
    end

can_parse_by_name(file_name) click to toggle source

Returns a parser that can handle the extension for `file_name`. This does not depend upon the file being readable.

    # File lib/rdoc/parser.rb, line 144
    def self.can_parse_by_name file_name
      _, parser = RDoc::Parser.parsers.find { |regexp,| regexp =~ file_name }

      # The default parser must not parse binary files
      ext_name = File.extname file_name
      return parser if ext_name.empty?

      if parser == RDoc::Parser::Simple and ext_name !~ /txt|rdoc/ then
        case check_modeline file_name
        when nil, 'rdoc' then # continue
        else return nil
        end
      end

      parser
    rescue Errno::EACCES
    end

check_modeline(file_name) click to toggle source

Returns the file type from the modeline in `file_name`

    # File lib/rdoc/parser.rb, line 165
    def self.check_modeline file_name
      line = open file_name do |io|
        io.gets
      end

      /-\*-\s*(.*?\S)\s*-\*-/ =~ line

      return nil unless type = $1

      if /;/ =~ type then
        return nil unless /(?:\s|\A)mode:\s*([^\s;]+)/i =~ type
        type = $1
      end

      return nil if /coding:/i =~ type

      type.downcase
    rescue ArgumentError
    rescue Encoding::InvalidByteSequenceError # invalid byte sequence

    end

for(top_level, file_name, content, options, stats) click to toggle source

Finds and instantiates the correct parser for the given `file_name` and `content`.

    # File lib/rdoc/parser.rb, line 191
    def self.for top_level, file_name, content, options, stats
      return if binary? file_name

      parser = use_markup content

      unless parser then
        parse_name = file_name

        # If no extension, look for shebang
        if file_name !~ /\.\w+$/ && content =~ %r{\A#!(.+)} then
          shebang = $1
          case shebang
          when %r{env\s+ruby}, %r{/ruby}
            parse_name = 'dummy.rb'
          end
        end

        parser = can_parse parse_name
      end

      return unless parser

      content = remove_modeline content

      parser.new top_level, file_name, content, options, stats
    rescue SystemCallError
      nil
    end

new(top_level, file_name, content, options, stats) click to toggle source

Creates a new Parser storing `top_level`, `file_name`, `content`, `options` and `stats` in instance variables. In +@preprocess+ an RDoc::Markup::PreProcess object is created which allows processing of directives.

    # File lib/rdoc/parser.rb, line 276
    def initialize top_level, file_name, content, options, stats
      @top_level = top_level
      @top_level.parser = self.class
      @store = @top_level.store

      @file_name = file_name
      @content = content
      @options = options
      @stats = stats

      @preprocess = RDoc::Markup::PreProcess.new @file_name, @options.rdoc_include
      @preprocess.options = @options
    end

parse_files_matching(regexp) click to toggle source

Record which file types this parser can understand.

It is ok to call this multiple times.

    # File lib/rdoc/parser.rb, line 225
    def self.parse_files_matching(regexp)
      RDoc::Parser.parsers.unshift [regexp, self]
    end

process_directive(code_object, directive, value) click to toggle source

Processes common directives for CodeObjects for the C and Ruby parsers.

Applies `directive`'s `value` to `code_object`, if appropriate

    # File lib/rdoc/parser.rb, line 94
    def self.process_directive code_object, directive, value
      warn "RDoc::Parser::process_directive is deprecated and wil be removed in RDoc 4.  Use RDoc::Markup::PreProcess#handle_directive instead" if $-w

      case directive
      when 'nodoc' then
        code_object.document_self = nil # notify nodoc
        code_object.document_children = value.downcase != 'all'
      when 'doc' then
        code_object.document_self = true
        code_object.force_documentation = true
      when 'yield', 'yields' then
        # remove parameter &block
        code_object.params.sub!(/,?\s*&\w+/, '') if code_object.params

        code_object.block_params = value
      when 'arg', 'args' then
        code_object.params = value
      end
    end

remove_modeline(content) click to toggle source

Removes an emacs-style modeline from the first line of the document

    # File lib/rdoc/parser.rb, line 232
    def self.remove_modeline content
      content.sub(/\A.*-\*-\s*(.*?\S)\s*-\*-.*\r?\n/, '')
    end

use_markup(content) click to toggle source

If there is a `markup: parser_name` comment at the front of the file, use it to determine the parser. For example:

    # markup: rdoc
    # Class comment can go here

    class C
    end

The comment should appear as the first line of the `content`.

If the content contains a shebang or editor modeline the comment may appear on the second or third line.

Any comment style may be used to hide the markup comment.

    # File lib/rdoc/parser.rb, line 253
    def self.use_markup content
      markup = content.lines.first(3).grep(/markup:\s+(\w+)/) { $1 }.first

      return unless markup

      # TODO Ruby should be returned only when the filename is correct
      return RDoc::Parser::Ruby if %w[tomdoc markdown].include? markup

      markup = Regexp.escape markup

      _, selected = RDoc::Parser.parsers.find do |_, parser|
        /^#{markup}$/i =~ parser.name.sub(/.*:/, '')
      end

      selected
    end

zip?(file) click to toggle source

Checks if `file` is a zip file in disguise. Signatures from www.garykessler.net/library/file_sigs.html

    # File lib/rdoc/parser.rb, line 118
    def self.zip? file
      zip_signature = File.read file, 4

      zip_signature == "PK\x03\x04" or
        zip_signature == "PK\x05\x06" or
        zip_signature == "PK\x07\x08"
    rescue
      false
    end