Class CodeRay::TokenStream
In: lib/coderay/tokens.rb
Parent: Tokens

TokenStream

The TokenStream class is a fake Array without elements.

It redirects the method << to a block given at creation.

This allows scanners and Encoders to use streaming (no tokens are saved, the input is highlighted the same time it is scanned) with the same code.

See CodeRay.encode_stream and CodeRay.scan_stream

Methods

<<   dump   new   optimize   stream?   text_size  

Attributes

size  [R]  The Array is empty, but size counts the tokens given by <<.

Public Class methods

new(&block)

Creates a new TokenStream that calls block whenever its << method is called.

Example: 

  require 'coderay'

  token_stream = CodeRay::TokenStream.new do |kind, text|
    puts 'kind: %s, text size: %d.' % [kind, text.size]
  end

  token_stream << [:regexp, '/\d+/']
  #-> kind: rexpexp, text size: 5.

[Source]

     # File lib/coderay/tokens.rb, line 350
350:     def initialize &block
351:       raise ArgumentError, 'Block expected for streaming.' unless block
352:       @callback = block
353:       @size = 0
354:     end

Public Instance methods

<<(token)

Calls block with token and increments size.

Returns self.

[Source]

     # File lib/coderay/tokens.rb, line 359
359:     def << token
360:       @callback.call(*token)
361:       @size += 1
362:       self
363:     end
dump()

A TokenStream cannot be dumped. Use Tokens.

[Source]

     # File lib/coderay/tokens.rb, line 372
372:     def dump
373:       raise NotImplementedError, 'A TokenStream cannot be dumped.'
374:     end
optimize()

A TokenStream cannot be optimized. Use Tokens.

[Source]

     # File lib/coderay/tokens.rb, line 377
377:     def optimize
378:       raise NotImplementedError, 'A TokenStream cannot be optimized.'
379:     end
stream?()

Whether the object is a TokenStream.

Returns true.

[Source]

     # File lib/coderay/tokens.rb, line 329
329:     def stream?
330:       true
331:     end
text_size()

This method is not implemented due to speed reasons. Use Tokens.

[Source]

     # File lib/coderay/tokens.rb, line 366
366:     def text_size
367:       raise NotImplementedError,
368:         'This method is not implemented due to speed reasons.'
369:     end

[Validate]