Class: Ferret::Analysis::RegExpAnalyzer

Summary

Using a RegExpAnalyzer is a simple way to create a custom analyzer. If implemented in Ruby it would look like this;

  class RegExpAnalyzer
    def initialize(reg_exp, lower = true)
      @lower = lower
      @reg_exp = reg_exp
    end

    def token_stream(field, str)
      if @lower
        return LowerCaseFilter.new(RegExpTokenizer.new(str, reg_exp))
      else
        return RegExpTokenizer.new(str, reg_exp)
      end
    end
  end

Example

  csv_analyzer = RegExpAnalyzer.new(/[^,]+/, false)

Methods

new token_stream

Public Class methods

RegExpAnalyzer.new(reg_exp, lower = true) → analyzer

Create a new RegExpAnalyzer which will create tokenizers based on the regular expression and lowercasing if required.

reg_exp:	the token matcher for the tokenizer to use
lower:	set to false if you don‘t want to downcase the tokens

Public Instance methods

analyzer.token_stream(field_name, input) → token_stream

Create a new TokenStream to tokenize input. The TokenStream created may also depend on the field_name. Although this parameter is typically ignored.

field_name:	name of the field to be tokenized
input:	data from the field to be tokenized