Summary

The WhiteSpaceAnalyzer recognizes tokens as maximal strings of non-whitespace characters. If implemented in Ruby the WhiteSpaceAnalyzer would look like;

  class WhiteSpaceAnalyzer
    def initialize(lower = true)
      @lower = lower
    end

    def token_stream(field, str)
      return WhiteSpaceTokenizer.new(str, @lower)
    end
  end

As you can see it makes use of the WhiteSpaceTokenizer.