How to ignore accents in a highlight function

Question

I have a micro-mini search engine that highlights search terms in my rails application. The search ignores accents and the highlight is case insensitive. Almost perfect. But, for example, if I have a record with the text "pão de queijo" and I search for "pao de queijo", the recordwill return, but the iext icon is not . Similarly, if I search for "pÃo de queijo", the record is returned, but not highlighted properly.

My code is as simple as:

<%= highlight(result_pessoa.observacoes, search_string, '<span style="background-color: yellow;">\1</span>') %>

Answer 1

I just sent a Rails patch that resolves this.

http://rails.lighthouseapp.com/projects/8994-ruby-on-rails/tickets/3593-patch-support-for-highlighting-with-ignoring-special-chars

  # Highlights one or more +phrases+ everywhere in +text+ by inserting it into
  # a <tt>:highlighter</tt> string. The highlighter can be specialized by passing <tt>:highlighter</tt>
  # as a single-quoted string with \1 where the phrase is to be inserted (defaults to
  # '<strong class="highlight">\1</strong>')
  #
  # ==== Examples
  #   highlight('You searched for: rails', 'rails')
  #   # => You searched for: <strong class="highlight">rails</strong>
  #
  #   highlight('You searched for: ruby, rails, dhh', 'actionpack')
  #   # => You searched for: ruby, rails, dhh
  #
  #   highlight('You searched for: rails', ['for', 'rails'], :highlighter => '<em>\1</em>')
  #   # => You searched <em>for</em>: <em>rails</em>
  #
  #   highlight('You searched for: rails', 'rails', :highlighter => '<a href="search?q=\1">\1</a>')
  #   # => You searched for: <a href="search?q=rails">rails</a>
  #
  #   highlight('Šumné dievčatá', ['šumňe', 'dievca'], :ignore_special_chars => true)
  #   # => <strong class="highlight">Šumné</strong> <strong class="highlight">dievča</strong>tá  
  #
  # You can still use <tt>highlight</tt> with the old API that accepts the
  # +highlighter+ as its optional third parameter:
  #   highlight('You searched for: rails', 'rails', '<a href="search?q=\1">\1</a>')     # => You searched for: <a href="search?q=rails">rails</a>
  def highlight(text, phrases, *args)
    options = args.extract_options!
    unless args.empty?
      options[:highlighter] = args[0] || '<strong class="highlight">\1</strong>'
    end
    options.reverse_merge!(:highlighter => '<strong class="highlight">\1</strong>')

    if text.blank? || phrases.blank?
      text
    else
      haystack = text.clone
      match = Array(phrases).map { |p| Regexp.escape(p) }.join('|')
      if options[:ignore_special_chars]
        haystack = haystack.mb_chars.normalize(:kd)
        match = match.mb_chars.normalize(:kd).gsub(/[^\x00-\x7F]+/n, '').gsub(/\w/, '\0[^\x00-\x7F]*')
      end
      highlighted = haystack.gsub(/(#{match})(?!(?:[^<]*?)(?:["'])[^<>]*>)/i, options[:highlighter])
      highlighted = highlighted.mb_chars.normalize(:kc) if options[:ignore_special_chars]
      highlighted
    end
  end

Answer 2

, UTF-8 MySQL?

MySQL (, , ) .

Ruby. 1.8 Ruby Unicode. , , Rails, gsub, . , , Unicode, ICU4R.

Answer 3

, , Rails ActiveSupport.

Answer 4

It looks like you are using two different methods to determine if a match has occurred: one for your search and one for hilight. Use the same method as your highlight search, and it should raise it, no?