Go: Removing accents from strings

Question

I am new to Go and I am trying to implement a function to convert accented characters to their equivalent without accent. I am trying to follow the example given on this blog (see the heading "Performing Magic").

I tried to compile from this:

package main import ( "fmt" "unicode" "bytes" "code.google.com/p/go.text/transform" "code.google.com/p/go.text/unicode/norm" ) func isMn (r rune) bool { return unicode.Is(unicode.Mn, r) // Mn: nonspacing marks } func main() { r := bytes.NewBufferString("Your Śtring") t := transform.Chain(norm.NFD, transform.RemoveFunc(isMn), norm.NFC) r = transform.NewReader(r, t) fmt.Println(r) } 

This does not work in the least, and I honestly don’t know what that means. Any ideas?

Answer 1

r should be or type io.Reader and you cannot type r like that. First, you need to read the contents in a byte fragment:

  var ( s = "Your Śtring" b = make([]byte, len(s)) r io.Reader = strings.NewReader(s) ) t := transform.Chain(norm.NFD, transform.RemoveFunc(isMn), norm.NFC) r = transform.NewReader(r, t) r.Read(b) fmt.Println(string(b)) 

This works, but for some reason returns "Your Stri", two bytes less than necessary.

This is the version that actually does what you need, but I'm still not sure why the blog example works so weird.

 s := "Yoùr Śtring" b := make([]byte, len(s)) t := transform.Chain(norm.NFD, transform.RemoveFunc(isMn), norm.NFC) _, _, e := t.Transform(b, []byte(s), true) if e != nil { panic(e) } fmt.Println(string(b)) 

Answer 2

Please note that Go 1.5 (August 2015) or Go 1.6 (Q1 2016) may introduce a new rune package with conversion operations.

This includes the ( runes/example_test.go ) runes.Remove function, which will help convert résumé to resume :

 func ExampleRemove() { t := transform.Chain(norm.NFD, runes.Remove(runes.In(unicode.Mn)), norm.NFC) s, _, _ := transform.String(t, "résumé") fmt.Println(s) // Output: // resume } 

This is still under review (April 2015) .