Runes and range over a string

In the previous lesson a string was a row of tiles, where a Cyrillic letter takes two tiles. The len function counts the tiles, the bytes, so for a Cyrillic word it returns twice as many as there are letters. To count and walk through the characters themselves, Go has a separate function and a separate type:

package main

import (
    "fmt"
    "unicode/utf8"
)

func main() {
    s := "мир"
    fmt.Println(len(s), utf8.RuneCountInString(s)) // 6 3
}

Let's walk through the output:

1. len(s) returns 6: the word "мир" has three letters of two bytes each.
2. utf8.RuneCountInString(s) counts whole letters and returns 3.

A rune is a character

A rune is the number of a character in the Unicode table, a value of type rune. In the tile picture, a rune is one whole letter, however many tiles it takes. Unicode is a single list of characters for all languages, where each has its own number, and UTF-8 stores that number in one to four bytes.

How a rune looks in code:

• single quotes create exactly a rune: 'м' is the number 1084;
• the type rune is another name for int32, so runes are compared with ==, like numbers;
• a single-quoted literal takes the type it is compared with.

So s[0] == 'G' compiles, but comparing the byte s[i] with a rune variable does not.

Counting characters

The function utf8.RuneCountInString(s) from the unicode/utf8 package returns the number of runes, that is, the characters of the string. The letter é also takes two bytes in UTF-8, so the string "café" has five bytes and four runes.

Interviewers like to ask: "How is a byte different from a rune?" The answer has two parts:

• a byte is one tile, a number from 0 to 255, a piece of the text's encoding;
• a rune is the Unicode number of a whole character, and in UTF-8 it takes one to four bytes.

Walking through a string

A loop can walk through a string in two ways: by runes and by bytes. Let's look at each.

range by runes

The loop for i, r := range s walks through a string rune by rune. The variable r gets the next rune, and i gets the index of the byte where it starts. That is why the indexes skip one for two-byte letters:

for i, r := range "мир" {
    fmt.Print(i, ":", string(r), " ") // 0:м 2:и 4:р
}

Let's walk through the output letter by letter:

1. The letter м starts at byte 0 and takes bytes 0 and 1.
2. So и starts at byte 2.
3. And р starts at byte 4.

The conversion string(r) turns a rune into a one-character string. Without it, fmt.Print would print the rune's number, not the letter. If you do not need the index, write _ in place of its name: for _, r := range s.

A loop over bytes

A plain loop for i := 0; i < len(s); i++ goes byte by byte. It visits the tiles one at a time and does not join the halves of a letter.

For "мир" such a loop takes six steps, and on each step s[i] gives one byte, not a letter. It suits a string of only Latin letters, digits and spaces: there every character takes one byte.

A slice of runes

To access characters by position, convert the string to a slice of runes, []rune(s). It is like taking the letters off the thread and laying them out in cells, one whole letter per cell. The properties of such a slice:

• each of its elements is a whole character, and len returns the number of characters;
• a slice of runes is a copy, and you can change its elements;
• the reverse conversion string(rs) builds a new string from the slice;
• the conversion []rune(s) walks the whole string, so do it once before a loop, not on every iteration.

rs := []rune("мир")
rs[0] = 'п'
fmt.Println(len(rs), string(rs)) // 3 пир

Let's walk through the example line by line:

1. rs := []rune("мир") gives a slice of three runes.
2. rs[0] = 'п' changes the first rune: a slice, unlike a string, can be changed.
3. string(rs) builds a new string, and the program prints 3 пир.

A rune and a visible letter

The picture "a rune is a whole letter" has a limit. Not every sign a person sees is one rune. The letter é can be written as one rune or as two: the letter e and a separate accent mark. In the second case utf8.RuneCountInString counts two runes, though the screen shows one letter.

Common mistake

The length of a name is checked with len, although the limit is set in letters:

name := "Анна"
if len(name) > 4 { // ✗
    fmt.Println("too long")
}

The program prints too long, although the name has four letters: len returns 8, two bytes per letter. Count the characters with utf8.RuneCountInString(name).

Key points

• A rune is a character's Unicode number, of type rune (another name for int32); a byte is a piece of its UTF-8 encoding.
• utf8.RuneCountInString(s) counts characters, and len(s) counts bytes.
• for i, r := range s walks a string rune by rune, and i is the index of the byte where the rune starts.
• []rune(s) gives a slice of characters you can change, and string(rs) builds a new string from it.