Localize.Number.Lexicon (Localize v1.5.0)

What the number lexer and grammar know about a locale, computed once and cached.

A lexicon is built for a locale, a number system and the options a number is read with (:currency, :separators and :root_separators of Localize.Number.Parser.parse/2). It holds the digits the lexer reads, the characters that read as a minus or a plus sign, the dashes that may stand for a zero fraction, and one to three readings: the separator sets a number is read with, each as the characters that fold onto its decimal and group separator under CLDR's lenient-parse data, with the grouping sizes of the locale the set belongs to.

Folding the lenient-parse families into these sets is what lets the lexer classify a character with one lookup where the parser otherwise rewrote the string once per family.

Summary

Types

reading()

One set of separators a number is read with.

t()

A lexicon.

Functions

fetch(language_tag, number_system, options)

Returns the lexicon for a locale, a number system and reading options.

number_system(language_tag, string, options)

Chooses the number system a string's digits are read in.

Types

reading()

@type reading() :: %{
  tag: Localize.LanguageTag.t(),
  decimal: MapSet.t(String.t()),
  group: MapSet.t(String.t()),
  space_group?: boolean(),
  period?: boolean(),
  sizes: {pos_integer(), pos_integer()}
}

One set of separators a number is read with.

:tag is the locale whose grouping sizes and separators these are: the locale read in, or und for root_separators: true.
:decimal and :group are the characters that read as that separator: the separator itself and every character CLDR's lenient-parse data folds onto it.
:space_group? is true where the group separator is a space, in which case every space character groups.
:period? is true where a period is one of the two separators, so a period in a number is never foreign to this reading.
:sizes are the primary and secondary grouping sizes.

t()

@type t() :: %Localize.Number.Lexicon{
  dashes: MapSet.t(String.t()),
  digits: %{required(String.t()) => String.t()},
  locale: Localize.LanguageTag.t(),
  marks: MapSet.t(String.t()),
  minus: MapSet.t(String.t()),
  number_system: atom(),
  period: MapSet.t(String.t()),
  period_separator?: boolean(),
  plus: MapSet.t(String.t()),
  readings: [reading(), ...]
}

A lexicon.

:locale is the language tag it was built for.
:number_system is the number system whose digits are read, with Latin digits always read too.
:digits maps each digit character to its Latin digit, the map Localize.Number.Transliterate.transliterate_digits/2 reads.
:minus and :plus are the characters that read as a sign.
:dashes are the characters that may stand for a zero fraction ("5.–") and are not a minus.
:period is the characters that fold onto a period, for telling a foreign decimal point from the locale's separators.
:marks is every character that is a separator in some reading, or folds onto a period: what the lexer reads as a mark without consulting Unicode's categories.
:period_separator? is true where a period is a separator of some reading.
:readings are the separator sets, the locale's own first.

Functions

fetch(language_tag, number_system, options)

@spec fetch(Localize.LanguageTag.t(), atom(), Keyword.t()) ::
  {:ok, t()} | {:error, Exception.t()}

Returns the lexicon for a locale, a number system and reading options.

Arguments

language_tag is a Localize.LanguageTag.t/0.
number_system is the number system whose digits are read, as number_system/3 chooses it.
options is a keyword list of options.

Options

:currency is a currency code or a Localize.Currency.t/0. When set, the number is read with the separators the currency is formatted with and, where the locale formats plain numbers with other separators, with those too.
:separators names the set of separators to read with, :standard by default.
:root_separators set to true adds the root locale's separators as a reading.

Returns

{:ok, lexicon}, or
{:error, exception} when the locale has no symbols for the number system or the currency is unknown.

Examples

iex> {:ok, en} = Localize.validate_locale(:en)
iex> {:ok, lexicon} = Localize.Number.Lexicon.fetch(en, :latn, [])
iex> [reading] = lexicon.readings
iex> MapSet.member?(reading.decimal, ".") and MapSet.member?(reading.group, ",")
true

iex> {:ok, de_ch} = Localize.validate_locale(:"de-CH")
iex> {:ok, lexicon} = Localize.Number.Lexicon.fetch(de_ch, :latn, [])
iex> [reading] = lexicon.readings
iex> MapSet.member?(reading.group, "'") and MapSet.member?(reading.group, "’")
true

number_system(language_tag, string, options)

@spec number_system(Localize.LanguageTag.t(), String.t(), Keyword.t()) ::
  {:ok, atom()} | {:error, Exception.t()}

Chooses the number system a string's digits are read in.

The locale's own number system, unless the string is written in Latin digits alone, in which case Latin, with the locale's Latin separators. :number_system in the options overrides the choice.

Arguments

language_tag is a Localize.LanguageTag.t/0.
string is the text to be read.
options is a keyword list; :number_system names a number system to read with.

Returns

{:ok, number_system}, or
{:error, exception} when the number system has no digits.

Examples

iex> {:ok, ar_eg} = Localize.validate_locale(:"ar-EG")
iex> Localize.Number.Lexicon.number_system(ar_eg, "١٢٣", [])
{:ok, :arab}

iex> {:ok, ar_eg} = Localize.validate_locale(:"ar-EG")
iex> Localize.Number.Lexicon.number_system(ar_eg, "123", [])
{:ok, :latn}