Localize.Number.Lexicon (Localize v1.7.0)

Copy Markdown View Source

What the number lexer and grammar know about a locale, computed once and cached.

A lexicon is built for a locale, a number system and the options a number is read with (:currency, :separators and :root_separators of Localize.Number.Parser.parse/2). It holds the digits the lexer reads, the characters that read as a minus or a plus sign, the dashes that may stand for a zero fraction, and one to three readings: the separator sets a number is read with, each as the characters that fold onto its decimal and group separator under CLDR's lenient-parse data, with the grouping sizes of the locale the set belongs to.

Folding the lenient-parse families into these sets is what lets the lexer classify a character with one lookup where the parser otherwise rewrote the string once per family.

Summary

Types

One set of separators a number is read with.

t()

A lexicon.

Functions

Returns the lexicon for a locale, a number system and reading options.

Chooses the number system a string's digits are read in.

Types

reading()

@type reading() :: %{
  tag: Localize.LanguageTag.t(),
  decimal: MapSet.t(String.t()),
  group: MapSet.t(String.t()),
  space_group?: boolean(),
  period?: boolean(),
  sizes: {pos_integer(), pos_integer()}
}

One set of separators a number is read with.

  • :tag is the locale whose grouping sizes and separators these are: the locale read in, or und for root_separators: true.

  • :decimal and :group are the characters that read as that separator: the separator itself and every character CLDR's lenient-parse data folds onto it.

  • :space_group? is true where the group separator is a space, in which case every space character groups.

  • :period? is true where a period is one of the two separators, so a period in a number is never foreign to this reading.

  • :sizes are the primary and secondary grouping sizes.

t()

@type t() :: %Localize.Number.Lexicon{
  dashes: MapSet.t(String.t()),
  digits: %{required(String.t()) => String.t()},
  locale: Localize.LanguageTag.t(),
  marks: MapSet.t(String.t()),
  minus: MapSet.t(String.t()),
  number_system: atom(),
  period: MapSet.t(String.t()),
  period_separator?: boolean(),
  plus: MapSet.t(String.t()),
  readings: [reading(), ...]
}

A lexicon.

  • :locale is the language tag it was built for.

  • :number_system is the number system whose digits are read, with Latin digits always read too.

  • :digits maps each digit character to its Latin digit, the map Localize.Number.Transliterate.transliterate_digits/2 reads.

  • :minus and :plus are the characters that read as a sign.

  • :dashes are the characters that may stand for a zero fraction ("5.–") and are not a minus.

  • :period is the characters that fold onto a period, for telling a foreign decimal point from the locale's separators.

  • :marks is every character that is a separator in some reading, or folds onto a period: what the lexer reads as a mark without consulting Unicode's categories.

  • :period_separator? is true where a period is a separator of some reading.

  • :readings are the separator sets, the locale's own first.

Functions

fetch(language_tag, number_system, options)

@spec fetch(Localize.LanguageTag.t(), atom(), Keyword.t()) ::
  {:ok, t()} | {:error, Exception.t()}

Returns the lexicon for a locale, a number system and reading options.

Arguments

Options

  • :currency is a currency code or a Localize.Currency.t/0. When set, the number is read with the separators the currency is formatted with and, where the locale formats plain numbers with other separators, with those too.

  • :separators names the set of separators to read with, :standard by default. A locale without the named set reads with its standard one.

  • :root_separators set to true adds the root locale's separators as a reading.

Returns

  • {:ok, lexicon}, or

  • {:error, exception} when the locale has no symbols for the number system, the currency is unknown or :separators is not an atom.

Examples

iex> {:ok, en} = Localize.validate_locale(:en)
iex> {:ok, lexicon} = Localize.Number.Lexicon.fetch(en, :latn, [])
iex> [reading] = lexicon.readings
iex> MapSet.member?(reading.decimal, ".") and MapSet.member?(reading.group, ",")
true

iex> {:ok, de_ch} = Localize.validate_locale(:"de-CH")
iex> {:ok, lexicon} = Localize.Number.Lexicon.fetch(de_ch, :latn, [])
iex> [reading] = lexicon.readings
iex> MapSet.member?(reading.group, "'") and MapSet.member?(reading.group, "’")
true

number_system(language_tag, string, options)

@spec number_system(Localize.LanguageTag.t(), String.t(), Keyword.t()) ::
  {:ok, atom()} | {:error, Exception.t()}

Chooses the number system a string's digits are read in.

The locale's own number system, unless the string is written in Latin digits alone, in which case Latin, with the locale's Latin separators. Latin digits written with a separator of the locale's own system that its Latin set does not use (ar-EG "1٬234٫5") are read in the locale's system, whose digits include the Latin ones. :number_system in the options overrides the choice.

Arguments

  • language_tag is a Localize.LanguageTag.t/0.

  • string is the text to be read.

  • options is a keyword list; :number_system names a number system to read with.

Returns

  • {:ok, number_system}, or

  • {:error, exception} when the number system has no digits.

Examples

iex> {:ok, ar_eg} = Localize.validate_locale(:"ar-EG")
iex> Localize.Number.Lexicon.number_system(ar_eg, "١٢٣", [])
{:ok, :arab}

iex> {:ok, ar_eg} = Localize.validate_locale(:"ar-EG")
iex> Localize.Number.Lexicon.number_system(ar_eg, "123", [])
{:ok, :latn}

iex> {:ok, ar_eg} = Localize.validate_locale(:"ar-EG")
iex> Localize.Number.Lexicon.number_system(ar_eg, "1٬234٫5", [])
{:ok, :arab}