Localize.Number.Scanner (Localize v1.5.0)

Copy Markdown View Source

Scans a string into the numbers it holds and the text around them, from the tokens of Localize.Number.Lexer.

The scanner walks the tokens once. At each token it takes the longest run that has the shape of a number below, and hands the run to Localize.Number.Grammar: the run is a number where the grammar reads one, and a piece of text of its own where it does not, since the grammar's rules are stricter than the shape. Everything else is text, returned as it was written, with consecutive pieces of text joined into one.

Grammar

scan      = { number | text } ;
number    = bidi, [ sign, bidi ], integer, [ fraction ], [ exponent ] ;
bidi      = { bidi-control } ;
integer   = digits, { digits | "_" | group, digits } ;
fraction  = decimal, zero-dash | decimal, { digits | "_" }- ;
zero-dash = dash, [ dash ] ;
exponent  = ( "e" | "E" ), [ sign ], digits ;

decimal and group are the marks of the lexicon's first reading, the separators of the locale, the number system and the options in hand, and a space is a group where that reading groups with one. The shape is narrower than the grammar's: a sign comes only before the digits and touches them, there are no parentheses and no space inside the number, so "- 5" is text and a number and "(5)" three pieces. The rules that complete the shape are the named functions below:

  • a group separator is followed by a plausible group: two digits or more leniently, exactly the locale's group strictly (plausible_group?/3);

  • a decimal separator is followed by a digit, an underscore, or the dashes of a zero fraction, which are read leniently only (fraction/2, zero_dash/2);

  • a zero fraction is one or two dashes with no digit after them, and ends the number (zero_dash/2);

  • an exponent is an ASCII e or E, a sign and digits, after the integer or after a fraction written with digits (exponent/1);

  • a bidi control before the digits, or between the sign and the digits, is part of the number; one after the digits is text (number_run/2).

Summary

Functions

scan(string, lexicon, options)

Scans a string into numbers and text with a lexicon.

Functions

scan(string, lexicon, options)

@spec scan(String.t(), Localize.Number.Lexicon.t(), Keyword.t()) :: [
  String.t() | integer() | float() | Decimal.t()
]

Scans a string into numbers and text with a lexicon.

Arguments

Options

  • :number is the type to read each number as: :integer, :float, :decimal, or nil to read an integer where the text is one and a float otherwise.

  • :grouping is how the grammar checks group separators: :lenient, :strict or :sizes.

  • :lenient is false to take only groups of the locale's sizes and no zero fraction. The default is true.

  • :max_number_bytes is the longest run, in bytes, that is read as a number; a longer one is text. The default is no limit.

  • :max_decimal_exponent bounds the exponent of a decimal read.

Returns

  • A list of strings and numbers.

Examples

iex> {:ok, fr} = Localize.validate_locale(:fr)
iex> {:ok, lexicon} = Localize.Number.Lexicon.fetch(fr, :latn, [])
iex> Localize.Number.Scanner.scan("2 000 et 3 000", lexicon, [])
[2000, " et ", 3000]

iex> {:ok, en} = Localize.validate_locale(:en)
iex> {:ok, lexicon} = Localize.Number.Lexicon.fetch(en, :latn, [])
iex> Localize.Number.Scanner.scan("v1.2.3 and 4,5", lexicon, [])
["v", 1.2, ".", 3, " and ", 4, ",", 5]