czech_plus.logic.lexer

Package for parsing input from the cards.

Submodules

• czech_plus.logic.lexer.tokens

Package Contents

Classes

BaseLexer	Main class for transforming raw strings to tokens.
NounLexer	Lexer for nouns.
VerbLexer	Lexer for verbs.
AdjectiveLexer	Lexer for adjectives.

Functions

assert_that(→ None)

By default, Anki removes all asserts from the code, so we need to craft own assert.

Attributes

_HOOK_GENERATOR_SIGNATURE
_HOOK_SIGNATURE
_ON_NEXT_HOOK

assert_that(statement: bool, msg: str = '', /) → None

By default, Anki removes all asserts from the code, so we need to craft own assert.

Example

assert next(generator) == str  # if this statement will not be executed - everything will fail.

_HOOK_GENERATOR_SIGNATURE: typing_extensions.TypeAlias

_HOOK_SIGNATURE: typing_extensions.TypeAlias

_ON_NEXT_HOOK: typing_extensions.TypeAlias

class BaseLexer

Bases: abc.ABC

Main class for transforming raw strings to tokens.

property _hooks: dict[str, _HOOK_SIGNATURE]

Dict, where first element is symbol for hook, and value is a hook.

See _handle_hook() for hook signature description.

SEPARATE_SYMBOL = ','

ADDITIONAL_SEPARATE_SYMBOL: str | None

ESCAPE_SYMBOL = '\\'

WORD_ESCAPE_SYMBOL = '!'

SKIP_SYMBOL = '_'

_ESCAPE_WORD_STOP_SYMBOLS

lex(string: str) → collections.abc.Iterator[tokens.BaseToken | str]

Lex string argument.

Yields:

Token or string.

_handle_hook(hook: _HOOK_SIGNATURE) → _ON_NEXT_HOOK

Handle hook and yield result.

Parameters:

hook – Hook callable, that returns generator.

Yields:

None or token.

Note

You can send these params, with generator.send method. It takes string or None, where string means next symbol, and None - end of input.

Returns:

Note

This value can be accessed with StopIteration.value.

Tuple with two elements, where both are bools. First is whether you need to rerun this symbol with new hook, and second one whether you need to skip next symbol.

Example

handle_hook_generator = _handle_hook(hook)
while True:
    try:
        token = next(handle_hook_generator)
    except StopIteration as exception:
        rerun, skip = exception.value  # return statement in generator
        break
    else:
        # some token handling code

Or you can ignore rerun and skip variables:

for token in _handle_hook():
    # some token handling code

_escape_one_symbol() → _HOOK_GENERATOR_SIGNATURE

Escape one symbol.

See _handle_hook() for signature description.

_escape_entire_word() → _HOOK_GENERATOR_SIGNATURE

Escape entire word.

See _handle_hook() for signature description.

_separate_words() → _HOOK_GENERATOR_SIGNATURE

Separate words.

See _handle_hook() for signature description.

_additional_separate_words() → _HOOK_GENERATOR_SIGNATURE

Separate words with additional separate symbol.

See _handle_hook() for signature description.

_skip_word() → _HOOK_GENERATOR_SIGNATURE

Skip word.

See _handle_hook() for signature description.

class NounLexer

Bases: BaseLexer

Lexer for nouns.

class VerbLexer

Bases: BaseLexer

Lexer for verbs.

property _hooks: dict[str, _HOOK_SIGNATURE]

Dict, where first element is symbol for hook, and value is a hook.

See _handle_hook() for hook signature description.

SEPARATE_SYMBOL = '.'

ADDITIONAL_SEPARATE_SYMBOL = ','

_ESCAPE_WORD_STOP_SYMBOLS

_future_form_start() → _HOOK_GENERATOR_SIGNATURE

Find future form.

See _handle_hook() for signature description.

_future_form_end() → _HOOK_GENERATOR_SIGNATURE

Find future form.

See _handle_hook() for signature description.

class AdjectiveLexer

Bases: BaseLexer

Lexer for adjectives.