`czech_plus.logic.lexer.tokens`#

Module for tokens, that then used inside lexer.

Module Contents#

Classes#

`BaseToken`	Base class for all tokens.
`SeparatorToken`	Token for separator symbol.
`AdditionalSeparatorToken`	Token for additional separator symbol.
`EscapedToken`	Token for escape symbol.
`SkipToken`	Token for skip symbol.
`BaseFutureFormToken`	Base token for verb's future form.
`FutureFormTokenStart`	Open token for verb's future form.
`FutureFormTokenEnd`	Close token for verb's future form.

class BaseToken[source]#

Bases: abc.ABC

Base class for all tokens.

class SeparatorToken[source]#

Bases: BaseToken

Token for separator symbol.

class AdditionalSeparatorToken[source]#

Bases: BaseToken

Token for additional separator symbol.

class EscapedToken[source]#

Bases: BaseToken

Token for escape symbol.

content: str[source]#

class SkipToken[source]#

Bases: BaseToken

Token for skip symbol.

class BaseFutureFormToken[source]#

Bases: BaseToken, abc.ABC

Base token for verb’s future form.

class FutureFormTokenStart[source]#

Bases: BaseFutureFormToken

Open token for verb’s future form.

class FutureFormTokenEnd[source]#

Bases: BaseFutureFormToken

Close token for verb’s future form.