Tokenization
(Redirected from Tokenizing)
Look up tokenization or tokenisation in Wiktionary, the free dictionary.
Tokenization may refer to:
- Tokenization (lexical analysis) in language processing
- Tokenization in search engine indexing
- Tokenization (data security) in the field of data security
- Word segmentation
- Transformer (deep learning architecture)
See also
[edit]- Tokenism of minorities
This disambiguation page lists articles associated with the title Tokenization.
If an internal link led you here, you may wish to change the link to point directly to the intended article.
If an internal link led you here, you may wish to change the link to point directly to the intended article.