This document is the API specification for the Deep Java Library (DJL) Huggingface tokenizers extension.

The Tokeniszers module contains the DJL extension for Huggingface tokenizers.