tiktoken 1.0.0 copy "tiktoken: ^1.0.0" to clipboard
tiktoken: ^1.0.0 copied to clipboard

outdated

Tiktoken is a BPE tokeniser for use with OpenAI's models. It exposes APIs used for processing text using tokens.

1.0.0 #

  • Initial version.
22
likes
0
points
1.29k
downloads

Publisher

verified publishermarinvitlov.com

Weekly Downloads

Tiktoken is a BPE tokeniser for use with OpenAI's models. It exposes APIs used for processing text using tokens.

Homepage
Repository (GitHub)
View/report issues

License

unknown (license)

More

Packages that depend on tiktoken