Skills

A general purpose storage interface

beidou base view plugin

A tokenzier for Sass' SCSS syntax

Parses and stringifies CSS selectors

Tiny JavaScript tokenizer.

Tokenize a shell string into argv array

Tokenize Excel formulas

Small library that provides functions to tokenize a string into an array of words with or without punctuation

Tokenize paragraphs into sentences, and smaller tokens.

Provide a high level wrapper for kuromoji.js

Forked version of kuromoji with better compatibility for browsers

The API for building streaming tokenizers and lexers.

Simple HTML Tokenizer is a lightweight JavaScript library that can be used to tokenize the kind of HTML normally found in templates.

A simple tool to generate bert tokens and input features

Official AstroPay client for easy frontend integration to Tokenizer

Simple algorithm to tokenize Chinese texts into words using CC-CEDICT.

A wide purpose tokenizer for node.js which looks like a stream

Wix Restaurants credit-cards tokenizer

String Tokenizer for Node.js using ICU's BreakIterators

A Vue 3 UI Kit for Tokenizer - Complete component library

A fast tokenizer/lexer for JavaScript

A regular expression (RexExp) stream tokenizer.

A fast tokenizer/lexer core that is well tested.

Extract a shader's name from its source