Skills

Simple synchronous string tokenizer using Regex

Bunyan stream used to format logger input based on a JSON schema.

tokenizer for ecmascript

SIFT distance algorithm

Regex Based Tokenizer used in parsers developed by Porifa

Geometric Algebra for javascript with operator overloading and algebraic literals

Temporary fork that allows html in shortcode attributes. For the official package, see: https://www.npmjs.com/package/shortcode-tokenizer

Library to find urls, mentions, hashtags and more.

Tokenizer level utilities for the js-compiler.

Chinese Synonyms for Natural Language Processing and Understanding.

iterator based tokenizer for writing parsers

Set of utilities to work with Markdown stream.

Split text into tokens.

Tokenizes utterances that contain a mix of English and Chinese words.

filepath parses UNC (long and short and cmd aliases) local filesystem path, and unix paths, checks for invalid path characters

Arc - jsonex computation engine

Parser generator based on the atok tokenizer

A streaming JSON tokenizer

Flexibly generate design tokens from your Figma files

Get keywords (tokens) from any search query

Lightweight Japanese morphological analyzer for browser and Node.js

fast (~300 MB/sec) and simple incremental JSON parsing that can start and stop at any byte offset

render Yozora Markdown AST node `Admonition` into HTML string

Syntax Highlighting using the CSS Custom Highlight API