Skills

Basic Khmer Tokenizer

A fork with support for documentation of Salve, a Javascript library which implements a validator able to validate an XML document on the basis of a subset of RelaxNG.

Tokenize a graphql schema

Lightweight wikitext library for syntax highlighting and editing.

React component to convert a text string into React.ReactNodes

日本語用、テスト文章生成ライブラリ。マルコフ連鎖を使用しています

Tokenize a string into multiple token classes using a set of regular expressions, and get them back in order!

An extended version of moo with features needed by the Kreia parser engine.

Parser generator for JavaScript

Tokenize a string with regex.

WASM bindings for the tiktoken BPE tokenizer

Micromark syntax extension for pandoc-style citations.

Utility functions for a lightweight full-text searching and indexing library ndx.

Agnostic browser tokenizer for epayco

Tokenizer

Tokenizer for english sentences

tokenize code bricks for JavaScript

It is a package to help you parse a string into different types of sentences

Tokenize Excel formulas

JSON stream parsing library.

XML Stream parser (native js - no dependency)

A lightweight, flexible TypeScript library for building parsers using parser combinators. Create powerful parsers by combining simple, reusable parsing functions.

Micromark syntax extension for Discord-style spoilers.

jsx-tokenizer to transform words into tokens