Module tokenize

Source
Expand description

Break a string into a set of directory-object Items.

This module defines Item, which represents a basic entry in a directory document, and NetDocReader, which is used to break a string into Items.

Modules§

object 🔒
Useful constants for netdoc object syntax

Structs§

Item 🔒
A single part of a directory object.
MaybeItem 🔒
Represents an Item that might not be present, whose arguments we want to inspect. If the Item is there, this acts like a proxy to the item; otherwise, it treats the item as having no arguments.
NetDocReader 🔒
A peekable cursor into a string that returns Items one by one.
NetDocReaderBase 🔒
A cursor into a string that returns Items one by one.
Object 🔒
A tagged object that is part of a directory Item.

Traits§

ItemResult 🔒
Extension trait for Result<Item> – makes it convenient to implement PauseAt predicates

Functions§

b64check 🔒
Check that all the characters in s are valid base64.
base64_decode_multiline 🔒
Helper: as base64::decode(), but allows newlines in the middle of the encoded object.
is_sp 🔒
Return true iff a given character is “space” according to the rules of dir-spec.txt
keyword_ok 🔒
Return true iff ‘s’ is a valid keyword or annotation.
tag_keywords_ok 🔒
Return true iff ‘s’ is a valid keywords string for a BEGIN/END tag.
validate_utf_8_rules 🔒
Check additional UTF-8 rules that the netdoc metaformat imposes on our documents.