# text/v1 — Character/word/pattern counting

Deterministic text operations that are exactly the class of task LLMs get
wrong: BPE tokenization groups characters into tokens, so the unit an LLM
"sees" doesn't match the unit being counted or indexed — the canonical
example is an LLM confidently miscounting the letter `r` in "strawberry".

All operations work on Unicode **code points** (runes), not grapheme
clusters. That's correct for the common cases (ASCII, precomposed accented
characters like `é`, most single-codepoint emoji), but a decomposed accent
(base letter + combining mark) or a multi-codepoint emoji sequence (e.g. a
ZWJ family emoji) will not reverse or index "visually" correctly — Go's
standard library has no grapheme-cluster segmenter, and pulling in a
dependency for that edge case wasn't worth it here.

Every endpoint takes `{"text": "..."}` plus operation-specific fields, and
returns its result under an operation-specific key (not a generic
`{"valid":...}`/`{"result":...}` wrapper — each shape is documented below).
HTTP 400 with `{"error": "..."}` covers malformed request bodies and
invalid operation-specific input (empty substring, out-of-range index,
unknown sort mode).

### `POST /v1/text/count`

Counts non-overlapping occurrences of `substring` in `text`.

```bash
curl -s -X POST https://apished.com/v1/text/count \
  -H 'Content-Type: application/json' \
  -d '{"text": "strawberry", "substring": "r"}'
# {"count":3}
```

`substring` must be non-empty (an empty substring's occurrence count is an
ill-defined edge case, rejected with a 400 rather than exposed).

### `POST /v1/text/wordcount`

Counts whitespace-delimited words (any run of Unicode whitespace counts as
one separator; leading/trailing whitespace is ignored).

```bash
curl -s -X POST https://apished.com/v1/text/wordcount \
  -H 'Content-Type: application/json' \
  -d '{"text": "the quick brown fox"}'
# {"words":4}
```

### `POST /v1/text/reverse`

Reverses `text` by Unicode code point.

```bash
curl -s -X POST https://apished.com/v1/text/reverse \
  -H 'Content-Type: application/json' \
  -d '{"text": "café"}'
# {"reversed":"éfac"}
```

### `POST /v1/text/palindrome`

Checks whether `text` reads the same forwards and backwards.

- `ignoreCase` (optional, default `false`) — fold case before comparing.
- `ignoreNonAlnum` (optional, default `false`) — strip non-letter/non-digit
  characters before comparing (needed for phrase-style palindromes).

Both default to `false`: a strict, literal comparison of exactly what you
send.

```bash
curl -s -X POST https://apished.com/v1/text/palindrome \
  -H 'Content-Type: application/json' \
  -d '{"text": "A man, a plan, a canal: Panama", "ignoreCase": true, "ignoreNonAlnum": true}'
# {"palindrome":true}
```

### `POST /v1/text/charat`

Returns the 0-indexed character at `index`. Index 0 is the first
character; out-of-range indices return a 400 naming the valid range.

```bash
curl -s -X POST https://apished.com/v1/text/charat \
  -H 'Content-Type: application/json' \
  -d '{"text": "strawberry", "index": 0}'
# {"char":"s"}
```

### `POST /v1/text/sort`

Sorts `text`'s characters or words alphabetically.

- `by` (optional, default `"chars"`) — `"chars"` sorts individual
  characters and rejoins with no separator; `"words"` splits on
  whitespace, sorts the words, and rejoins with a single space.

```bash
curl -s -X POST https://apished.com/v1/text/sort \
  -H 'Content-Type: application/json' \
  -d '{"text": "banana", "by": "chars"}'
# {"sorted":"aaabnn"}
```

### `POST /v1/text/stats`

Descriptive statistics for `text`: counts, a character-type breakdown, an
exact per-character frequency table, and word-length summary stats.

```bash
curl -s -X POST https://apished.com/v1/text/stats \
  -H 'Content-Type: application/json' \
  -d '{"text": "Hello, World! 123"}'
```

```json
{
  "runes": 17,
  "bytes": 17,
  "words": 3,
  "lines": 1,
  "uniqueRunes": 13,
  "charTypes": { "letters": 10, "digits": 3, "spaces": 2, "punctuation": 2, "other": 0 },
  "charFrequency": { "H": 1, "e": 1, "l": 3, "o": 2, ",": 1, " ": 2, "W": 1, "r": 1, "d": 1, "!": 1, "1": 1, "2": 1, "3": 1 },
  "wordLengths": { "min": 3, "max": 6, "average": 5, "shortest": "123", "longest": "Hello," }
}
```

- `runes` / `bytes` — Unicode code point count vs. UTF-8 byte length; they
  differ whenever `text` has multi-byte characters (see the Unicode note
  above).
- `lines` — number of `\n`-delimited segments (`0` for an empty string,
  otherwise `strings.Count(text, "\n") + 1` — a trailing newline counts as
  starting an empty final line).
- `uniqueRunes` — count of distinct code points.
- `charTypes` — every rune classified into exactly one bucket, checked in
  this order: `spaces` (`unicode.IsSpace`), `letters` (`IsLetter`),
  `digits` (`IsDigit`), `punctuation` (`IsPunct`), else `other` (symbols,
  emoji, control characters — most emoji land in `other`, not
  `punctuation`).
- `charFrequency` — exact count per distinct character, case-sensitive, no
  folding.
- `wordLengths` — `min`/`max`/`average` word length in runes, plus the
  actual `shortest`/`longest` word. **Ties resolve to the first
  occurrence** — e.g. if two words share the max length, `longest` is
  whichever one appears first in `text`.

An empty `text` returns all-zero counts, an empty `charFrequency` object,
and zeroed/empty `wordLengths` — not an error.

**Not included: sentence count.** Splitting on `.`/`!`/`?` is a well-known
unreliable heuristic (breaks on abbreviations like "Mr. Smith", decimals,
etc.) — not deterministic enough to fit this module's premise.

## Files

- `textmodule.go` — module wiring (routes, HTTP handlers)
- `text.go` — pure text operations
- `text_test.go`, `stats_test.go` — test vectors, including multi-byte Unicode input