Why I didn’t use AI to classify AI input

The obvious answer

When people first hear what Lacy does, the question is always the same: “So you trained a classifier?” Makes sense on the surface. Binary classification problem, shell command or natural language. Throw a model at it. Transformers eat this for breakfast.

I tried it. Early prototypes used a small language model to score inputs. Worked on benchmarks. Then I plugged it into a real shell and everything fell apart.

The latency wall

Lacy updates a color indicator as you type. Green means it goes to your shell. Magenta means it routes to the AI agent. This updates on every keystroke in ZSH via zle-line-pre-redraw.

Classification has to run in under 10 milliseconds. Not per submission. Per character. Every keypress triggers lacy_shell_classify_input(), which redraws the indicator based on what you’ve typed so far.

Model inference, even a tiny one, takes 50-200ms. That’s visible lag on every keystroke. The indicator stutters, lags behind your typing, flips colors a half-second late. It felt broken even when the classification was correct.

String matching takes microseconds. The indicator just updates.

Zero dependencies, zero startup cost

Lacy is a shell plugin. It loads when your terminal starts. Every millisecond of startup time is felt by the developer, every single morning, every single terminal tab.

An ML classifier needs a runtime. Python, ONNX, a WASM module, something. That means either bundling a binary (30MB+ for a shell plugin) or requiring a runtime dependency that many developers won’t have installed.

Lacy’s detection is pure Bash 4+. It works in ZSH and Bash. It sources a single file (lib/core/detection.sh) and uses command -v plus array membership checks. No binaries, no runtimes, no network calls. The entire detection system is a function you can read in under 100 lines.

The edge cases are the same

The hard cases for natural language detection in a shell context are inputs like:

• find out how the auth system works (find is a real command)
• make sure the tests pass (make is a real command)
• test the login flow (test is a real command)

These are hard for a model too. The first word is a legitimate shell command. The rest is English. A transformer trained on shell commands and English sentences would struggle with the exact same boundary that string matching struggles with.

The difference is that with string matching, I can handle these cases with a simple, explainable heuristic: let the shell try it first, catch the failure, and check if the error output looks like a shell trying to parse English. That’s Lacy’s Layer 2 detection. No model needed.

What the heuristic actually looks like

Lacy’s classification is a decision tree with five rules:

• First word is an agent word (like why, how, explain, about 150 common English words)? Route to agent.
• First word is a shell reserved word (like do, then, in)? Route to agent. These pass command -v but are never standalone commands.
• First word is a valid command? Route to shell.
• Single unknown word? Route to shell (probably a typo).
• Multiple words, first word unknown? Route to agent (natural language).

That covers the vast majority of inputs correctly with zero ambiguity. The remaining edge cases (valid command + natural language arguments) are handled by post-execution rerouting, which I wrote about in a separate post.

Debuggability matters

When a user reports that Lacy misrouted their input, I can trace the exact path through the decision tree. First word was go. go is in the agent words list. It’s also a valid command. The input had 2+ bare words after it with NL markers. Result: agent.

With a model, the answer is “the classifier gave it a 0.62 confidence score.” Why? Unclear. What do you change? Retrain with more data? Adjust the threshold? The feedback loop is long and opaque.

With a word list, the fix is: add the word, remove the word, or add a special case. The change is immediate, testable, and obvious.

The 95% that’s easy

Here’s the thing most people miss: natural language and shell commands are syntactically very different. They don’t require a model to tell apart. ls -la looks nothing like what files are in this directory. The first word alone is usually enough.

95% of inputs are trivially classifiable by checking if the first word is a known command. The remaining 5% are ambiguous cases where a real command name is followed by English words. A model doesn’t meaningfully outperform simple heuristics on that 5%, because the ambiguity is genuine and context-dependent.

For that 5%, Lacy uses a fallback that no model can replicate: it lets the shell try first and catches the error. The shell itself is the best classifier for whether input is valid shell syntax.

The result

Lacy’s detection has zero dependencies, zero latency, works in any Bash 4+ or ZSH environment, and handles the hard edge cases better than a classifier would. The entire system is about 400 lines of shell script split across two files.

Sometimes the boring solution is the right one. AI is a great tool. Using it to detect its own input, in a context where latency and simplicity are critical, is not the right use case.