A software tool that breaks down input text.

An automated analysis program for lexical scanning.

One who analyzes and interprets written language tokens

To analyze and break down into lexical units.

To separate a string of characters into tokens.

To scan input for specific patterns or symbols.