6.17 The Text Interpreter

The text interpreter processes Forth source code, and performs the semantics of the words in the source code. The text interpreter works on Forth source code coming from the standard input (the user input device in Forth standard terminology), from a file (through included and friends), from evaluate, or from a loaded block.

The text interpreter is also called the outer interpreter, in contrast to the inner interpreter of traditional Forth implementations which interprets the threaded code that is the output of the Forth compiler on these implementations; many Forth systems nowadays compile to native code and have no inner interpreter. Concerning Gforth’s approach, see Engine.

The text interpreter works line by line (using refill). Therefore, most parsing words (e.g., ' and s") only try to find their parsed arguments in the current line; if a word parses across line boundaies, that is stated explicitly in the documentation of the word. However, a block is interpreted like one line, their division into “lines” is just for display purposes; Evaluate interprets the string as a whole. The current line/block/string is available to programs through source, and its content is valid until the current line is changed; it has to be treated as read-only region.

Within a line, the text interpreter parses for white-space-delimited words (with parse-name). The text interpreter’s position within the line is stored in >in, and by changing >in, a program can change where the text interpreter or parsing word continues to parse.

The text interpreter then tries to recognize the word with one of the recognizers in the system recognizer sequence. On success, this produces a translator and additional data. It then performs the interpretation, compilation, or postponing action of the translator, depending on the state of the text interpreter: Is it inside ]]...[[? If not, what is the value of state? E.g., when a word from the search order is recognized, and the text interpreter is in compile state, the compilation semantics of this word will be performed. This translator action may perform additional parsing, e.g., when the recognizer has recognized a parsing word. If no recognizer recognizes the word, the text interpreter throws -13 (“undefined word”).

The text interpreter then continues to parse, recognize, and translate words. At the end of the line, it refills and continues with the next line. At the end of an input source (e.g., at the end of a file), the text interpreter returns to its caller, and that caller (e.g., included) restores the previous input stream before returning itself.

If an exception is not caught earlier, the same unnesting of input streams and the associated calls are performed. If not catch intervenes, the throw is eventually caught by the system. In Gforth, the system then prints an error message with a backtrace of both the call stack and the stack of nested input streams (see Error messages). In interactive mode, Gforth then calls a text interpreter for the user input device, while in scripting mode (while processing OS command-line arguments, see Scripting mode) Gforth then terminates with a non-zero exit code.

You can read about this in more detail in Input Sources.

source ( – c-addr u  ) core “source”

c-addr u is the input buffer, i.e., the line/block/evaluated-string currently processed by the text interpreter

>in ( – addr  ) core “to-in”

Addr contains the offset into source where the text interpreter or parsing words parse next. Addr can be different for different tasks, and for different input streams within a task.

tib ( – addr  ) core-ext-obsolescent “t-i-b”
#tib ( – addr  ) core-ext-obsolescent “number-t-i-b”

Addr is the address of a cell containing the number of characters in the terminal input buffer. Addr can be different for different tasks, and for different input streams within a task. OBSOLESCENT: source supersedes the function of this word.

state ( – a-addr  ) core,tools-ext

Don’t use state! State is the state of the text interpreter, and ordinary words should work independently of it; in particular, state does not tell you whether the interpretation semantics or compilation semantics of a word are being performed. See State-smartness–Why it is evil and how to exorcise it. For an alternative to state-smart words, see How to define combined words.
A-addr is the address of a cell containing the compilation state flag. 0 => interpreting, -1 => compiling. A standard program must not store into state, but instead use [ and ].