It is irrelevant whether the system is running deterministically; what makes us call it a “choice maker” is whether we can identify with a high-level description of the process which takes place when the program runs.#
The act of pulling out a string’s meaning involves, in essence, establishing all the implications of its connections to all other strings, and this leads, to be sure, down an open-ended trail. So “semantic” properties are connected to open-ended searches because, in an important sense, an object’s meaning is not localized within the object itself.#
In the course of developing a program with the aim of achieving an “intelligent” (viz., human-like) internal representation of what is “out there”, at some point one will probably be forced into using structures and processes which do not admit of any straightforward interpretations—that is, which cannot be directly mapped onto elements of reality. These lower layers of the program will be able to be understood only by virtue of their catalytic relation to layers above them, rather than because of some direct connection they have to the outer world.#
It is possible for a program to modify itself—but such modifiability has to be inherent in the program to start with, so that cannot be counted as an example of “jumping out of the system”.#
Consistency is not a property of a formal system per se, but depends on the interpretation which is proposed for it.#