Lexical Analysis Questions and Answers

Answer: A

Without this rule the scanner would return a bare less than sign rather than the two character comparison operator at the start of an inequality. The scanner therefore continues while a longer lexeme remains possible and retreats to the last accepting position when it fails.

Answer: B

Rule order breaks ties. This is why keywords are listed before the general identifier pattern: both match the lexeme if, but the earlier keyword rule wins and the correct token is produced. Reversing the order silently turns every keyword in the language into an identifier.

Answer: C

Both are true and the reason gives the mechanism, and reversing the order would make every keyword scan as an identifier. The alternative design recognises all words as identifiers and looks each one up in a table of reserved words.

Answer: D

Installing keywords in the symbol table before scanning begins lets one identifier pattern handle everything. A lookup after each word decides whether it is a keyword or a plain identifier, which keeps the automaton small.

Answer: A

Early Fortran allowed IF as a variable name. Deciding the token then requires context that the scanner does not have, which is why nearly all modern languages reserve their keywords. Reserving them costs a few identifiers and removes an entire class of difficulty.

Answer: B

Correct answer: reduce the cost of reading characters one at a time from the file

Reading is done a block at a time into one half of the buffer while the scanner works through the other, so a system call is made once per block rather than once per character. The two-half arrangement also allows a lexeme to straddle the boundary, since the beginning of it survives in the half that has not yet been refilled.

Answer: C

Correct answer: combine the end of buffer test with the character comparison

Without a sentinel every character read needs two tests: is this the end of the buffer, and is it the character I am looking for. Placing a character that cannot occur in the source at the end of each half means the scanner tests only for that one value, and checks whether it has reached the end of the buffer only in the rare case where the sentinel actually appears.

Answer: D

One pointer stays at the lexeme start while the other advances. When a pattern is matched the lexeme lies between them, and the start pointer is then moved forward ready for the next lexeme. Retraction moves the forward pointer back without disturbing the lexeme beginning.

Answer: A

Correct answer: a failure that the scanner must detect and report to the user

With a two-half buffer the scanner can hold a lexeme up to one buffer length; a longer one would require overwriting the half containing its own beginning. The scanner must therefore detect the condition and report it rather than silently truncating. In practice the buffer is made large enough that only a pathological string literal could reach the limit.

Answer: B

Deciding between a one character and a two character comparison operator needs one character of lookahead. Some constructs, notably the classic Fortran DO statement, need arbitrarily much, which is one reason such designs are now avoided.