Lexical Analysis Questions and Answers

Answer: C

Neither form nests, so both are regular and can be matched by patterns. Start conditions are a convenient way to switch the scanner into a mode in which only the closing delimiter matters. Without modes the patterns would have to describe every possible comment body explicitly.

Answer: D

A start condition partitions the rules so that, for instance, only the comment terminating rules apply inside a comment. It is the standard way of handling constructs with modes. String literals with embedded expressions are handled by the same mechanism.

Answer: A

Later phases work on progressively smaller representations, but the scanner touches every character. This is why buffering, sentinels and table compression receive so much attention. A few instructions saved per character are multiplied by the size of the whole file.

Answer: B

The first three are all conventional scanner duties. Type compatibility requires knowledge of declarations and expression structure, which belongs firmly to semantic analysis. The scanner has no view of declarations and so cannot make any such judgement at all.

Answer: C

The pattern is letter(letter|digit)* with the underscore usually counted as a letter. Requiring the first character to be a letter is what distinguishes identifiers from numeric literals. Most languages count the underscore as a letter for exactly this purpose.

Answer: D

Correct answer: one or more digits with no sign or decimal point permitted

An unsigned integer is precisely one or more decimal digits, written digit+. The sign is excluded because it is unsigned, and admitting zero digits would let the empty string be a number. The exponent and the decimal point belong to the pattern for a floating point literal, which is built up separately from the same digit class.

Answer: A

The longest match rule would take -1 as one lexeme, leaving the subtraction unrepresented. Keeping signs out of numeric patterns is why nearly every language scans the minus as an operator. The parser then decides from context whether the operator is unary or binary.

Answer: B

Counting newlines during scanning is cheap and gives every token a source position. Some compilers record a column as well so a diagnostic can point at the exact character. Accurate positions cost almost nothing to maintain and improve every message issued.

Answer: C

The conventional interface is a single function, historically yylex, returning one token per call. Attribute values are passed through a shared variable alongside the returned token name. Reentrant scanner generators pass a structure instead so several scans can proceed at once.

Answer: D

Correct answer: the compiler will not build, or will misinterpret the token stream

The token names are a contract between the two phases, usually held in a shared header that the parser generator produces and the scanner includes. If they disagree the build fails, or worse the numeric codes silently mismatch and the parser interprets one token as another. This is why the header is generated rather than written twice by hand.