Lexical Analysis Questions and Answers

Answer: A

Correct answer: folding the case of letters before or while matching the patterns

The simplest approach converts each letter to a single case as it is read, or uses patterns that accept either case, so that one set of rules serves the whole language. Writing two complete sets doubles the specification for no benefit, and deferring the question to a later phase would leave the scanner unable to recognise keywords at all.

Answer: B

Correct answer: extending the character classes and the alphabet of the automaton

Nothing about the method breaks: the automaton simply works over a larger alphabet, and the character classes that define what may start or continue an identifier are widened to include the additional characters. The transition table grows, which is one more reason for grouping characters into classes rather than tabulating each one.

Answer: C

Correct answer: keep the specification short and the transition table compact

Grouping the twenty-six letters into a single class means the specification says letter once instead of listing them, and the transition table needs one column for the class rather than twenty-six. Both effects matter: the first for writing and maintaining the rules, the second for the size of the generated scanner.

Answer: D

Converting the text to a value in the scanner saves later phases from repeating the work. Some compilers instead intern literals in a table so that identical constants share one entry. Interning saves space when the same constant appears many times in one program.

Answer: A

Correct answer: beyond the scanner, since it depends on the surrounding context

The two uses of the minus sign are lexically identical, so no pattern can tell them apart; what distinguishes them is what came before - an operand implies binary, an operator or an opening bracket implies unary. That is contextual information the scanner does not have, so it emits one token and the parser decides which production applies.

Answer: B

The scanner may create or find a symbol table entry, but declaration status depends on scope and on statements the scanner has no view of. Semantic analysis makes that determination once the tree and table are complete.

Answer: C

Giving each keyword its own token name lets the grammar mention it directly in productions. No attribute is required because the token name already determines the lexeme completely. Identifiers by contrast need an attribute, since the name itself must be carried along.

Answer: D

The scanner keeps a stack of indentation widths and emits tokens as levels open and close. The grammar then treats block structure with ordinary productions over those tokens. The stack of open levels is the extra state that a pure finite automaton could not hold.

Answer: A

Both are true and the reason gives the mechanism, and this pushes the scanner slightly beyond a pure finite automaton. The stack of levels is extra state carried in the semantic actions rather than in the automaton itself.

Answer: B

The pattern matches the literal including its escapes, and the semantic action builds the actual string value. Doing the translation in the scanner means later phases see a plain value. The original spelling is usually retained as well so diagnostics can quote the source.