Lexical Analysis Questions and Answers

Q: A lexical analyzer uses the following patterns to recognize three tokens T1, T2, and T3 over the alphabet {a,b,c}.\(T_{1}:\ a?\,(b|c)^{*}a\)\(T_{2}:\ b?\,(a|c)^{*}b\)\(T_{3}:\ c?\,(b|a)^{*}c\)Note that 'x?' means 0 or 1 occurrence of the symbol x. Note also that the analyzer outputs the token that matches the longest possible prefix.If the string \(bbaacabc\) is processed by the analyzer, which one of the following is the sequence of tokens it outputs?

The analyzer outputs \(T_{3}T_{3}\).It always takes the longest prefix any pattern matches. On bbaacabc:\(T_{1}\) can match bba (no leading a, then bb, then a): 3 characters.\(T_{2}\) can match only bb, since \((a|c)^{*}\) cannot absorb the second b: 2 characters.\(T_{3}\) can match bbaac (no leading c, then bbaa from \((b|a)^{*}\), then c): 5 characters.So the first token is \(T_{3}\), for bbaac. The rest is abc: \(T_{3}\) matches all of it (ab, then c), while \(T_{1}\) matches only a and \(T_{2}\) only ab. So the second token is \(T_{3}\) too.The options starting with \(T_{1}\) or \(T_{2}\) take a shorter prefix than the longest-match rule allows.

Q: Consider the following definition of a lexical token id for an identifier in a programming language, using extended regular expressions:letter \(\to\) [A-Za-z]digit \(\to\) [0-9]id \(\to\) letter (letter | digit)\(^{*}\)Which one of the following Non-deterministic Finite-state Automata with \(\epsilon\)-transitions accepts the set of valid identifiers? (A double-circle denotes a final state)

The correct automaton is (C).In (C), a letter takes the start state to the final state. From there, \(\epsilon\)-moves lead into two branches, one reading a single letter and one reading a single digit, and both rejoin a state with an \(\epsilon\)-move back to the final state. So after the first letter the machine can read any number of letters and digits in any order: exactly letter (letter | digit)*.(A) has an \(\epsilon\)-move from the start state straight to the final state, so it also accepts the empty string and strings starting with a digit. (B) commits to one branch after the first letter, so it accepts a1 and ab but not a1b, which mixes letters and digits. (D) loops only on the pair 'letter digit', so it rejects ab.

Q: The grouping of characters into tokens are carried through :

Characters are grouped into tokens by the scanner.The scanner, or lexical analyser, is the first phase of a compiler. It reads the source program character by character and groups the characters into tokens such as keywords, identifiers, constants and operators, removing white space and comments.The parser takes the tokens and checks that they form valid syntax. Code optimisation improves intermediate code, and the code generator produces target code; both come much later than token formation.

Q: Which global variable in LEX points to matched string and which global variable contains the length of the matched string ?

The variables are yytext, yyleng.In a LEX-generated scanner, yytext points to the text of the lexeme just matched, and yyleng holds its length. Actions in the rules use them, for example to copy the matched identifier or number.yylex is the scanning function itself, which returns the next token, and yylval is the variable used to pass a token's value to a YACC parser. lextext, lexval and lexyy are not LEX's global variables (lex.yy.c is the name of the generated file).

Q: The number of tokens in the following C statement isprintf("i = %d, &i = %x", i, &i);

The statement has 10 tokens.A string literal is a single token however much it contains, so the statement splits into:printf | ( | "i = %d, &i = %x" | , | i | , | & | i | ) | ;That is 10 tokens. 26 and 21 come from counting the characters inside the string as separate tokens, and 3 counts only the identifiers.The tokens fall into the usual classes: identifiers (printf, i), a string literal, punctuators ( ( ) , ; ) and the operator &.

Q: The lexical analysis for a modern computer language such as Java needs the power of which one of the following machine models in a necessary and sufficient sense ?

Lexical analysis needs finite state automata.The tokens of a programming language, identifiers, keywords, numbers, operators and string literals, are all described by regular expressions. Every regular language is recognised by a finite automaton, so finite automata are sufficient; and since tokens form regular sets, nothing weaker would do, so they are also necessary.Pushdown automata are needed for parsing the nested, context-free syntax, which is the next phase, and a Turing machine is far more power than lexical analysis requires.

Q: In analyzing the compilation of PL/I (Programming Language One) program, the term "Lexical analysis" is associated with

Lexical analysis is associated with recognition of basic elements and creation of uniform symbols.In the classic model of a PL/I compiler, the lexical phase scans the source, recognises its basic elements (identifiers, literals, operators and keywords) and replaces each by a uniform symbol, entering identifiers and literals in tables for later phases.Recognising syntactic constructs through reductions is syntax analysis, and producing optimised matrices or assembly code belongs to the optimisation and code generation phases.

Q: In a compiler, keywords of a language are recognized during

Keywords are recognised during the lexical analysis of the program.The lexical analyser (scanner) reads the character stream and groups it into tokens such as identifiers, numbers, operators and keywords. Keywords are usually recognised there, often by first matching an identifier and then looking it up in a table of reserved words.The parser works on the tokens the scanner has already produced, and code generation and dataflow analysis come much later in the compiler.

Q: Which of the following is not a white space character ?

The character that is not white space is the SEMICOLON.White space characters are those that produce blank space and separate tokens: the blank (space), horizontal tab, newline, and also vertical tab, form feed and carriage return. In C the function isspace() returns true for exactly these.The semicolon is a printing punctuation character. In C and C++ it is the statement terminator, a token in its own right, not white space.

A lexical analyzer uses the following patterns to recognize three tokens T₁, T₂, and T₃ over the alphabet {a,b,c}.

\(T_{1}:\ a?\,(b|c)^{*}a\)
\(T_{2}:\ b?\,(a|c)^{*}b\)
\(T_{3}:\ c?\,(b|a)^{*}c\)

Note that 'x?' means 0 or 1 occurrence of the symbol x. Note also that the analyzer outputs the token that matches the longest possible prefix.

If the string \(bbaacabc\) is processed by the analyzer, which one of the following is the sequence of tokens it outputs?

A.\(T_{1}T_{2}T_{3}\)
B.\(T_{1}T_{1}T_{3}\)
C.\(T_{2}T_{1}T_{3}\)
D.\(T_{3}T_{3}\)

Exam Year(s)

GATE CS - 2018,

Answer: D

The analyzer outputs \(T_{3}T_{3}\).

It always takes the longest prefix any pattern matches. On bbaacabc:

\(T_{1}\) can match bba (no leading a, then bb, then a): 3 characters.
\(T_{2}\) can match only bb, since \((a|c)^{*}\) cannot absorb the second b: 2 characters.
\(T_{3}\) can match bbaac (no leading c, then bbaa from \((b|a)^{*}\), then c): 5 characters.

So the first token is \(T_{3}\), for bbaac. The rest is abc: \(T_{3}\) matches all of it (ab, then c), while \(T_{1}\) matches only a and \(T_{2}\) only ab. So the second token is \(T_{3}\) too.

The options starting with \(T_{1}\) or \(T_{2}\) take a shorter prefix than the longest-match rule allows.

Discuss in Forum

Consider the following definition of a lexical token id for an identifier in a programming language, using extended regular expressions:

letter \(\to\) [A-Za-z]
digit \(\to\) [0-9]
id \(\to\) letter (letter | digit)\(^{*}\)

Which one of the following Non-deterministic Finite-state Automata with \(\epsilon\)-transitions accepts the set of valid identifiers? (A double-circle denotes a final state)

Exam Year(s)

GATE CS - 2023,

Answer: C

The correct automaton is (C).

In (C), a letter takes the start state to the final state. From there, \(\epsilon\)-moves lead into two branches, one reading a single letter and one reading a single digit, and both rejoin a state with an \(\epsilon\)-move back to the final state. So after the first letter the machine can read any number of letters and digits in any order: exactly letter (letter | digit)*.

(A) has an \(\epsilon\)-move from the start state straight to the final state, so it also accepts the empty string and strings starting with a digit. (B) commits to one branch after the first letter, so it accepts a1 and ab but not a1b, which mixes letters and digits. (D) loops only on the pair 'letter digit', so it rejects ab.

Discuss in Forum

The grouping of characters into tokens are carried through :

A.Code optimization
B.Code generator
C.Scanner
D.Parser

Exam Year(s)

MH SET - 2018, SET Exam - 2018,

Answer: C

Characters are grouped into tokens by the scanner.

The scanner, or lexical analyser, is the first phase of a compiler. It reads the source program character by character and groups the characters into tokens such as keywords, identifiers, constants and operators, removing white space and comments.

The parser takes the tokens and checks that they form valid syntax. Code optimisation improves intermediate code, and the code generator produces target code; both come much later than token formation.

Discuss in Forum

Which global variable in LEX points to matched string and which global variable contains the length of the matched string ?

A.yylex, yylval
B.lextext, lexval
C.yytext, yyleng
D.lexyy, yylex

Exam Year(s)

MH SET - 2018, SET Exam - 2018,

Answer: C

The variables are yytext, yyleng.

In a LEX-generated scanner, yytext points to the text of the lexeme just matched, and yyleng holds its length. Actions in the rules use them, for example to copy the matched identifier or number.

yylex is the scanning function itself, which returns the next token, and yylval is the variable used to pass a token's value to a YACC parser. lextext, lexval and lexyy are not LEX's global variables (lex.yy.c is the name of the generated file).

Discuss in Forum

The number of tokens in the following C statement is

printf("i = %d, &i = %x", i, &i);

A.3
B.26
C.10
D.21

Exam Year(s)

Tripura Public Service Commission - 2019,

Answer: C

The statement has 10 tokens.

A string literal is a single token however much it contains, so the statement splits into:

printf | ( | "i = %d, &i = %x" | , | i | , | & | i | ) | ;

That is 10 tokens. 26 and 21 come from counting the characters inside the string as separate tokens, and 3 counts only the identifiers.

The tokens fall into the usual classes: identifiers (printf, i), a string literal, punctuators ( ( ) , ; ) and the operator &.

Discuss in Forum

The lexical analysis for a modern computer language such as Java needs the power of which one of the following machine models in a necessary and sufficient sense ?

A.Finite state automata
B.Deterministic pushdown automata
C.Non-deterministic pushdown automata
D.Turing Machine

Exam Year(s)

Tripura Public Service Commission - 2019,

Answer: A

Lexical analysis needs finite state automata.

The tokens of a programming language, identifiers, keywords, numbers, operators and string literals, are all described by regular expressions. Every regular language is recognised by a finite automaton, so finite automata are sufficient; and since tokens form regular sets, nothing weaker would do, so they are also necessary.

Pushdown automata are needed for parsing the nested, context-free syntax, which is the next phase, and a Turing machine is far more power than lexical analysis requires.

Discuss in Forum

In analyzing the compilation of PL/I (Programming Language One) program, the term "Lexical analysis" is associated with

A.recognition of basic syntactic constructs through reductions
B.recognition of basic elements and creation of uniform symbols
C.creation of more optional matrix
D.use of macro processor to produce more optimal assembly code

Exam Year(s)

Tripura Public Service Commission - 2019,

Answer: B

Lexical analysis is associated with recognition of basic elements and creation of uniform symbols.

In the classic model of a PL/I compiler, the lexical phase scans the source, recognises its basic elements (identifiers, literals, operators and keywords) and replaces each by a uniform symbol, entering identifiers and literals in tables for later phases.

Recognising syntactic constructs through reductions is syntax analysis, and producing optimised matrices or assembly code belongs to the optimisation and code generation phases.

Discuss in Forum

In a compiler, keywords of a language are recognized during

A.parsing of the program
B.the code generation
C.the lexical analysis of the program
D.dataflow analysis

Exam Year(s)

Tripura Public Service Commission - 2025,

Answer: C

Keywords are recognised during the lexical analysis of the program.

The lexical analyser (scanner) reads the character stream and groups it into tokens such as identifiers, numbers, operators and keywords. Keywords are usually recognised there, often by first matching an identifier and then looking it up in a table of reserved words.

The parser works on the tokens the scanner has already produced, and code generation and dataflow analysis come much later in the compiler.

Discuss in Forum

Which of the following is not a white space character ?

A.BLANK
B.TAB
C.SEMICOLON
D.NEWLINE

Exam Year(s)

Tripura Public Service Commission - 2026,

Answer: C

The character that is not white space is the SEMICOLON.

White space characters are those that produce blank space and separate tokens: the blank (space), horizontal tab, newline, and also vertical tab, form feed and carriage return. In C the function isspace() returns true for exactly these.

The semicolon is a printing punctuation character. In C and C++ it is the statement terminator, a token in its own right, not white space.

Discuss in Forum

10.

The principal task of the lexical analyser is to

A.group the input characters into lexemes and produce tokens
B.verify that the program conforms to the grammar of the language
C.work out what type each expression in the program will take
D.emit machine instructions corresponding to the source statements

Answer: A

Correct answer: group the input characters into lexemes and produce tokens

The scanner is the compiler’s first phase and its job is purely one of recognition: read the character stream, decide where one lexeme ends and the next begins, and hand the parser a stream of tokens with their attributes. Checking the grammar belongs to the parser, type checking to semantic analysis, and instruction selection to the back end.

Discuss in Forum

Loading…

Topics

Other Subjects

Tags: Lexical Analysis Questions and Answers || Lexical Analysis MCQ Questions and Answers || Lexical Analysis GK Questions and Answers || Lexical Analysis GK MCQ Questions || Lexical Analysis Multiple Choice Questions and Answers || Lexical Analysis GK || GK on Lexical Analysis || Compiler Design Questions and Answers || Compiler Design MCQ Questions and Answers || Compiler Design GK Questions and Answers || GK on Compiler Design

Lexical Analysis Questions and Answers

Topics

Other Subjects

About Us

Stay informed about all important Dates & Notifications