Tokens

Laboratory 3, January 29

Special Note

Be sure in the future to bring the resources to lab that you would like for program development, such as the textbook, class notes, a C++ or Java book as appropriate, and so forth. You can bring less individually if you organize who brings what in your team.

Turning in Your Project

Turn in the project printout due at the beginning of this laboratory. Do not staple this printout to this laboratory's worksheet. The two need to be kept separate.

Objectives

The objectives of this laboratory exercise are

to acquaint you with the definition of µPascal;
to give you experience with EBNF;
to ensure that your team can use cvs to manage your project properly; and
to help you get started with your scanner implementation.

Preparation

Be logged on to esus. Print a cover sheet to use as your lab worksheet with the following information printed in a large font and centered

Your group number
Lab 3
Today's date
Your team names and esus e-mail addresses

Do not write on this page. This is a cover page only. Staple the remaining pages that you use to complete this lab to this page.

To Do

Remember that this is a working laboratory. You should be actively collaborating with other team members, and you should feel free to ask the TA any questions regarding technical aspects of the project.

Check out the EBNF and Tokens definitions for mPascal. They can be found on the Resources page on this web.
Find the token names float_literal and identifier in the tokens resource page.
- A float_literal is a string that has one or more digits followed by an optional exponent, or
  it is a string that has one or more digits, followed by a decimal point, followed by one or more digits, followed by an optional exponent. If present in either case, the exponent is formed by including the letter e or E, followed by an optional + or -, followed by one or more digits.
- An identifier must start with a letter or underscore, and following that may contain letters, digits, and the underscore character. An identifier may not end with an underscore character and may not have two underscore characters in a row.
Write down regular expressions in EBNF form for float_literal and identifier on your worksheet.
Make a regular expression using EBNF notation for white_space. Include this on your worksheet.

Construct finite state automata for float_literal, identifier, and white_space on your worksheet. Use the fsa-animator tool to do this.
On your worksheet give a pseudocode design of a procedure for float_literal that implements the finite state automaton of the previous bullet using a case based approach as given in Monday's lecture. Remember that this is pseudocode, so you can use case statements for the design in a way that your actual programming language may not directly allow.

Recall that the form of the dispatcher given in class (look at the lecture again on the Web if necessary). When a call to scan a new token is received, the dispatcher first calls a routine to skip white space and then examines the first non-white space character to determine which fsa to call. Work together on a separate sheet on a design for the routine Skip_White_Space. Notice that end of line (eoln) characters are considered to be white space, so they would be encountered during the execution of Skip_White_Space. This means that Skip_White_Space will need to keep track of the current line and column numbers.
In any remaining time, delegate responsibilities for implementing the complete scanner, due next week, and begin design and implementation of the rest of the scanner. The team leader should be responsible for coding the dispatcher and the routine for skipping white space. The individual fsa scanning stubs should be divided up evenly among the remaining team members. Be sure that each individual has at least one "interesting" fsa to complete. Also, include the new definitions of identifier and float_literal as new tokens in the scanner (these will be modifications to the mPascal definition).

To turn in

As usual, staple all of your worksheets together and make sure the first page has your team identification exactly as described above. Turn in theses worksheets, which should also include the division of labor for team members for the upcoming assignment.