OSE - Makeit User Guide

Graham Dumpleton
Dumpleton Software Consulting Pty Limited
PO BOX 3150
Parramatta, 2124
N.S.W, Australia
email: grahamd@nms.otc.com.au

Table of Contents

Optional Modules
OSE - Makeit User GuideOptional Modules

Optional Modules

1 Introduction

Additional support is included into makeit by listing the names of the modules which you require. This chapter describes the major modules which are available in the standard environment.

2 Shell Scripts

Support for shell scripts is enabled by listing `sh' in the MODULES variable. If a directory contains any shell scripts, the shell scripts must have a `.sh' extension.

For a shell script to be converted into an executable program, the name of the program needs to be listed in the PROGRAMS variable. The name of the program is determined by dropping the `.sh' extension from the name of the shell script.

When the `programs' or `all' target is given to makeit, any shell scripts with the program name listed in the PROGRAMS variable will be copied to the makeit subdirectory and made executable. In the process of copying the shell scripts, you may optionally pass the scripts through a filter program to perform substitutions on the body of the script.

To process the shell scripts with sed, the variable SHFILTER is defined to the command to run. For example, if the pattern `@MKTAG@', was to be replaced with the tag for the platform being used, SHFILTER would be set to the following:

  SHFILTER := sed -e `s/@MKTAG@/$(MKTAG)/'
For backward compatability only, the variable SHPATSUBST is retained. The variable SHPATSUBST though, should not be used, as it will be removed in the next major version of OSE. The SHPATSUBST variable allowed the listing of only the required substitution pattern, with sed automatically being run.

For example, to perform the above substitution, as well as change the pattern `@OSE_HOME@' to the location of OSE, you would specify:

  SHPATSUBST := `s/@MKTAG@/$(MKTAG)/' \
`s%@OSE_HOME@%$(OSE_HOME)%'
The character `%' is used as the separator for the pattern in this case as `/' may appear in `$(OSE_HOME)', which would cause the substitution pattern to fail. Note that if the SHPATSUBST variable is used, no spaces can appear anywhere in the pattern. It is for this reason that SHFILTER should now be used and why SHPATSUBST will be removed.

3 Yacc/Lex

Support for the code generators Yacc and Lex are enabled by listing `yacc' and `lex' respectively in the MODULES variable. If a directory contains any Yacc files they must use a `.y' extension. If a directory contains any Lex files they must use a `.l' extension.

By default, the Yacc and Lex modules will generate code files with a `.c' extension for the C programming language. To have the modules generate code files with a `.cc' extension for the C++ programming language, the string `cc' must be listed in the variables YACC_OPTIONS and LEX_OPTIONS respectively. Whichever language the output files are for, a module for that language must be included by the user. Thus if C code is being generated, you must also list `c' in the MODULES variable. If C++ code is being generated you must list `cc' in the MODULES variable.

If you ask makeit to generate code files with a `.cc' extension it will try to run `yacc++' and `lex++' for the Yacc and Lex modules respectively, in order to generate the code files. If you do not have Yacc and Lex code generation programs by these names, you should set the variables YACC and LEX to the name of a program which will generate code files which can be compiled with a C++ compiler.

The treatment of Yacc and Lex files is similar. When the appropriate tool is run, a code output file is generated which is placed into the makeit subdirectory. The name of this code file will have the same basename part as the input file. For example, if the input file for Lex was `lexan.l', the output file will either be `lexan.c' or `lexan.cc' depending on whether `cc' had been listed in the options variable for that module. Since the name is calculated by dropping the input file suffix and adding a suffix for the language being generated, you should not have in one directory, Yacc and Lex input files, with the same basename part. Doing so will result in the output file of one, overwriting the other.

By default, output code files from Yacc and Lex are treated as if they were listed in the NONLIBSRC variable. This means that in order for the files to be generated and compiled you will need to add a dependency to your makefile indicating with which program the compiled code output file should be linked. For example, if the object file generated from compiling the file `lexan.c', generated from `lexan.l', needs to be linked with the program `myprogram', you would add the following at the end of your makefile,

  $(MK)/myprogram : $(MK)/lexan.o
Normally, you would not wish to place the object file, created by compiling a Yacc or Lex output file, into a library. As a result the default will not do so, and is the reason why you need to add the above dependency. If, however, you do need to have the object files placed into the library, you should add the appropriate definition,

  LEX_OPTIONS := archive_in_library
or

  YACC_OPTIONS := archive_in_library
If you use these definitions, in general you cannot have more than one Yacc or Lex file in a directory. This is because when you attempt to link with the library, the linker will complain about multiply defined symbols. If you have a Yacc or Lex program which allows you to add a definition into your input files, in order to get it to add a prefix to all the function and variable names, this restriction does not apply.

An alternate approach to relying on your Yacc or Lex program to add different prefixes to variables listed in the generated scanner or parser, is to pass the output of Yacc or Lex through a filter program. The purpose of this filter program is to change the names of the variables. Passing the output of Yacc or Lex through a filter program, can be achieved by defining the variables YACCFILTER and LEXFILTER to that of a program to run. For example:

  YACCFILTER = sed -e \
`s/[^_a-zA-Z]yy/$(basename $(notdir $@))_yy/'
This will result in all names in the output from Yacc, which commence with `yy', being prefixed with the basename part of the name of the Yacc input file.

Note that `$(basename $(notdir $@))' needs to be evaluated at the time that the action is run, as `$@' only has a valid value at that time. Therefore, `=' is used to define the variable instead of `:=' as it results in the evaulation of the variable YACCFILTER being delayed until the point the action defined by the variable, is run.

If you are using the `archive_in_library' option, but want makeit to ignore certain files, the EXCLUDE variable can be used in much the same way as it is for C and C++ code. Namely, you should add to your makefile, a definition for the EXCLUDE variable, which lists the full name of the Yacc of Lex input file in it. For example:

  EXCLUDE := lexan.l
If you include the header file which is generated by Yacc into any code files, you need to add additional dependencies to your makefile. For example, if your Yacc input file is called `parser.y', a header file, `parser.h' will be generated along with the code file `parser.c'. If a library code, file `myfile.c' included the file `parser.h', you would add the following to your makefile.

  $(MK)/myfile.o : $(MK)/parser.h
If it were a program code file which included the header file, you need to add two dependencies. For example, if your program code file was called `myprogram.c', you must add the following to your makefile.

  $(MK)/myprogram : $(MK)/parser.h
$(MK)/myprogram.o : $(MK)/parser.h
An additional dependency is required to ensure that the header file generated by Yacc or Lex is created when generating dependency information. Without the dependency, information generated will be incomplete. The dependency must take the form:

  depend.setup :: $(MK)/parser.h
The default action of makeit is to link in the standard system Yacc and Lex libraries, when these modules are used. If you do not need to link in these libraries, or wish to link in versions of these libraries specific to a version of Yacc or Lex, you can set the variables YACCLIB and LEXLIB. For example, if you are using the Lex program \file{flex}, you would add the following to your makefile,

  LEX := flex
LEXLIB := -lfl
If you need to pass flags to the Yacc and Lex programs you can define them in the YFLAGS and LFLAGS variables.

4 Rpcgen

The rpcgen module supports the use of the program rcpgen as distributed with Open Network Computing (ONC) software from Sun MicroSystems. If you wish to use the module, list the name `rpcgen' in the MODULES file. Any rpcgen input files in a directory must end with a `.x' extension.

From a rpcgen input file, four output files will be generated. If the input file is named `proto.x', rpcgen generates a header file `proto.h', XDR routines in `proto_xdr.c', server side stubs in `proto_svc.c', and client side stubs in `proto_clnt.c'. When generated, these files will be placed into the makeit subdirectory.

If you want makeit to ignore completely specific rpcgen input files, you should list the full name of those files in the EXCLUDE variable. For example:

  EXCLUDE := proto.x
If you want makeit to suppress generation of specific code output files, you should list the name of the generated code file as it would appear in the makeit subdirectory. For example, if you did not define any XDR routines and therefore did not require the XDR file to be generated, you would include the following definition in your makefile.

  EXCLUDE := $(MK)/proto_xdr.c
When the output files containing the XDR routines and client server stubs, `proto_xdr.c' and `proto_clnt.c', are compiled, the generated object files will be automatically archived into the library. The server side stubs file, `proto_svc.c', is treated as if it were listed in the NONLIBSRC variable. For the server side stubs file to be compiled, you will need to add a dependency to the end of your makefile, specifying to which program it should be linked. For example, if the name of your server program is `myserver' you would need to add

  $(MK)/myserver : $(MK)/proto_svc.o
If the server side stubs file contains a main() routine, the basename part of the file should be listed in the PROGRAMS variable.

  PROGRAMS := proto_svc
This will cause the server side stubs file to be compiled directly into an executable program.

If a code file other than those generated by rpcgen, includes the header file generated by rpcgen, you need to create an explicit dependency between the object file for that code file, and the header file. For example:

  $(MK)/myfile.o : $(MK)/proto.h
Since the header file is generated, it resides in the makeit subdirectory. Your dependency has to reflect this as shown.

An additional dependency is also required to ensure that the header file generated by rpcgen is created when generating dependency information. Without the dependency, information generated will be incomplete. The dependency must take the form:

  depend.setup :: $(MK)/proto.h
By default, all the code files generated by rpcgen have a `.c' extension. If the version of rcpgen you have, supports generation of C++ code, you can list `cc' in the RCPGEN_OPTIONS variable. If you do this, you will need to ensure that `cc' is also listed in the MODULES variable. In addition, you will need to ensure that any options required by the version of rpcgen you have to generate code for C++ compilers is supplied. Any general options for rcpgen can be supplied by defining the RPCGENFLAGS variable. The option to enable generation of C++ code by rpcgen is typically `-C'.

Flags can also be passed to rpcgen when generating specific output files. The variables which should be defined to do this are listed in the table below.

--------------------------------------------------------------
Variable         Purpose                                        
--------------------------------------------------------------
RPCGENXDRFLAGS   Passed to rpcgen when generating the           
                 XDR routines file. The default value of        
                 this variable is `-c'.                         
RPCGENSVCFLAGS   Passed to rpcgen when generating the           
                 server stubs file. The default value of        
                 this variable is `-s tcp -s udp'.              
RPCGENCLNTFLAGS  Passed to rpcgen when generating the           
                 client stubs file. The default value of this   
                 variable is `-l'.                              
RPCGENHDRFLAGS   Passed to rpcgen when generating the           
                 header file. The default value of this         
                 variable is `-h'.                              
--------------------------------------------------------------
If you do define these variables, you should make sure you include the original set of options in what you define, unless you do intentionally wish to completely override the options. Totally overriding the options passed to rpcgen generally is only necessary for the server side stubs file. For example,

  RPCGENSVCFLAGS := -I -K 20
creates a server which can be started from inetd and which exits after 20 seconds of inactivity.

If you need to have options passed to rpcgen when generating any of the files, the variable RPCGENFLAGS should be used. The default name of the rpcgen program is rpcgen. If you need to override this, you should define the variable RPCGEN.

5 Installation

Installation procedures often vary from one project to another. The `install' module supplied with makeit should be general enough to cater to most cases. In order to be able to use this module, you should list `install' in the MODULES variable. Once configured, the installation procedure can be triggered by executing makeit with the target `install'. Before makeit installs any files, it first ensures that the files to be installed are up to date. If necessary it will automatically rebuild anything which is to be installed.

The types of files which the module can be used to install are described in the following sections.

5.1 Executables

The destination directory for program executables is specified by defining the variable BINDIR in your makefile. For example:

  BINDIR := /usr/local/bin
By default, programs installed will include both those generated as a result of being listed in the PROGRAMS variable, and those which are created from generated source code. If you do not want all these installed, say for instance because some of them are test programs, the EXECUTABLES variable can be defined.

When programs are copied into the destination directory, the `install' program supplied on BSD systems will be used. On those platforms which `install' is not available, or which have the SYSV version of the `install' program, the `cp' program will be used. If the `install' program is available, programs will be installed with file permissions of `0775'. If it is necessary to change the permissions given to a program when it is installed, you can set the variable INSTALL.BINFLAGS. The default setting for the variable is `-c -m 0775'. One change you might like to make to this variable is to add the option to make the `install' program strip the program executable, when it is copied to the destination directory. For example:

  INSTALL.BINFLAGS := -c -s -m 0775
Note that the names of the programs listed in the EXECUTABLES variable should include only the name of the program; it should not include any directory name. When makeit goes to install the programs though, it will expect to find them in the makeit subdirectory. If you are defining your own rules, you will need to make sure that they are placed into the makeit subdirectory so makeit can find them.

5.2 Libraries

The destination directory for both standard and shared libraries is specified by defining the LIBDIR variable in your makefile. The library will only be installed, if you also define the name of the library in the LIBRARY variable. The name of the library should not include any `lib' prefix or `.a' suffix. For example, if your library is called `mylib', set LIBRARY as follows:

  LIBRARY := mylib
If libraries need to have ranlib run on them after being installed makeit will do it automatically. Although the variable INSTALL.LIBFLAGS can be modified to change the flags passed to the `install' program from the default of `-c -m 0664', you need to remember that ranlib will only be successful if the library is writable. Therefore you should not change the mode to remove write permission on the library when it is installed.

When the standard library is installed it will be installed with a name equivalent to the expansion of `lib$(LIBRARY).a'. The name used for the shared library, when installed, will depend upon the platform being used. On SYSVR3 machines, the name of the shared library will be either `lib$(LIBRARY).sl' or `lib$(LIBRARY)_s.a'.

On Sun and SYSVR4 machines, a shared library version number will also be encoded into the name of the library. The version number must be provided in your makefile by setting the LIBVERSION variable. The name of the shared library will thus be `lib$(LIBRARY).$(LIBVERSION).so'. A symbolic link will also be created from the library with the version number encoded, to the name `lib$(LIBRARY).so'. If there was a library of this name previously linked to an older version of the shared library, the old link will be removed first. Note that the library version number must provide at least a major and minor version number. For example:

  LIBVERSION := 1.0

5.3 Include Files

The destination directory for include files is specified by defining the INCDIR variable in your makefile. By default, makeit will copy all files with an extension of `.h', `.hh', `.hxx', `.h++' and `.H' from your directory to the installation directory. If you wish to specify which header files should be installed, you can set the INCLUDES variable in your makefile. An instance of where you would want to do this, is if you had template classes and needed to install the template code files as well as the header files. For example:

  INCLUDES := $(wildcard *.h *.hh) vector.c
If the `install' program is used to copy files into the destination directory, it will use the options `-c -m 0444'. If you wish to override the mode used when the files are installed, you can set the INSTALL.INCFLAGS variable in your makefile.

When the header files are installed they are not modified in any way. If you want to modify header files as they are installed, you will need to construct your own variant of the `install' module.

5.4 Manual Pages

The root destination directory for manual pages is specified by defining the MANDIR variable in your makefile. When manual pages are installed, subdirectories will be created under the root directory you specify, corresponding to the different sections of the UNIX manual page system. For example, manual pages from Section 1 will be placed into a directory `man1' underneath the directory specified in MANDIR.

To specify which files should be installed into a particular section of the manual page directory, you need to list them in a variable in your makefile. For Section 1 manual pages, you should list the names of the files in the SECTION1 variable, Section 2 manual pages in the SECTION2 manual page and so on up to Section 8.

When makeit searches for the manual pages, it expects to find them as they are listed in the variable listing them. For example, if you kept the manual pages in the working directory, you would define,

  SECTION1 := $(wildcard *.1)
If you kept the manual pages in subdirectories corresponding to the section of the manual into which they should be installed, you would define,

  SECTION1 := $(wildcard man1/*.1)
When the manual pages are installed, the names of the files are not changed, thus the files should have the correct extension for the section of the manual page directory in which they are being installed. For example, Section 1 manual pages should have a `.1' extension.

If you also have a `mandesc' file, you should define the variable MANDESC in your makefile to the name of the file. When installed, this file will be placed into the directory specified by MANDIR and will be called `mandesc'. The `mandesc' file is used by manual page browsers such as xman and is used to describe what the manual pages describe. For the full details of what to place in a `mandesc' file consult the manual page for xman. A simple example of a `mandesc' file is included below.

  no default sections
1(1) OSE 4.0pl0
3(3) OSE 4.0pl0
If the \file{install} program is used to install the manual pages and `mandesc' file, the options passed to `install' can be altered by defining the INSTALL.MANFLAGS variable in your makefile. The default value of the variable is `-c -m 0444'.

5.5 Auxiliary Files

The `install' module also supports the installation of auxiliary files. This may be used to install data files in `lib' or `etc' directories. The actual destination directory is specified by defining the AUXDIR variable in your makefile. Any files which should be installed should be listed in the AUXILIARIES variable in your makefile. By default, the `install' program, if used, will use the options `-c -m 0444'. To override the options passed to the `install' program, you can set the INSTALL.AUXFLAGS variable in your makefile.

5.6 Additional Directories

Each of the above provides a variable to specify the name of the installation directory for that category of file. If need to have additional directories created, be they parent directories of those given for the above, or are needed for your own explicit rules, you can list these directories in the INSTALLDIRS variable. Makeit will sort the names of all directories to ensure that parent directories are created before any subdirectories.

6 Test Environment

Support for a simple scheme to perform program testing can be included by listing `check' in the MODULES variable. The scheme is particularly suited for constructing test programs for single classes or small groups of classes, where the success of a test can be determined by comparing the output of the test program with pre-determined output. The scheme is not suited for testing real time or interactive applications.

6.1 Program Test Harness

To be able to run tests on classes, you need to construct a test harness. This consists of a main() routine and some mechanism for selecting a test to run. It has been found that it is better to include a number of tests within a test harness, but for them to be run independently. This means that you can test individual features of a class, as well as testing the class as a whole, simply by writing appropriate tests.

A simple test harness adequate for this purpose is included below. Note that it is placed in a file of its own and would include the header file of the class being tested, in addition to those shown.

  #include <OTC/OTC.h>
#include <iostream.h>
#include <stdlib.h>

void test1() { }
void test2() { }

typedef void (*testFunc)();

testFunc tests[] =
{
test1,
test2
};

void testTerminate()
{
exit(-1);
}

main(int argc, char* argv[])
{
u_int const numTests = sizeof(tests)/sizeof(tests[0]);

set_terminate(testTerminate);

if (argc != 2)
{
cout << numTests << endl;
return 1;
}
else
{
int testNum = atoi(argv[1]);
if (testNum > 0 && u_int(testNum) <= numTests)
{
tests[testNum-1]();
return 0;
}
else
return 1;
}
}
The name of the file in which the test harness is placed, is selected by the user. It it suggested that you settle on a particular naming convention across all projects for consistency. The naming convention used within OSE is for the file to have a similar basename as the file including the class being tested, but to prefix the name with an underscore. For example, the test harness for the OTC_String class which is contained in the header file `string.hh' would be called '_string.cc'. The benefit of this scheme is that all test harnesses appear at the start of the directory listing, rather than intermixed with other files.

Whatever naming convention you use, the name of the resulting executable must be listed in the description file for makeit.

  PROGRAMS := _string
This will ensure that the file is compiled into a program and not mistakenly placed into the library.

Once compiled, the above test harness, when run, expects a single argument which is the number of the test to run. Test numbers start from `1' and the number of tests which exist can be determined by running the program with no arguments. The above example shows only two tests; to add more, it is a simple matter of adding extra test functions and adding the names of the functions to the `tests' array.

6.2 Test Functions

The first thing that you should do in a test function is to create an instantiation of the OTC_Tracer class. This is valuable, as it delimits exactly when the function is entered and exited. Do not use the OTCLIB_DOTRACE() or OTCLIB_TRACER() macros; use the class directly so that it can never be compiled out of the test harness.

  #include <OTC/debug/tracer.hh>

void test1()
{
OTC_Tracer tracer("void test1");

// ...
}
Exactly what tests you should carry out, will depend a great deal on the complexity of the class. If, for example, the class is purely an encapsulation of data with no processing occurring, ie., it is primarily accessor functions, there is little point in doing testing.

Testing is more than viable when member functions can be invoked, more or less independently. In these situations, it is relatively easy to construct tests. At the other end of the scale is, where there is some complex interaction between member functions of a class. Constructing tests for these cases is more difficult and interactive testing may be more suitable, as it allows the user to invoke operations in any order, depending on output from previous operations.

In general, tests should cover all possibilities of creating an object and should in some way, exercise all member functions of the class. An attempt should be made to test only one feature of the class in each test. Each test should only make use of features which have previously been tested.

6.3 Input and Output Files

Although test harnesses are useful when debugging classes during development, they should not be used for just that. It is important to have an infrastructure, so that as more features are added to a class, or bugs fixed, that all tests can be rerun to ensure correct operation of the class. This can only be achieved successfully if a record is kept of the input for tests and the expected output. In addition to this, there needs to be a way of running the tests automatically, to eliminate the laborious task of running them manually.

In support of this, the module provides the target `check', which when invoked, results in a search for output files against which tests can be compared. For each of these files, the appropriate test is run using an optional input file. In order for this to work, a strict naming convention has to be followed. The naming convention is that the output file for a test should have a basename, equivalent to the name of the executable which should be run, suffixed by `.out.' and the number of the test. For example, for the `_string' test harness the output files would be called:

  _string.out.1
_string.out.2
_string.out.3
...
Any input files should have the same name except that `out' is replaced with `in'.

  _string.in.1
_string.in.2
_string.in.3
...
Given the above, the `check' target would cycle through the tests executing them in the following manner:

  _string 1 < _string.in.1 2>&1 | diff - _string.out.1
Worth noting, is that the standard error is redirected onto the standard output, and is used in the comparison with the expected output. A test will only be run if an output file exists, however, it is optional for an input file to exist.

If changes are made to a test, and as a result, the output file needs to be updated, makeit can be invoked with the name of the test file as the target. You can use this method initially, to create an output file, by creating an empty test file, and running makeit to update it.

  touch _string.out.4
makeit _string.out.4

6.4 Short Suffixes

If you are working on a version of UNIX which only allows a maximum of 14 characters for filenames, the `in' and `out' extensions for test harness files are often too long. If this is the case, a shorter extension can be used. This feature is enabled by listing `short_suffixes' in the CHECK_OPTIONS variable. The result is that the extensions `i' and `o' are used instead. For example, `_string.o.1' and `_string.i.1'.