A Summary of the Features of SRE-FILTER ver 1.11

This outline descibe the features and options of the SRE-FILTER (ver 1.11) HTTP (Web) server package for the GoServe Internet server.

More detailed information is available in a long (>150K) documentation file. This outline is meant to indicate what is available, and as a synopsis/cheatsheet for current users.

In a nutshell, SRE-FILTER has the following features:

The SRE-FITLER niche is the non-cutting edge, small-to-medium load site being maintained by non-professionals who want a fair degree of functionality in an easily maintained system.


Initial Setup

All variables, file names and directory names mentioned here are, unless otherwise specified, set in the INITFILT.80 file. They can be changed either by editing INITFILT.80, or (in most cases) by using FILTINST.HTM. For more details on the parameters contained in INITFILT.80, please see INITFILT.DOC.


After unzipping the SRE-FILTER package (SREFILTR.ZIP) to a temporary directory, initial setup is accomplished with SRE-FILTER's INSTALL.CMD program (a REXX program). At installation, the user provides the names of several directories (or chooses the recommended defaults).

User configurable files

The INSTALL program will create several REXX files (SREFILTR.80 and INITFILT.80), and a "macrospace" procedure library (SREFPRC1.RXL). In addition, several "user-configurable" parameters/options files are installed.

Although the names of these user-configurable files can be changed by the user, in this summary we refer to them by their default names .

Parameter initialization

INITFILT.80 contains a number of user-configurable parameters; many of which are discussed below. These parameters can be modified either by editing INITFILT.80 (with a text editor). More conveniently, after installation of SRE-FILTER you can use FILTINST.HTM -- which provides a fill in form listing the major SRE-FILTER parameters.

KEYPHRASE Support

Two files are used for keyphrase support (keyphrases are used in Server Side Includes, see below for details).
  1. REPSTRGS.IN -- for customized REPLACEment strings.
  2. INTERPET.IN -- for REXX-code blocks used by the INTERPRET keyphrase.

ALIAS creation

ALIASES.IN -- A user configurable set of aliases used for document redirection, implementing searchable indices, and a variety of other tasks.

Virtual directories

A user configurable list of "virtual directories", consisting of a URL substring and an actual drive:directory.

Auditing files

Two files are used to keep track of the the number of times your documents have been requested:
  1. COUNTER.CNT -- keep track of files that have a REPLACE HITS or a REPLACE COUNTS keyphrase.
  2. RECRDALL.CNT -- keep track of all files (when RECORD_OPTION is active)

Logon and Access control files

USERS.IN is a text file containing a list of usernames,passwords, and privileges.

Two other files are used to provide "access control" on a per-request basis:

  1. ALL_FILE.CTL -- access control, used when ACCESS_ALLOW is active
  2. SENDFILE.CTL -- access and auditing for files transfered using the SENDFILE facility.

External REXX procedures

SRE-FILTER comes with several files that provide useful features (facilities):
  1. GETAFILE.80 -- A file viewer and transfer facility
  2. DOSEARCH.80 -- A simple but powerful string search utility for text files.
  3. POSTFILT.80 -- A sample of a "post-filter": an event-driven e-mail notifier.
  4. PREFILTR.80 -- A sample of a "pre-filter"; it enables access to the GOREMOTE "server remote control" package.
  5. STATUS.80 -- A sample "server side" include program

Logon controls

SRE-FILTER provides several logon controls. The primary control is provided by the CHECKLOG variable, with possible values:

Defining classes of users

You can pre-specify the "user-class" of clients using using their IP addresses. This provides a means of permitting, or denying, access on a client-by-client basis (without the use of an explicit logon).

Superusers

OWNERS are defined using the OWNERS variable: which should contain a space delimited list of IP addreses. OWNERs are given SUPERUSER privileges; which allows uninhibited access to all files and facilities of SRE-FILTER.

In-House Users

The IP addresses, and logon privileges, of IN-HOUSE users are defined using the INHOUSEIPS. "stem" variable. The wildcard character, *, can be used in any of the domain fields. Thus, INHOUSEIPS.1="*.*.*.*" means "everyone is an IN-HOUSE user". IN-HOUSE users have more privileges then the public, though not as many as OWNERs. Note that OWNERs are automatically granted INHOUSE privileges.

Unallowed Users

In some cases, there may be individuals (or groups), with known IP addresses, that you want to keep out. This can be accomplished using the UNALLOWEDIPS.n variable (uses the same syntax as INHOUSEIPS.n).
Note that SRE-FILTER first check if the client is an OWNER or an IN-HOUSE user; if neither is true, UNALLOWEDIPS.n is checked. Thus, setting UNALLOWEDIPS.1="*.*.*.*" will keep everyone (except OWNERS and IN-HOUSE clients) out.

Domain Name Checking

Optionally, you can check the name of the requester. This is a security measure designed to limit entry to clients with "fake" IP addresses.

The USERS_FILE

When "logon" is required, USERS.IN is checked for username, password, and privilege information. Note that a privilege of INHOUSE or SUPERUSER gives INHOUSE and SUPERUSER privileges (respectively) to a client with this username/password.

Note that logon is never required of OWNERS and IN-HOUSE clients.

PUBLIC_FILES

In cases where you want to allow free access to a limited set of resources, a set of PUBLIC_FILES can be specified. These are files (and directories) that are available to everyone, with or without logon privileges.

Finding the requested document

In many cases, a request will not specify a document, or specifies a document that can not be found. SRE-FILTER has several means of guessing what the client really wants.

Note that in most cases requested files are assumed to be "relative" to the data directory. This is a security feature of SRE-FILTER that is typical of most servers.

The DEFAULT document

In the extreme (and typical) case the client asks for nothing (an empty request string is recieved). When this occurs, the DEFAULT variable is used; it should contain the name of your "home page" document (traditionally, INDEX.HTM is used as your "home page").

Default documents for directories

Upon recieving a request string that specifies a directory (that ends with a /, or that ends with a name containing no extension), the AUTO_NAME variable is examined to ascertain which document should be used. AUTO_NAME should contain a space delimited list of file names; SRE-FILTER will check each name in succession, stopping when it finds a file name that exists (by default, SRE-FILTER will look for these files in the "directory" selected in the request string).

The * character (by itself) tells SRE-FILTER to look for a file with the directory.HTM name; where directory is the "last" name in the directory requested (that is, if PROJECTS/HARRY/ is requested, * will mean HARRY.HTM}.

Specifying a HOME directory

The HOME_DIR variable is used whenever a ~ is seen in the request string -- the contents of HOME_DIR are substituted into the request string. Thus, if HOME_DIR=USERS/PERSONAL/, then ~GERALD/CV.HTM becomes USERS/PERSONAL/GERALD/CV.HTM.

Specifying a "virtual" directory.

By default, files are assumed to be in the default data directory, and server side programs are assumed to be in the working directory. While a good security feature (files not in these directories are unaccessible), this can be an inconvenience. To remedy this inconvenience, one can define "virtual directories" using VIRTUAL.IN. Basically, SRE-FILTER will compare the "starting portion" of a URL to see if it matches an entry in VIRTUAL.IN. If it does, the "candidate directory" portion of this entry is used (the portion of the URL after the "starting portion" is used as the file name). Thus, you can make available a wide, but controllable, set of directories on (or LAN accessible from) your server

Redirection using ALIASES

The ALIASES.IN file is used to redefine specific request strings. Among it's many uses is internal and external URL redirection. Internal redirection is used to "map" the requested URL (which might be an easily remembered string) to a different URL on your server (which might be a file deep in a directory tree). External redirection is used when a file has been moved to a new server. Note that "wildcard matching" and "wildcard matching with substitution" can be used in the ALIAS file, which allows requests for any document in a (perhaps no longer existent) directory to be redirected to a new location.

Document not found?

If, despite the above tricks, the requested document can not be found, a "not found" document is automatically returned. You can specify a short message to include in this "not found" document by changing the NOT_FOUND_URL variable -- it should contain a link to some default document (say, your home page).

To facilitate this, you can use the HOME_NAME variable in the definition of NOT_FOUND_URL. HOME_NAME should contain the "colloquial" (not necessarily IP) name of your site.


Server Side Includes Using KEYPHRASES

A major advantage of SRE-FILTER is the ease with which a variety of Server Side Includes can be invoked-- just by adding special keyphrases to your document.

Two types of server side inclusions are supported:

  1. SRE-FILTER syntax:

    SRE-FILTER syntax for server side includes is:
    DELIM1 KEYWORD ARGUMENT DELIM2
    DELIM1 and DELIM2 can be either (respectivley):

  2. { and },
  3. or <!-- and -->
  4. NCSA HTTPD syntax:

    Most of the NCSA HTTPD server side include commands are supported. These are invoked using a syntax of:
    <!-- #command option="value" -->


Headers and Footers

In addition to Keyphrases, you can specify the automatic addition of a Header and a Footer to all documents.
Headers are set in the HEADER. "stem" variables,
Footers are set in the FOOTERS. "stem" variables.

Headers and Footers can contain keyphrases.

String REPLACEments

The REPLACE keyphrase is used to add static and dynamically determined strings to your document.

Dynamic Strings

Dynamic strings are determined at the time of the request. Typically, they are variables that SRE-FITLER (or GOSERVE) determines on-the-fly. Examples of such variables are the time of day, the number of "hits" for the document, and the creation date of the document.

SRE-FILTER provides a number of these variables, which we list here (see the main documentation for details:

DATE DATEGMT TIME TIMEGMT CREATION CREATION_DATE CREATION_TIME READ_HEAD SERVER USERNAME BROWSER REFERER HITS COUNTS OPTION_COUNTS OPTION_HITS READ_HEAD URL.

Static Strings

Static strings are predetermined; one could enter their contents directly into your documents. However, an approriate REPLACE keyphrase may be easier to remember. More importantly, if a phrase occurs in a number of documents, and it changes on a frequent basis (say, a daily message), then one can change the phrase just by changing the appropriate static variable rather then editing a zillion documents.

There are a few static variables built into SRE-FILTER, that can be modified in INITFILT.80:

WEBMASTER HOME_NAME INHOUSE.n SUPERUSERS.n FILTER_NAME
FILTER_NAME is set in SREFILTR.80, and is not designed to be changed.

In addition, you can add your own "static" variable names to the REPSTRGS.IN file. Although the replacement strings specified in REPSTRGS.IN names can be several lines long, for longer "inclusions" it is probably better to use the INCLUDE keyphrase.

FILE includes

The INCLUDE keyphrase is used to include files into the requested document. The included files can be anywhere, they need not be in the data directory.

OPTION includes (to access information in the Request String)

OPTIONs are used to pass information, from a client, to be incorporated into an HTML document. Specifically, options may appear after a ? character (the ? is after the document name), with each of several options seperated by a & character.
With appropriate specification of the invoking URLs, this provides some ability to customize documents (say, with the name of the document from which the request occurred).

INTERPRET include (running REXX code blocks)

The INTERPRET keyphrase is used to execute REXX code blocks, and insert the results into the document. This provides a simple means of doing a limited amount of Server Side Processing; especially when the results may be required by different documents. In a sense, the INTERPRET keyphrase allows one to create new "dynamic" variables, without delving into the SRE-FILTER source code!

For example, the USE_TABLE code block in INTERPET.IN provides a simple mechanism of automatically deciding whether to send a "TABLE containing" document, or a "non-TABLE containing" document, to a client (based on the name of the client's browser).

SELECTive includes

The SELECT keyphrase is a little different: it dictates whether the "block of HTML code" that follows the SELECT keyphrase is to be retained. Thus, it is best thought of as a conditional "server side exclude". In a sense, the SELECT keyphrase is similar in function to the combination of INTERPRET and INCLUDE keyphrases discussed above.

NCSA HTTPD style includes

Most of the NCSDA HTTPD server side include options are supported. These include: FLASTMOD, FSIZE, ECHO (most variables), and CONFIG (most options). EXEC is rudimentarily implmented (it treated as an INTERPRET FILE AFILE.RXX keyphrase).

Automatic response header creation

If desired, SRE-FILTER will parse a requested document, extract all LINK and META HTTP-EQUIV elements found in the <HEAD>, and create the appropriate response headers. This is especially useful in response to HEAD requests.

Imagemaps,and Searchable Indices, and Server Side Processing

Imagemaps

SRE-FILTER provides support for server side processing of images maps using NCSA style .MAP files. This done by
  1. specifying a <ISMAP>able image as the target of a URL;
  2. the "location portion" of the URL contains a MAPIMAGE/ substring.
  3. the remainder of the "location portion" points to a MAP file
SRE-FILTER uses this .MAP file, along with the pixel location (that is found after the ? in the request string), to perform document redirection to the appropriate URL.

Searchable Indices

Searchable indices are supported in SRE-FILTER by using an ALIAS. In combination with the DOSEARCH facility, it is quite easy to set up a searchable index against an ASCII text file (that can be anywhere on the server). For example, by creating the following ALIAS:
SEARCH2.HTM DOSEARCH?RESOURCE.LST
a request for SEARCH2.HTM will invoke (and execute) a searchable index against the text file RESOURCE.LST (assuming that SEARCH2.HTM contains an <ISINDEX> tag).

Server Side Processing

The addition of custom-written Server Side Processing routines is a easily accomplished with SRE-FILTER. These routines are best implemented with REXX procedures, but it is not difficult to invoke any executable from within SRE-FILTER.

CGI-BIN programs

SRE-FILTER supports CGI-BIN programs. It uses procedures adapated from the GoHTTP package. REXX procedures and binary executables can be readily invoked -- in the simplest case, just byjust by copying the program to the CGI_BIN_DIR directory. minimal configuration.

Auditing and Access Controls

Assigning Privileges

OWNERs are automatically granted SUPERUSER and the INHOUSE_PRIVS privileges. the IN-HOUSE users are automatically given the INHOUSE_PRIVS privileges, and can be assigned client specific privileges. For all others, a privilege list is read from the USERS.IN file (it follows the username and password). Occasionally, a logon is forced solely to collect privilege information. Lastly, everyone is granted the PUBLIC_PRIVS privileges.

Auditing Requests for File

There are several means of keeping track of the number of times files have been requested (for more thorough auditing, you can use the GOAUDIT.80 file maintained by GOSERVE).

The REPLACE HITS / REPLACE COUNTS counter file

When REPLACE HITS or REPLACE COUNTS keyphrase is encountered (in an HTML document), the COUNTER.CNT file is examined. If the requested file name is found, the "count of hits" is read and augmented. If not found, a new entry is automatically added. In addition, COUNTER.CNT can contain an optional (2-part) message to be used when displaying the "you are the nth hit" message.

RECORD_OPTION and recording all requests

The RECORD_OPTION variable is used to tell SRE-FILTER to record all request strings in RECRDALL.IN.
If RECORD_OPTION=YES, then the argument list (such as ?xxx in aurl?xxx) is dropped. If RECORD_OPTION=YES_ALL, then the request string is used as is. If you use the YES_ALL option, be aware that the RECRDALL.IN file can become quite large.
Note that wildcard matching can be very useful when requests for temporary files occur -- it reduces the clutter in RECRDALL.IN without loosing any essential information.

Hint: if a more sophisticated transaction recorder is available (written in REXX), you can "plug it in" to SRE-FILTER using the POST_FILTER option.

ACCESS_CONTROL and controlling access to your server.

The ACCESS_CONTROL variable is used to limit access to your server on a per request string basis. If ACCESS_CONTROL = YES, then request string specific "access information" is checked for all users (except SUPERUSERS). If ACCESS_CONTROL=INHOUSE, then access information is checked for non-INHOUSE (and non-SUPERUSERS) only.

ALL_FILE.CTL contains the access information. It should contain a list of request strings followed by an allowed_privilege list. When the client's request string (or a wildcard match to it) is found , this allowed_privilege list is compared to the the client's privileges: if there is a common element, the request is honored.
If there is no match in ALL_FILE.CTL, or none of the client's privileges appears in the allowed_privilege list, the request will not be honored.

Note that if the allowed_privilege list equals the * character, then the request string is always honored. Conversely, if the allowed_privilege list equals NO, then it is never honored (assuming, of course, that ACCESS_CONTROL is binding).

The SEND_FILE

The SENDFILE facility uses SENDFILE.CTL to keep track of requests, and to control access to, requested files (that is, files requested through the SENDFILE facility). It's function is similar to RECRDALL.CTL, except that when a file name is not found, transfer is allowed (and a new entry is added to SENDFILE.CTL).

Useful Facilities that come with SRE-FILTER

A few facilites, and some general purpose Server Side Processing routines, are included with SRE-FILTER.

File Uploads

SRE-FILTER supports two mechanisms for clients to transfer files to the server.
  1. Getting a URL: This requires that the client has the ability to place the files on her own web server. She can then direct SRE-FILTER to get this file, and save it in SRE-FILTER's upload directory.
  2. Uploading a file from the client's own computer: If the client is using a browser that supports the type=file FORM attribute, then she can easily transfer files directory from her own machine to the server (for example, NetScape 2.01 has this capability).

Special Requests

Several special requests are recognized as "special requests" (some of which require CONTROL or SUPERUSER privileges). These include:
!ping !statistics !host+ip.add.res.s !reset !save !varname !host !macro.

The GETAFILE file viewer and transfer facility

GETAFILE.80, invoked with a request string of GETAFILE?option_list, will display directories (and permit easy traversal of the data directory tree), and allow clients to transfer specific files. It's sort of an FTP equivalent, with extra options. These options include the ability to display the contents of a file on screen, a few tricks for automatically generating "file descriptions", and support for TABLES of entries. In addition, GETAFILE can be instructed to use the SENDFILE facility to record and control file transfer.

GETAFILE.HTM is a program that illustrates the use of GETAFILE, using an HTML FORM.

There may be cases where you do not want clients to have this sort of access to all your files. You can set the NO_GETAFILE_CONTROL to limit access to GETAFILE.

The SENDFILE file transfer facility

SENDFILE is a facility that can be used to record, and optionally control, transfer of files. It is called with a file name, and an optional "extra privilege". It uses the SENDFILE.CTL file to record information.

The DOSEARCH text file search facility

DOSEARCH.80 is a simple but powerful text-file search routine. DOSEARCH looks for occurences of words and phrases in paragraphs. It supports a variety of boolean conditions, and it provides several display options (such as highlighting matches).

The DOSEARCH.HTM demonstrates the use of DOSEARCH

DOSEARCH, when combined with an appropriate ALIAS, provides a simple means of implementing searchable indices.

The Message-Box Facility

For use as a cheap form of E-Mail, or as a no-frills bulletin board system, a simple message-box facility is built into SRE-FILTER.

A request for MESSAGE?option_list will write user-identification fields and a message (that is, the information included in the option_list) to a named message box (also noted in the option_list).

A request for !VIEWMESS?message=messbox instructs SRE-FILTER to transfer the contents of the "messbox" message-box to the client.

A request for !ASKMESSBOX instructs SRE-FILTER to check for VIEWMESS privileges, and if found, to construct an HTML document that offers the user a choice of available message boxes.


A Note on User Modifications to SRE-FILTER

SRE-FILTER has several mechanisms by which user extensions can be incorporated. These are:
  1. Creating your own "pre-filter" (by modifying PREFILTR.80).
  2. Creating your own routines for handling Server Side Processing Requests, or using CGI-BIN programs.
  3. Creating your own "post-filter" (by modifying POSTFILT.80)
  4. Adding INTEPRET code blocks.
  5. Creative use of the SELECT keyphrase can accomplish quite a bit of customization of HTML documents.
  6. A library of useful routines is available (via macrospace) to ambitious REXX programmers (see SREFPRC1.DOC for details).