| [unpublished] INTERNET-DRAFT | Ken A L Coar |
| draft-coar-cgi-v12-00.{html,txt} | The Apache Group |
| D.R.T. Robinson | |
| ESI | |
| 29 May, 1998 |
The revision history of this draft is being maintained using Web-based GUI notation, such as struck-through characters and colour-coded sections. The following legend describes how to determine the origin of a particular revision according to the colour of the text:
The Common Gateway Interface (CGI) is a simple interface for running external programs, software or gateways under an information server in a platform-independent manner. Currently, the supported information servers are HTTP servers.
The interface has been in use by the World-Wide Web since 1993. This
specification defines the
"current practice" parameters of the
'CGI/1.1' interface developed and documented at the U.S. National
Centre for Supercomputing Applications [NCSA-CGI], and the
enhancements to that
interface known as 'CGI/1.2', which is an
extension of the 'CGI/1.1' interface developed and documented at the
U.S. National Centre for Supercomputing Applications
[NCSA-CGI].
This document also defines the use of the CGI/1.2 interface
on the UnixNIX(R) and AmigaDOS(tm) systems.
1 Introduction..............................................TBD 1.1 Purpose................................................TBD 1.1.1 Differences from CGI/1.1.............................TBD 1.2 Requirements...........................................TBD 1.3 Specifications.........................................TBD 1.4 Terminology............................................TBD 2 Notational Conventions and Generic Grammar................TBD 2.1 Augmented BNF..........................................TBD 2.2 Basic Rules............................................TBD 3 Protocol Parameters.......................................TBD 3.1 URL Encoding...........................................TBD 3.2 The Script URI.........................................TBD 4EnvironmentRequest Metadata (Meta-Variables)..............TBD 5 Invoking the Script.......................................TBD 6 The CGI Script Command Line...............................TBD 7 Data Input to the CGI Script..............................TBD 8 Data Output from the CGI Script...........................TBD 8.1 Non-Parsed Header Output...............................TBD 8.2 Parsed Header Output...................................TBD 9 Requirements for Servers..................................TBD 10 Recommendations for Scripts..............................TBD 11 System Specifications....................................TBD 11.1 AmigaDOS..............................................TBD 11.2 UnixNIX...............................................TBD 12 Security Considerations..................................TBD 12.1 Safe Methods..........................................TBD 12.2 HTTP Header Fields Containing Sensitive Information...TBD 12.3 Script Interference with the Server...................TBD 13 Acknowledgments..........................................TBD 14 References...............................................TBD 15 Authors' Addresses.......................................TBD
Together the HTTP [3],[8] server and the CGI script are responsible for servicing a client request by sending back responses. The client request comprises a Universal Resource Identifier (URI) [1], a request method and various ancillary information about the request provided by the transport mechanism.
The CGI defines the abstract parameters, known as
environment meta-variables,
which describe the client's request. Together with a
concrete programmer interface this specifies a platform-independent
interface between the script and the HTTP server.
The following is a summary of the differences proposed between CGI/1.1 and CGI/1.2:
In the absence of a complete formal specification of CGI/1.1 "current practice", this document may be used as a reference for script developers writing CGI/1.1-compliant applications if none of the CGI/1.2-specific features are utilised.
This specification uses the same words as RFC 1123 [5] to define the significance of each particular requirement. These are:
An implementation is not compliant if it fails to satisfy one or more of the 'must' requirements for the protocols it implements. An implementation that satisfies all of the 'must' and all of the 'should' requirements for its features is said to be 'unconditionally compliant'; one that satisfies all of the 'must' requirements but not all of the 'should' requirements for its features is said to be 'conditionally compliant'.
Not all of the functions and features of the CGI are defined in the main part of this specification. The following phrases are used to describe the features which are not specified:
This specification uses many terms defined in the HTTP/1.1 specification [8]; however, the following terms are used here in a sense which may not accord with their definitions in that document, or with their common meaning.
All of the mechanisms specified in this document are described in both prose and an augmented Backus-Naur Form (BNF) similar to that used by RFC 822 [6]. This augmented BNF contains the following constructs:
The following rules are used throughout this specification to describe basic parsing constructs.
alpha = lowalpha | hialpha
lowalpha = "a" | "b" | "c" | "d" | "e" | "f" | "g" | "h"
| "i" | "j" | "k" | "l" | "m" | "n" | "o" | "p"
| "q" | "r" | "s" | "t" | "u" | "v" | "w" | "x"
| "y" | "z"
hialpha = "A" | "B" | "C" | "D" | "E" | "F" | "G" | "H"
| "I" | "J" | "K" | "L" | "M" | "N" | "O" | "P"
| "Q" | "R" | "S" | "T" | "U" | "V" | "W" | "X"
| "Y" | "Z"
digit = "0" | "1" | "2" | "3" | "4" | "5" | "6" | "7"
| "8" | "9"
OCTET = <any 8-bit byte>
CHAR = <any character>
CTL = <any control character>
SP = <space character>
HT = <horizontal tab character>
NL = <newline>
LWSP = SP | HT | NL
tspecial = "(" | ")" | "@" | "," | ";" | ":" | "\" | <">
| "/" | "[" | "]" | "?" | "<" | ">" | "{" | "}"
| SP | HT
token = 1*<any CHAR except CTLs or tspecials>
quoted-string = ( <"> *qdtext <"> ) | ( "<" *qatext ">")
qdtext = <any CHAR except <"> and CTLs but including LWSP>
qatext = <any CHAR except "<", ">" and CTLs but
including LWSP>
Note that newline (NL) need not be a single character, but can be a character sequence.
Some variables and constructs used here are described as being 'URL-encoded'. This encoding is described in section 2.2 of RFC 1738 [4]. In a URL encoded string an escape sequence consists of a percent character ("%") followed by two hexadecimal digits, where the two hexadecimal digits form an octet. An escape sequence represents the graphic character which has the octet as its code within the US-ASCII [12] coded character set, if it exists. If no such graphic character exists, then the escape sequence represents the octet value itself.
An alternate "shortcut" encoding for representing the space character exists and is in common use. Scripts should be prepared to recognise both '+' and '%20' as an encoded space in a URL.
Note that some unsafe characters may have different semantics if they are encoded. The definition of which characters are unsafe depends on the context. See section 2.2 of RFC 1738 [4] for authoritative treatment of this issue.
A 'Script URI' can be defined; this describes the resource identified
by the environment meta-variables.
Often, this URI will be the same as
the URI requested by the client (the 'Client URI'); however, it need
not be. Instead, it could be a URI invented by the server, and so it
can only be used in the context of the server and its CGI interface.
The script URI has the syntax of generic-RL as defined in section 2.1 of RFC 1808 [7], with the exception that object parameters and fragment identifiers are not permitted:
The various components of the script URI are defined by some of the
environment meta-variables (see below);
script-uri = protocol "://" SERVER_NAME ":" SERVER_PORT enc-script
enc-path-info "?" QUERY_STRING
where 'protocol' is found from SERVER_PROTOCOL, 'enc-script' is a URL-encoded version of SCRIPT_NAME and 'enc-path-info' is a URL-encoded version of PATH_INFO.
Environment variables are usedEach CGI
implementation must define a mechanism
to pass data about the request from
the server to the script.
TheyThe meta-variables containing these
data
are accessed by the script in a system
defined manner. In all cases, a missing
environment meta-variable is
equivalent to a zero-length (NULL) value, and
vice versa.
The
representation of the characters in the
environment meta-variables is
system defined.
Case is not significant in the
meta-variable
names, in that there cannot be two
different variables
whose names differ in case only. Here they are
shown using a canonical representation of capitals plus underscore
("_"). The actual representation of the names is system defined; for
a particular system the representation may be defined differently
tothan
this.
The variables are:
AUTH_TYPE CONTENT_LENGTH CONTENT_TYPE GATEWAY_INTERFACE HTTP_* PATH_INFO PATH_TRANSLATED QUERY_STRING REMOTE_ADDR REMOTE_HOST REMOTE_IDENT REMOTE_USER REQUEST_METHOD SCRIPT_NAME SERVER_NAME SERVER_PORT SERVER_PROTOCOL SERVER_SOFTWARE
If the script URI would require access authentication for external access, then this variable is found from the 'auth-scheme' token in the request, otherwise NULL.
AUTH_TYPE = "" | auth-scheme
auth-scheme = "Basic" | token
HTTP access authentication schemes are described in section 11 of the HTTP/1.1 specification [8]. The auth-scheme is not case-sensitive.
CONTENT_LENGTH = "" | 1*digit
CONTENT_TYPE = "" | media-type
media-type = type "/" subtype *( ";" parameter)
type = token
subtype = token
parameter = attribute "=" value
attribute = token
value = token | quoted-string
The type, subtype and parameter attribute names are not case-sensitive. Parameter values may be case sensitive. Media types and their use in HTTP are described section 3.7 of the HTTP/1.1 specification [8]. Example:
application/x-www-form-urlencoded
There is no default value for this variable. If and only if it is
unset, then the script may attempt to determine the media type from
the data received. If the type remains unknown, then
the script may choose to either assume a
content-type of
application/octet-stream
or reject the request with either a
406 ("Not Acceptable") or 415 ("Unsupported Media Type")
errorshould be assumed.
GATEWAY_INTERFACE = "CGI" "/" 1*digit "." 1*digit
Note that the major and minor numbers are treated as separate integers and hence each may be incremented higher than a single digit. Thus CGI/2.4 is a lower version than CGI/2.13 which in turn is lower than CGI/12.3. Leading zeros must be ignored by scripts and should never be generated by servers.
This document defines the 1.2 version of the CGI interface.
Environment Meta-variables with
names beginning with "HTTP_" contain
header data read from the client, if the protocol used was HTTP. The
HTTP header field name is converted to upper case, has all occurrences of
"-" replaced with "_" and has "HTTP_" prepended to give the
environment meta-variable name.
The header data may be presented as sent
by the client, or may be rewritten in ways which do not change its
semantics. If multiple header fields with the same field-name are received
then they must be rewritten as a single header field having the same
semantics. Similarly, a header field that is received on more than one line
must be merged onto a single line. The server must, if necessary,
change the representation of the data (for example, the character
set) to be appropriate for a CGI
environment meta-variable.
The server is not required to create
environment meta-variables for all
the header fields that it receives. In particular, it may remove any
header fields carrying authentication information, such as "Authorization";
and
it may remove header fields whose value is available to the script
via other variables, such as "Content-Length" and "Content-Type".
PATH_INFO = "" | ( "/" path )
path = segment *( "/" segment )
segment = *pchar
pchar = <any CHAR except "/">
The PATH_INFO string is the trailing part of the <path> component of the script URI that follows the SCRIPT_NAME part of the path.
protocol "://" SERVER_NAME ":" SERVER_PORT enc-path-info
where 'enc-path-info' is a URL-encoded version of PATH_INFO. If
PATH_INFO is NULL then PATH_TRANSLATED is set to NULL.
PATH_TRANSLATED = *CHAR
PATH_TRANSLATED need not be supported by the server. The server may choose to set PATH_TRANSLATED to NULL for reasons of security, or because the path would not be interpretable by a CGI script; such as the object it represented was internal to the server and not visible in the file-system; or for any other reason.
The algorithm the server uses to derive PATH_TRANSLATED is obviously implementation defined; CGI scripts which use this variable may suffer limited portability.
QUERY_STRING = query-string
query-string = *qchar
qchar = unreserved | escape | reserved
unreserved = alpha | digit | safe | extra
reserved = ";" | "/" | "?" | ":" | "@" | "&" | "="
safe = "$" | "-" | "_" | "." | "+"
extra = "!" | "*" | "'" | "(" | ")" | ","
escape = "%" hex hex
hex = digit | "A" | "B" | "C" | "D" | "E" | "F" | "a"
| "b" | "c" | "d" | "e" | "f"
The URL syntax for a search string is described in RFC 1738 [4].
REMOTE_ADDR = hostnumber
hostnumber = digits "." digits "." digits "." digits
digits = 1*digit
REMOTE_HOST = "" | hostname
hostname = *( domainlabel ".") toplabel
domainlabel = alphadigit [ *alphahypdigit alphadigit ]
toplabel = alpha [ *alphahypdigit alphadigit ]
alphahypdigit = alphadigit | "-"
alphadigit = alpha | digit
REMOTE_IDENT = *CHAR
The data returned
isare
not appropriate for use as authentication
information.
If AUTH_TYPE is "Basic", then the user-ID sent by the client. If AUTH_TYPE is NULL, then NULL, otherwise undefined.
REMOTE_USER = "" | userid | *OCTET
userid = token
The method with which the request was made, as described in section 5.1.1 of the HTTP/1.0 specification [3] and section 5.1.1 of the HTTP/1.1 specification [8].
REQUEST_METHOD = http-method
http-method = "GET" | "HEAD" | "POST" | "PUT" | "DELETE"
| extension-method
extension-method = token
The method is case sensitive. Note that of the new methods defined by the HTTP/1.1 specification [8], OPTIONS and TRACE are not appropriate for the CGI/1.2 environment.
SCRIPT_NAME = "" | ( "/" [ path ] )
The leading "/" is not part of the path. It is optional if the path is NULL.
The SCRIPT_NAME string is some leading part of the <path> component of the script URI derived in some implementation defined manner.
SERVER_NAME = hostname | hostnumber
SERVER_PORT = 1*digit
SERVER_PROTOCOL = HTTP-Version | extension-version
HTTP-Version = "HTTP" "/" 1*digit "." 1*digit
extension-version = protocol "/" 1*digit "." 1*digit
protocol = 1*( alpha | digit | "+" | "-" | "." )
'protocol' is a version of the <scheme> part of the script URI, and is not case sensitive. By convention, 'protocol' is in upper case.
SERVER_SOFTWARE = *CHAR
This script is invoked in a system defined manner. Unless specified otherwise, this will be by treating the file containing the script as an executable program, and running it as a child process of the server.
Some systems support a method for supplying an array of strings to the CGI script. This is only used in the case of an 'indexed' query. This is identified by a "GET" or "HEAD" HTTP request with a URL search string not containing any unencoded "=" characters. For such a request, the server should parse the search string into words, using the rules:
search-string = search-word *( "+" search-word ) search-word = 1*schar schar = xunreserved | escape | xreserved xunreserved = alpha | digit | xsafe | extra xsafe = "$" | "-" | "_" | "." xreserved = ";" | "/" | "?" | ":" | "@" | "&"
After parsing, each word is URL-decoded, optionally encoded in a system defined manner and then the argument list is set to the list of words.
If the server cannot create any part of the argument list, then the server should generate no command line information. For example, the number of arguments may be greater than operating system or server limitations, or one of the words may not be representable as an argument.
As there may be a data entity attached to the request, there must be
a system defined method for the script to read
thisthese
data. Unless
defined otherwise, this will be via the 'standard input' file
descriptor.
There will be at least CONTENT_LENGTH bytes available for the script
to read. The script is not obliged to read the data, but it must not
attempt to read more than CONTENT_LENGTH bytes, even if more data
isare
available.
For non-parsed header (NPH) scripts (see below), the server should attempt to ensure that the script input comes directly from the client, with minimal buffering. For all scripts the data will be as supplied by the client.
There must be a system defined method for the script to send data back to the server or client; a script will always return some data. Unless defined otherwise, this will be via the 'standard output' file descriptor.
There are two forms of output that the script can give; non-parsed header (NPH) output, and parsed header output. A server is only required to support the latter; distinguishing between the two types of output (or scripts) is implementation defined.
The script must return a complete HTTP response message, as described
in Section 6 of the HTTP specifications [3],[8]. The script
shouldmust
use the SERVER_PROTOCOL variable to determine the appropriate format
for a response.
Note that this allows an HTTP/0.9 response to an
HTTP/1.0 request, for example.
The server should attempt to ensure that the script output is sent directly to the client, with minimal internal and no transport-visible buffering.
The script returns a CGI response message
as follows.:
CGI-Response = *( CGI-Header | HTTP-Header ) NL [ Entity-Body ]
CGI-Header = Content-type
| Location
| Status
| Script-Control
| extension-header
The response comprises a header and a body, separated by a blank line. The header fields are either CGI header fields to be interpreted by the server, or HTTP headers to be included in the response returned to the client if the request method is HTTP. At least one CGI-Header must be supplied, but no CGI header field can be repeated with the same field-name. If a body is supplied, then a Content-type header field is required, otherwise the script must send a Location or Status header field. If a Location header field is returned, then no HTTP-Headers may be supplied.
The CGI header fields have the generic syntax:
generic-header = field-name ":" [ field-value ] NL field-name = 1*<any CHAR, excluding CTLs, SP and ":"> field-value = *( field-content | LWSP ) field-content = *( token | tspecial | quoted-string )
The field-name is not case sensitive; a NULL field value is equivalent to the header field not being sent.
Content-Type = "Content-Type" ":" media-type NL
This is actually an HTTP-Header rather than a CGI-header field, but it is listed here because of its importance in the CGI dialogue as a member of the "one of these is required" set of header fields.
Location = "Location" ":"
( fragment-URI | rel-URL-abs-path ) NL
fragment-URI = URI [ # fragmentid ]
URI = scheme ":" *qchar
fragmentid = *qchar
rel-URL-abs-path = "/" [ hpath ] [ "?" query-string ]
hpath = fpsegment *( "/" psegment )
fpsegment = 1*hchar
psegment = *hchar
hchar = alpha | digit | safe | extra
| ":" | "@" | "& | "="
The
lLocation
value is either an absolute URI with optional fragment,
as defined in RFC 1630 [1], or an absolute path
and optional
query-string. If an absolute URI is returned by the script, then the
server will generate a '302 redirect' HTTP response message, and if no
entity body is supplied by the script, then the server will produce
one. If the Location value is a path, then the server will generate
the response that it would have produced in response to a request
containing the URL
protocol "://" SERVER_NAME ":" SERVER_PORT rel-URL-abs-path
The location header field may only be sent if the REQUEST_METHOD is HEAD or GET.
Status = "Status" ":" digit digit digit SP reason-phrase NL
reason-phrase = *<CHAR, excluding CTLs, NL>
The valid status codes are listed in section 6.1.1 of the HTTP/1.0 specifications [3]. If the SERVER_PROTOCOL is "HTTP/1.1", then the status codes defined in the HTTP/1.1 specification [8] may be used. If the script does not return a Status header field, then "200 OK" should be assumed.
If a script is being used to handle a particular error or condition encountered by the server, such as a 404 Not Found error, the script should use the Status CGI header field to propagate the error condition back to the client. E.g., in the example mentioned it should include a "Status: 404 Not Found" in the header data returned to the server.
Script-Control = "Script-Control" ":" 1#control-directive NL
control-directive = "no-abort"
| extension-directive
extension-directive = *<CHAR, excluding CTLs, NL>
The meanings of the different script control directives are:
If the script does not return a Script-Control header field, then the server is free to manage the script as it deems appropriate (e.g., killing the CGI process if the request is aborted by the client, or if the script neglects to respond within an arbitrary time interval selected by the server).
Servers must support the standard mechanism (described below) which
allows the script author to determine what URL to use in documents
which reference the script. Specifically, what URL to use in order to
achieve particular settings of the
environment meta-variables. This
mechanism is as follows:
The value for SCRIPT_NAME is governed by the server configuration and the location of the script in the OS file-system. Given this, any access to the partial URL
SCRIPT_NAME extra-path ? query-information
where extra-path is either NULL or begins with a "/" and satisfies any other server requirements, will cause the CGI script to be executed with PATH_INFO set to the decoded extra-path, and QUERY_STRING set to query-information (not decoded).
Servers may reject with error 404 any requests that would result in an encoded "/" being decoded into PATH_INFO or SCRIPT_NAME, as this might represent a loss of information to the script.
Although the server and the CGI script need not be consistent in their handling of URL paths (client URLs and the PATH_INFO data, respectively), server authors may wish to impose consistency. So the server implementation should define its behaviour for the following cases:
Servers may generate the script URI in any way from the client URI, or from any other data (but the behaviour should be documented).
Scripts should reject unexpected methods (such as DELETE etc.) with error 405 Method Not Allowed. If the script does not intend processing the PATH_INFO data, then it should reject the request with 404 Not Found if PATH_INFO is not NULL.
If the output of a form is being processed, check that CONTENT_TYPE is "application/x-www-form-urlencoded" [2].
If parsing PATH_INFO, PATH_TRANSLATED or SCRIPT_NAME then be careful of void path segments ("//") and special path segments ("." and ".."). They should either be removed from the path before use in OS system calls, or the request should be rejected with 404 Not Found. It is very unlikely that any other use could be made of these.
As it is impossible for the script to determine the client URI that initiated this request without knowledge of the specific server in use, the script should not return text/html documents containing relative URL links without including a <BASE> tag in the document.
When returning header fields, the script should try to send the CGI header fields as soon as possible, and preferably before any HTTP header fields. This may help reduce the server's memory requirements.
The implementation of the CGI on an AmigaDOS operating system platform should use environment variables as the mechanism of providing request metadata to CGI scripts.
The implementation of the CGI on a UNIX operating system platform should use environment variables as the mechanism of providing request metadata to CGI scripts.
For Unix compatible operating systems, the following are defined:
As discussed in the security considerations of the HTTP specifications [3],[8], the convention has been established that the GET and HEAD methods should be 'safe'; they should cause no side-effects and only have the significance of resource retrieval.
Some HTTP header fields may carry sensitive information which the server
should not pass on to the script unless explicitly configured to do
so. For example, if the server protects the script using the Basic
authentication scheme, then the client will send an Authorization
header field containing a username and password. If the server, rather
than the script, validates this information then the password should
not be passed on to the script via the HTTP_AUTHORIZATION
environment meta-variable.
The most common implementation of CGI invokes the script as a child process using the same user and group as the server process. It should therefore be ensured that the script cannot interfere with the server process, its configuration, or documents.
If the script is executed by calling a function linked in to the server software (either at compile-time or run-time) then precautions should be taken to protect the core memory of the server, or to ensure that untrusted code cannot be executed.
This work is based on a draft published in 1997 by David R. Robinson in 1997, which in turn was based on the original CGI interface that arose out of discussions on the www-talk mailing list. In particular, Rob McCool, John Franks, Ari Luotonen, George Phillips and Tony Sanders deserve special recognition for their efforts in defining and implementing the early versions of this interface.
This document has also greatly benefited from the comments and
suggestions made
by
Chris Adie, Dave Kristol,
and
Mike Meyer, David Morris, and Harald Alvestrand.
Ken A L Coar
MeepZor Consulting
26B Bay Ridge Drive
Nashua, NH 03062
U.S.A.
Tel: +1 (603) 891.2243
Fax: not available
Email:
Ken.Coar@Golux.Com
David Robinson
Electronic Share Information Ltd
Mount Pleasant House
2 Mount Pleasant
Huntingdon Road
Cambridge CB3 0RN
UK
Tel: +44 (1223) 566926
Fax: +44 (1223) 506288
Email:
drtr@esi.co.uk