This is the README of the test directory in the GPC source
distribution. It is also installed as BUGS in GPC binary
distributions.

If you read it in the latter form, you usually won't have the test
suite installed (you can download the source distribution to get
it), so skip the first section (`Running the tests'). The second
section (`Contributing tests') describes how to report bugs in GPC
so we can fix them as fast as possible.

Running the tests:
==================

The files in the test directory and subdirectories are for testing
GPC only and should not be of any other use.

Note: A few of the tests do not make sense on all systems. They are
equipped with checks and will be skipped if they find the system not
suitable. Skipped tests do *not* indicate a GPC bug, unless you have
a reason to be sure that the particular test should make sense on
your system.

- To run the whole test suite, type `make' in the test directory (or
  `make check' in the `p' object directory after building GPC).

  The output will show all errors encountered (hopefully none) and
  tests skipped, and finally display a summary giving the number of
  successful, failed and skipped tests. Any failed test indicates a
  bug in GPC which should be reported to the GPC mailing list,
  <gpc@gnu.de>. Don't forget to mention the operating system you ran
  the test on and any other relevant information about what you did.

- You can also type `make check-gpc-long' to get a long output which
  is a sequence of file names followed by `OK' for successful tests,
  `SKIPPED' for skipped tests, and anything else for failed tests.

- To run only some of the tests, you can type something like
  `make MASK="foo[1-7]*.pas"' or
  `make MASK="foo42.pas" check-gpc-long'.

- To clean up after running the tests, type `make clean'.

Contributing tests:
===================

If you want to contribute a test program, please note the following
formalities which are easy to fulfill and make it much easier for us
to reproduce and finally fix the problem:

- Include with your test program a description of the problem, with
  the full text of any error messages encountered, or a description
  of how some output varies from the expected output. Send it to the
  GPC mailing list, <gpc@gnu.de>. Always specify the operating
  system type and version (try `uname -a' if unsure) and the version
  of GPC as printed by `gpc -v'.

The rest of this section applies if the problem is with the compiler
itself, not an installation problem or something like this.

- Please provide a test program to reproduce the problem. The test
  program should be as short as possible, but BY ALL MEANS, send a
  COMPLETE program and MAKE SURE that it still reproduces the
  problem before you send it. Too often, users have sent code which
  contained obvious syntax errors far before the actual problem, or
  just code fragments we could only make wild guesses about. This is
  unproductive for us and doesn't help you get your problem solved.

- The file containing the main program must contain the keyword
  `program' (case-insensitively) to be recognized by the testing
  script. Other files, which can contain units or modules or data
  needed by the program, must not contain the word `program'.

- Since the test suite must run under very ... nah ... strange
  operating systems, too, file names must be distinguished in their
  first eight characters (case-insensitively) and should not contain
  anything but letters, numbers, hyphens, underscores and a single
  dot. The extension must be `.pas' to be recognized by the testing
  script. Furthermore, any ancillary files (units, modules,
  includes, data files) should not be longer than "8+3" characters.
  It is often a good idea to use your name, nickname or initials
  followed by a number as the file name.

- The test program, when run, should produce a line of output
  consisting of the string `OK' (followed by a newline), if
  everything went as expected, and something else (e.g. `failed',
  possibly followed by the reason of failure) if something went
  wrong.

- However, if the program is intended to check whether GPC catches
  an (intentional) error in the program, place the string `WRONG'
  somewhere in the test program, preferably in a comment in the line
  that contains the intentional error, and make the program output
  `failed' or something like this if it does compile erroneously.

The following special features of the testing script may be helpful
for creating tests:

- If the expected output is something else than `OK', place it in a
  file `<basename>.out' (where `<basename>' is the name of the test
  program without the `.pas' extension).

- If the test program expects some input, place it in a file
  `<basename>.in'. It will automatically be redirected to the
  program's standard input.

- If the test needs some special flags to be given to the GPC
  command line, place them in a comment preceeded by `FLAG', e.g.:

  (* FLAG --extended-pascal -Werror *)

- The source file name of the test program will be passed as the
  first command-line argument to the test program run.

- If a test needs to be run in a special way, you can accompany
  the program with a script `<basename>.run' that will do the actual
  test after the test program was compiled. In order to be portable,
  this script should run `/bin/sh' and only use standard tools. The
  source file name of the test program will be passed as the first
  command-line argument to the run script in this case. The compiled
  file is always called `a.out'.

- If a test needs to be compiled in a special way (e.g., to decide
  whether to skip the test), place the commands in a script
  (preferably called `<basename>.cmp'), and put the file name of the
  script (without directory) in a comment preceeded by
  `COMPILE-CMD:' into the source of the test program. The compile
  script will be run *instead* of any other action for this test
  program, so it gives you maximum flexibility to do whatever you
  need to do. In order to be portable, this script should run
  `/bin/sh' and only use standard tools. The first command-line
  argument to the compile script will be the compiler to use,
  including all options. The second argument will be the source file
  name of the test program. For some typical tests, there are
  standard compile scripts, e.g. i386.cmp which will skip the test
  unless run on an i386 compatible processor. This is necessary,
  e.g., when testing assembler code. Of course, it's still better to
  avoid having to use such scripts when possible.
