acpica/tests/aslts/HOW_TO_USE

How to execute the run-time tests
---------------------------------


1. Install the tests (see HOW_TO_INSTALL).


2. To configure the required/desired set of test cases and the modes of
   the execution runs, manually edit the settings file:

        aslts/bin/settings

   If necessary, tune the test suite to your current needs by setting the
   variables SETN and run4. These variables control the test suite options
   and are contained in this file:

        aslts/src/runtime/cntl/runmode.asl

3. Use the Do utility to run the specified set of tests in all specified
   modes of execution runs. It supports the automated logging of the results
   of test runs and allows results to be compared. See comments for the Do
   utility within the Do script file (aslts/bin/Do).

   a) Set the following environment variables:

        ASL - path to iASL compiler: (example)

            > export ASL="c:/acpica/libraries/iasl.exe"

        acpiexec - path to acpiexec utility: (example)

            > export acpiexec="c:/acpica/libraries/acpiexec.exe"

        ASLTSDIR - path to the aslts directory: (example)

            > export ASLTSDIR="c:/acpica/tests/aslts"

   b) Add the following directory to the PATH variable:

        aslts/bin

   c) If necessary, convert all scripts in the aslts/bin directory to unix
   line endings:

        > d2u aslts/bin/*

   d) Execute "Do" with one of the following commands:
      (Use 'Do 1' to run all tests)

     0 - Compile and install AML tests
     1 - Execute all configured tests in all enabled modes
     2 - Compare two multi-results of two runs of tests
     3 - Print out the names of all the available test cases
     4 - Calculate the current state of all bugs and report the summary
         tables
     5 - Prepare bdemo summary files of one multi-results directory for all
         modes
     6 - Concatenate bdemo summary files of two multi-results


4. If desired, any individual AML test can be generated from within its
   directory by running the iASL compiler on the MAIN.asl file for that test.
   For example:

        > iASL MAIN.asl


5. If desired, any individual AML test can be executed from the aslts/tmp/aml
   directory by invoking the AcpiExec utility with the name of the AML test
   and the batch execute option. For example:

        > cd aslts/tmp/aml
        > acpiexec -eMAIN 20090320/nopt/32/arithmetic.aml


6. When all tests are executed in batch mode (Do 1), the individual test
   results are placed in the following directory structure:

    aslts/tmp/RESULTS/
        <date.time.acpica_version>/
           norm/                      // normal interpeter mode, no slack
               32/                    // 32-bit table execution
               64/                    // 64-bit table execution
           slack/                     // interpreter slack mode enabled
               32/                    // 32-bit table execution
               64/                    // 64-bit table execution
           summary                    // test execution summary


7. After completion, each AML test reports its status as one of the following:

        [PASS|FAIL|BLOCKED|SKIPPED]

   PASS    - Success, no errors encountered in the functionality of the
             product.
   FAIL    - The test encountered errors - improper functionality of the
             product.
   BLOCKED - The test was blocked (was not run). This option is used for the
             tests which are temporarily causing abort or hang of execution
             due to the errors the product.
   SKIPPED - The test was skipped (was not run). This option is used in case
             where the result of the test is undefined under the particular
             conditions.


8. How to evaluate the results of the run-time tests.

   A. Successful run.

   After the run is completed, the following summary lines are displayed
   by ASLTS:

   a) "Run time (in seconds): 0x0000000000000031"
   b) "The total number of exceptions handled: 0x0000000000000005"
   c) "TEST ACPICA: 64-bit : PASS"

   Line (a) shows the run time in seconds measured by the ASL Timer operator.

   Line (b) reports the number of exceptions which took place during the test
   execution.

   Line (c) reports the mode of the run and the summary status:
       Mode is either 32-bit or 64-bit
       Status is one of [PASS|FAIL|BLOCKED|SKIPPED]


   B. Failed run.

   a) "Run time (in seconds): 0x0000000000000031"
   b) "The total number of exceptions handled: 0x0000000000000005"
   c) "TEST ACPICA: 64-bit : FAIL : Errors # 0x0000000000000009"

      The number of errors (9 here) is reported as
      Errors # 0x0000000000000009".

   C. Example error message:

      "---------- ERROR : 0x000000001903301A, 0x0000000000033017, m503"
      "TITLE            : Miscellaneous named object creation"
      "COLLECTION       : functional"
      "TEST CASE        : name"
      "TEST             : PCG0"
      "ERROR,    file   : package.asl"
      "          index  : 000000000000001A"
      "CHECKING, file   : package.asl"
      "          method : m123"
      "          index  : 0000000000000017"
      "(r):"
                          0x0000000000000025
      "(e):"
                          0x0000000000000027
      "---------- END."

   Explanations:

      0x000000001903301A,
      0x0000000000033017 - two 32-bit words of error opcode
                           (see "The layout of error opcode" below)

      m503 - This is usually the name of the executing control method (which
             in turn is usually a conglomeration of subtests) or some brief
             diagnostic message explanation/designation/naming of the error.

      "TITLE            : The common intention of the test
      "COLLECTION       : Functional/complex/exceptions/..
      "TEST CASE        : The name of test case (bfield, arithmetic, opackageel, ...)
      "TEST             : The name of test (simplest unit reported by diagnostics
                          and supplied with the satatus line)
      "ERROR,    file   : The name of file where the error reporting
                          function (err()) was invoked
      "          index  : Index of error inside that file where err() was invoked
                          (each invocation of err() differs with its index)
      "CHECKING, file   : The name of the file where the checking was initiated
      "          method : The name of method initiated the checking
      "          index  : Index of the checking inside the file "CHECKING, file"

      (r): - usually, the following value is a received one
      (e): - usually, the following value is an expected one


   D. The errors (currently 200 max) are summarized as follows at the end of
      the test output. Example of test "Reference":

      "========= ERRORS SUMMARY (max 200):"
      "reference, ref50.asl,      0000000000000003, ref50.asl, 0000000000000000, m22c"
      "reference, datastproc.asl, 000000000000000F, ref50.asl, 0000000000000001, m22c"
      "reference, ref50.asl,      0000000000000007, ref50.asl, 0000000000000000, m234"
      "reference, ref50.asl,      0000000000000007, ref50.asl, 0000000000000000, m234"
      "reference, datastproc.asl, 0000000000000001, ref50.asl, 0000000000000013, m365"
      "reference, datastproc.asl, 0000000000000001, ref50.asl, 0000000000000015, m365"
      "reference, datastproc.asl, 0000000000000001, ref50.asl, 0000000000000017, m365"
      "========= END."

   Explanations:

      "reference, datastproc.asl, 0000000000000001, ref50.asl, 0000000000000017, m365"

      reference        - The name of the test case
      datastproc.asl   - The name of the file where the error was revealed
                         and reported by invoking err(..,index,..)
      0000000000000001 - Index of error inside that (datastproc.asl) file
      ref50.asl        - The name of file where the checking was initiated
      0000000000000017 - Index of that checking inside that (ref50.asl) file
      m365             - Diagnostic message (usually, the name of the method
                         containing the conglomeration of tests)

      For more information, see the file aslts/TESTS.


9. The layout of the error opcode (three 32-bit words)

   0xctfffeee
   0xmmzzzuuu
   0xnnnnnnnn

   c - Index of tests collection
   t - Index of test inside the collection
   f - Absolute index of the file reporting the error
   e - Index of error (inside the file)
   z - Absolute index of the file initiating the checking
   u - Index of checking
   n - Name of Method initiating the checking
   m - Miscellaneous:
       1) in case of TCLD tests there is an index of bug stored (max 600)


How to use ASL-compilation control test collection
==================================================

The tests for the ASL Compiler to check its ability to detect, report and
reject wrong ASL code are contained in this directory:

   aslts/src/compilation/collection

At present, no utility is provided to perform automated run and verification
of these tests.

The tests contain ASL code with compile errors such that no output AML files
are expected. Expected are Warning and Error messages to be reported by ASL
Compiler for the incorrect ASL code. When implemented, the utility should
parse the output of the ASL Compiler for these files to verify the presence
of the expected messages.