iotests/297: Rewrite in Python and extend reach
Instead of checking iotests.py only, check all Python files in the
qemu-iotests/ directory. Of course, most of them do not pass, so there
is an extensive skip list for now. (The only files that do pass are
209, 254, 283, and iotests.py.)
(Alternatively, we could have the opposite, i.e. an explicit list of
files that we do want to check, but I think it is better to check files
by default.)
Unless started in debug mode (./check -d), the output has no information
on which files are tested, so we will not have a problem e.g. with
backports, where some files may be missing when compared to upstream.
Besides the technical rewrite, some more things are changed:
- For the pylint invocation, PYTHONPATH is adjusted. This mirrors
setting MYPYPATH for mypy.
- Also, MYPYPATH is now derived from PYTHONPATH, so that we include
paths set by the environment. Maybe at some point we want to let the
check script add '../../python/' to PYTHONPATH so that iotests.py does
not need to do that.
- Passing --notes=FIXME,XXX to pylint suppresses warnings for TODO
comments. TODO is fine, we do not need 297 to complain about such
comments.
- The "Success" line from mypy's output is suppressed, because (A) it
does not add useful information, and (B) it would leak information
about the files having been tested to the reference output, which we
decidedly do not want.
Suggested-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Signed-off-by: Max Reitz <mreitz@redhat.com>
Message-Id: <20210118105720.14824-3-mreitz@redhat.com>
Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
2021-01-18 13:57:12 +03:00
|
|
|
#!/usr/bin/env python3
|
2021-01-16 16:44:19 +03:00
|
|
|
# group: meta
|
2020-05-11 19:35:29 +03:00
|
|
|
#
|
|
|
|
# Copyright (C) 2020 Red Hat, Inc.
|
|
|
|
#
|
|
|
|
# This program is free software; you can redistribute it and/or modify
|
|
|
|
# it under the terms of the GNU General Public License as published by
|
|
|
|
# the Free Software Foundation; either version 2 of the License, or
|
|
|
|
# (at your option) any later version.
|
|
|
|
#
|
|
|
|
# This program is distributed in the hope that it will be useful,
|
|
|
|
# but WITHOUT ANY WARRANTY; without even the implied warranty of
|
|
|
|
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
|
|
|
|
# GNU General Public License for more details.
|
|
|
|
#
|
|
|
|
# You should have received a copy of the GNU General Public License
|
|
|
|
# along with this program. If not, see <http://www.gnu.org/licenses/>.
|
|
|
|
|
iotests/297: Rewrite in Python and extend reach
Instead of checking iotests.py only, check all Python files in the
qemu-iotests/ directory. Of course, most of them do not pass, so there
is an extensive skip list for now. (The only files that do pass are
209, 254, 283, and iotests.py.)
(Alternatively, we could have the opposite, i.e. an explicit list of
files that we do want to check, but I think it is better to check files
by default.)
Unless started in debug mode (./check -d), the output has no information
on which files are tested, so we will not have a problem e.g. with
backports, where some files may be missing when compared to upstream.
Besides the technical rewrite, some more things are changed:
- For the pylint invocation, PYTHONPATH is adjusted. This mirrors
setting MYPYPATH for mypy.
- Also, MYPYPATH is now derived from PYTHONPATH, so that we include
paths set by the environment. Maybe at some point we want to let the
check script add '../../python/' to PYTHONPATH so that iotests.py does
not need to do that.
- Passing --notes=FIXME,XXX to pylint suppresses warnings for TODO
comments. TODO is fine, we do not need 297 to complain about such
comments.
- The "Success" line from mypy's output is suppressed, because (A) it
does not add useful information, and (B) it would leak information
about the files having been tested to the reference output, which we
decidedly do not want.
Suggested-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Signed-off-by: Max Reitz <mreitz@redhat.com>
Message-Id: <20210118105720.14824-3-mreitz@redhat.com>
Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
2021-01-18 13:57:12 +03:00
|
|
|
import os
|
|
|
|
import subprocess
|
|
|
|
import sys
|
2021-10-19 17:49:14 +03:00
|
|
|
from typing import List
|
2020-05-11 19:35:29 +03:00
|
|
|
|
iotests/297: Rewrite in Python and extend reach
Instead of checking iotests.py only, check all Python files in the
qemu-iotests/ directory. Of course, most of them do not pass, so there
is an extensive skip list for now. (The only files that do pass are
209, 254, 283, and iotests.py.)
(Alternatively, we could have the opposite, i.e. an explicit list of
files that we do want to check, but I think it is better to check files
by default.)
Unless started in debug mode (./check -d), the output has no information
on which files are tested, so we will not have a problem e.g. with
backports, where some files may be missing when compared to upstream.
Besides the technical rewrite, some more things are changed:
- For the pylint invocation, PYTHONPATH is adjusted. This mirrors
setting MYPYPATH for mypy.
- Also, MYPYPATH is now derived from PYTHONPATH, so that we include
paths set by the environment. Maybe at some point we want to let the
check script add '../../python/' to PYTHONPATH so that iotests.py does
not need to do that.
- Passing --notes=FIXME,XXX to pylint suppresses warnings for TODO
comments. TODO is fine, we do not need 297 to complain about such
comments.
- The "Success" line from mypy's output is suppressed, because (A) it
does not add useful information, and (B) it would leak information
about the files having been tested to the reference output, which we
decidedly do not want.
Suggested-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Signed-off-by: Max Reitz <mreitz@redhat.com>
Message-Id: <20210118105720.14824-3-mreitz@redhat.com>
Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
2021-01-18 13:57:12 +03:00
|
|
|
import iotests
|
2021-10-19 17:49:14 +03:00
|
|
|
import linters
|
2020-05-11 19:35:29 +03:00
|
|
|
|
|
|
|
|
2021-10-19 17:49:14 +03:00
|
|
|
# Looking for something?
|
|
|
|
#
|
|
|
|
# List of files to exclude from linting: linters.py
|
|
|
|
# mypy configuration: mypy.ini
|
|
|
|
# pylint configuration: pylintrc
|
2021-10-19 17:49:10 +03:00
|
|
|
|
iotests/297: Rewrite in Python and extend reach
Instead of checking iotests.py only, check all Python files in the
qemu-iotests/ directory. Of course, most of them do not pass, so there
is an extensive skip list for now. (The only files that do pass are
209, 254, 283, and iotests.py.)
(Alternatively, we could have the opposite, i.e. an explicit list of
files that we do want to check, but I think it is better to check files
by default.)
Unless started in debug mode (./check -d), the output has no information
on which files are tested, so we will not have a problem e.g. with
backports, where some files may be missing when compared to upstream.
Besides the technical rewrite, some more things are changed:
- For the pylint invocation, PYTHONPATH is adjusted. This mirrors
setting MYPYPATH for mypy.
- Also, MYPYPATH is now derived from PYTHONPATH, so that we include
paths set by the environment. Maybe at some point we want to let the
check script add '../../python/' to PYTHONPATH so that iotests.py does
not need to do that.
- Passing --notes=FIXME,XXX to pylint suppresses warnings for TODO
comments. TODO is fine, we do not need 297 to complain about such
comments.
- The "Success" line from mypy's output is suppressed, because (A) it
does not add useful information, and (B) it would leak information
about the files having been tested to the reference output, which we
decidedly do not want.
Suggested-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Signed-off-by: Max Reitz <mreitz@redhat.com>
Message-Id: <20210118105720.14824-3-mreitz@redhat.com>
Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
2021-01-18 13:57:12 +03:00
|
|
|
|
2021-10-19 17:49:13 +03:00
|
|
|
def check_linter(linter: str) -> bool:
|
|
|
|
try:
|
2021-10-19 17:49:14 +03:00
|
|
|
linters.run_linter(linter, ['--version'], suppress_output=True)
|
2021-10-19 17:49:13 +03:00
|
|
|
except subprocess.CalledProcessError:
|
|
|
|
iotests.case_notrun(f"'{linter}' not found")
|
|
|
|
return False
|
|
|
|
return True
|
2021-10-19 17:49:09 +03:00
|
|
|
|
|
|
|
|
2021-10-19 17:49:13 +03:00
|
|
|
def test_pylint(files: List[str]) -> None:
|
2021-10-19 17:49:09 +03:00
|
|
|
print('=== pylint ===')
|
|
|
|
sys.stdout.flush()
|
|
|
|
|
2021-10-19 17:49:13 +03:00
|
|
|
if not check_linter('pylint'):
|
|
|
|
return
|
|
|
|
|
2021-10-19 17:49:14 +03:00
|
|
|
linters.run_linter('pylint', files)
|
2021-10-19 17:49:13 +03:00
|
|
|
|
|
|
|
|
|
|
|
def test_mypy(files: List[str]) -> None:
|
2021-10-19 17:49:09 +03:00
|
|
|
print('=== mypy ===')
|
|
|
|
sys.stdout.flush()
|
2021-10-19 17:49:13 +03:00
|
|
|
|
|
|
|
if not check_linter('mypy'):
|
|
|
|
return
|
|
|
|
|
|
|
|
env = os.environ.copy()
|
|
|
|
env['MYPYPATH'] = env['PYTHONPATH']
|
|
|
|
|
2021-10-19 17:49:14 +03:00
|
|
|
linters.run_linter('mypy', files, env=env, suppress_output=True)
|
2021-10-19 17:49:13 +03:00
|
|
|
|
|
|
|
|
|
|
|
def main() -> None:
|
2021-10-19 17:49:14 +03:00
|
|
|
files = linters.get_test_files()
|
2021-10-19 17:49:13 +03:00
|
|
|
|
|
|
|
iotests.logger.debug('Files to be checked:')
|
|
|
|
iotests.logger.debug(', '.join(sorted(files)))
|
|
|
|
|
|
|
|
for test in (test_pylint, test_mypy):
|
|
|
|
try:
|
|
|
|
test(files)
|
|
|
|
except subprocess.CalledProcessError as exc:
|
|
|
|
# Linter failure will be caught by diffing the IO.
|
|
|
|
if exc.output:
|
|
|
|
print(exc.output)
|
2021-10-19 17:49:07 +03:00
|
|
|
|
|
|
|
|
|
|
|
iotests.script_main(main)
|