When tokens in macro definitions need cstr_buf inside get_tok_str,
the second might overwrite the first (happens when tokens are
multi-character non-identifiers, see testcase) in macro_is_equal,
failing to diagnose a difference. Use a real local buffer.
Also:
- regenerate all tests/pp/*.expect with gcc
- test "insert one space" feature
- test "0x1E-1" in asm mode case
- PARSE_FLAG_SPACES: ignore \f\v\r better
- tcc.h: move some things
#ifndef guards are cached, however after #elif on the
same level, the file must be re-read.
Also: preprocess asm as such even when there is no
assembler (arm).
- "utf8 in identifiers"
from 936819a1b9
- CValue: remove member str.data_allocated
- make tiny allocator private to tccpp
- allocate macro_stack objects on heap
because otherwise it could crash after error/setjmp
in preprocess_delete():end_macro()
- mov "TinyAlloc" defs to tccpp.c
- define_push: take int* str again
Also:
- allow more than one item per line
- respect "quoted items" and escaped quotes \"
(also for LIBTCCAPI tcc_setoptions)
- cleanup some copy & paste
after several "fixes" and "improvements"
b3782c3cf55fb57bead4
feature did not work at all
- Use 'once' flag, not 'ifndef_macro'
- Ignore filename letter case on _WIN32
- Increment global pp_once for each compilation
fixes 5c35ba66c5
Implementation was consistent within tcc but incompatible
with the ABI (for example library functions vprintf etc)
Also:
- tccpp.c/get_tok_str() : avoid "unknown format "%llu" warning
- x86_64_gen.c/gen_vla_alloc() : fix vstack leak
This may be used to preprocess Fabrice Bellards initial revision
in this repository to demonstrate its capability to compile and
run itself (on i386 32-bit linux or windows).
Initial revision: 27f6e16bae
Also needed:
* an empty stdio.h
* a wrapper named tc.c with
void expr(void);
void decl(int);
void next(void);
#include "tcc.c"
* an hello.c such as
int main()
{
printf("Hello World\n");
return 0;
}
All files with unix LF only lines. Then ...
* preprocess the source
$ tcc -E -P10 -I. tcc.c -o tc1.c
* compile the compiler
$ tcc -w -I. tc.c -o tc -ldl
* run it to compile and
run itself to compile and
run itself to compile and
run itself to compile and
run hello.c
$ ./tc tc1.c tc1.c tc1.c hello.c
--> Hello World!
------------------------------------------------------
* On i386 windows this may be added to the tc.c wrapper
#ifdef _WIN32
#include <windows.h>
void *dlsym(int x, const char *func)
{
if (0 == strcmp(func, "dlsym"))
return &dlsym;
return GetProcAddress(LoadLibrary("msvcrt"), func);
}
#endif
Insert a space when it is required to prevent mistokenisation of
the output, and also in a few cases where it is not strictly
required, imitating GCC's behaviour.
* correct -E output for the case ++ + ++ concatenation
do this only for expanded from macro string
and only when tcc_state->output_type == TCC_OUTPUT_PREPROCESS
- uses new `TinyAlloc`-ators for small `TokenSym`, `CString` and
`TokenString` instances
- conditional `TAL_DEBUG` for mem leaks and double frees detection
- on `TAL_DEBUG` collects allocation origin (file + line)
- conditional `TAL_INFO` for allocators stats (in release mode too)
- chain a new allocator twice current capacity on buffer exhaustion
parse_print_line_comment() and parse_print_comment() are
combined and made more simply:
* don't worry about speed with -E option
* don't handle straya in comments
Do we need to handle strays in regular
parse_line_comment() and
parse_comment() ?
A problem was in TOK_ASMDIR_text:
- sprintf(sname, ".%s", get_tok_str(tok1, NULL));
+ sprintf(sname, "%s", get_tok_str(tok1, NULL));
When tok1 is '.text', then sname is '..text'
From: Vlad Vissoultchev
Date: Mon, 11 Apr 2016 01:26:32 +0300
Subject: Fix pragma once guard when compiling multiple source files
When compiling multiple source files directly to executable cached
include files guard was incorrectly checked for TOK_once in ifndef_macro
member.
If two source files included the same header guarded by pragma once, then
the second one erroneously skipped it as `cached_includes` is not cleared
on second `tcc_compile`
When tccboot kernels compiles with
'Identifiers can start and/or', this kernel don't start.
It is hard to find what is wrong.
PS: there was no test for identifiers in *.S with '.'
There was already support for -dD option but in contrast -dM dumps only `#define` directives w/o actual preprocessor output.
The original -dD output differs from gcc output by additional comment in front of `#define`s so this quirk is left for -dM as well.
made like in pcc
(pcc.ludd.ltu.se/ftp/pub/pcc-docs/pcc-utf8-ver3.pdf)
We treat all chars with high bit set as alphabetic.
This allow code like
#include <stdio.h>
int Lefèvre=2;
int main() {
printf("Lefèvre=%d\n",Lefèvre);
return 0;
}
modified version of the old one which don't allow '.'
in #define Identifiers. This allow correctly preprocess
the following code in *.S
#define SRC(y...) \
9999: y; \
.section __ex_table, "a"; \
.long 9999b, 6001f ; \
// .previous
SRC(1: movw (%esi), %bx)
6001:
A test included.
remove non-existent or duplicate directories from include paths
if -fnormalize-inc-dirs is specified. This will help
to compile current coreutils package
- Identifiers can start and/or contain '.' in PARSE_FLAG_ASM_FILE
- Move all GAS directives under TOK_ASMDIR prefix
This patches breaks compilation of the tccboot (linux 2.4.26
kernel). A test.S which fails with this patches:
#define SRC(y...) \
9999: y; \
.section __ex_table, "a"; \
.long 9999b, 6001f<---->; \
.previous
SRC(1:<>movw (%esi), %bx<------>)
// 029-test.S:7: error: macro 'SRC' used with too many args
A CString used to be copied into a token string, which is an int array.
On a 64-bit architecture the pointers were misaligned, so ASan gave
lots of warnings. On a 64-bit architecture that required memory
accesses to be correctly aligned it would not work at all.
The CString is now included in CValue instead.
tcc_normalize_inc_dirs: normally no problem to be absolutly
gcc compatible as long as it can be done the tiny way.
This reverts to the state before recent related commits and
reimplements a (small) part of it to fix the reported problem.
Also: Revert "parsing "..." sequence"
c3975cf27c
&& p[1] == '.'
is not a reliable way to lookahead
- avoid memory allocation by using its (int) token number
- avoid additional function parameter by using Attribute
Also: fix some strange looking error messages
include dirs are prepared as in gcc
- for each duplicate path keep just the first one
- remove each include_path that exists in sysinclude_paths
include_next streamlined by introducing inc_path_index
in the BufferedFile
This version looks rigth. Comparing to the original
algorithm:
1) Loop breaking. We remember a start point after wich
we can try next path. Do not search include stack after
this.
2) But compare next file patch with the start point.
Skip if it the same. Remove "./" before comparing.
PS: a problems with compaling a coreutils-8.24.51-8802e
remain. There are errors messages like:
src/chgrp
src/chown-core.c:42: multiple definition of `make_timespec'
src/chgrp.c:42: first defined here
A problem is in the lib/config.h
#define _GL_INLINE_ extern inline // gcc
#define _GL_INLINE_ inline // tcc
A long description from the lib/config.h
* suppress extern inline with HP-UX cc, as it appears to be broken
* suppress extern inline with Sun C in standards-conformance mode
* suppress extern inline on configurations that mistakenly use
'static inline' to implement functions or macros in standard
C headers like <ctype.h>.
GCC and Clang are excluded from this list. Why not tcc?
don't give an error and simply ingnore directive
if we detect a loop of the #include_next.
With this aproach coreutils-8.24.51-8802e
compiles, but with errors:
lib/libcoreutils.a: error: 'xnmalloc' defined twice
lib/libcoreutils.a: error: 'xnrealloc' defined twice
don't panic with
error: '.' expected
if there is only two '.' chars. Return tok='.' in such case.
An asm code to test:
jz do_move0 # .. then we have a normal low
# .. or else we have a high
* Documentation is now in "docs".
* Source code is now in "src".
* Misc. fixes here and there so that everything still works.
I think I got everything in this commit, but I only tested this
on Linux (Make) and Windows (CMake), so I might've messed
something up on other platforms...
functionality was broken some time ago and was removed
by the "tccpp: fix issues, add tests"
fix: LINE_MACRO_OUTPUT_FORMAT_NONE in pp_line()
means: output '\n' and not "don't output at all"
In the linux kernel sources:
#ifdef __STDC__
#define SYMBOL_NAME_LABEL(X) X##:
#else
#define SYMBOL_NAME_LABEL(X) X/**/:
#endif
tcc is a STDC compiler and must handle 'X##:' case.
* fix some macro expansion issues
* add some pp tests in tests/pp
* improved tcc -E output for better diff'ability
* remove -dD feature (quirky code, exotic feature,
didn't work well)
Based partially on ideas / researches from PipCet
Some issues remain with VA_ARGS macros (if used in a
rather tricky way).
Also, to keep it simple, the pp doesn't automtically
add any extra spaces to separate tokens which otherwise
would form wrong tokens if re-read from tcc -E output
(such as '+' '=') GCC does that, other compilers don't.
* cleanups
- #line 01 "file" / # 01 "file" processing
- #pragma comment(lib,"foo")
- tcc -E: forward some pragmas to output (pack, comment(lib))
- fix macro parameter list parsing mess from
a3fc543459a715d7143d
(some coffee might help, next time ;)
- introduce TOK_PPSTR - to have character constants as
written in the file (similar to TOK_PPNUM)
- allow '\' appear in macros
- new functions begin/end_macro to:
- fix switching macro levels during expansion
- allow unget_tok to unget more than one tok
- slight speedup by using bitflags in isidnum_table
Also:
- x86_64.c : fix decl after statements
- i386-gen,c : fix a vstack leak with VLA on windows
- configure/Makefile : build on windows (MSYS) was broken
- tcc_warning: fflush stderr to keep output order (win32)
This requires moving TOK_PLCHLDR handling, but the new logic should make
things easier even if (when?) GNU comma handling is removed.
(Somewhat confusingly, GCC no longer supports GNU commas. See
http://gcc.gnu.org/onlinedocs/cpp/Variadic-Macros.html for a description
of past and current GCC behaviour.)
Fix handling of escape characters, spaces, and line feeds in macros or
macro arguments that might yet be subject to # stringification.
Should this be an -f option? I think memory usage increases only very
slightly (in particular, while line feeds, stray \s, and spaces are
preserved, comments are not), so it's probably not worth it to make it
one.
Note that macro_subst now checks for stray \s which are still left in
the input stream after macro substitution, if desired.
This patch depends on the previous patch, so if you revert that, please
revert this patch, too.
See http://lists.nongnu.org/archive/html/tinycc-devel/2015-05/msg00002.html
This adds a PARSE_FLAG_ACCEPT_STRAYS parse flag to accept stray
backslashes in the source code, and uses it for pure preprocessing.
For absolutely correct behaviour of # stringification, we need to use
this flag when parsing macro definitions and in macro arguments, as
well; this patch does not yet do so. The test case for that is something
like
#define STRINGIFY2(x) #x
#define STRINGIFY(x) STRINGIFY2(x)
STRINGIFY(\n)
which should produce "\n", not a parse error or "\\n".
See http://lists.nongnu.org/archive/html/tinycc-devel/2015-05/msg00002.html
Perhaps a better fix would be to ensure tok is set to TOK_EOF rather
than 0 at the end of a macro stream.
This partially fixes test2 of the examples given in:
http://lists.nongnu.org/archive/html/tinycc-devel/2015-05/msg00002.html
It's still failing, but at least it's not running out of memory now.
The old code had an inverted condition, so
#define a(b)## b
would be accepted while
#define a(b,c) b ## ## c
would be rejected with the confusing error message "'##' invalid at
start of macro".
Quick fix for
http://lists.nongnu.org/archive/html/tinycc-devel/2015-04/msg00160.html.
I don't fully understand the intended semantics of when file->buf_ptr[0]
is valid, but the rest of the code doesn't have any obvious spots with
the same bug.
Feel free to revert this if I'm mistaken or we need to discuss this
change further.
