mirrors/zlib
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

Clean up code style in enough.c, update version.

Browse Source
This commit is contained in:
Mark Adler 2018-08-01 01:49:45 -07:00
parent 4c14b51587
commit 8ba2cdb6bd
1 changed files with 197 additions and 206 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

403
examples/enough.c
View File

@ -1,7 +1,7 @@
/* enough.c -- determine the maximum size of inflate's Huffman code tables over /* enough.c -- determine the maximum size of inflate's Huffman code tables over
* all possible valid and complete Huffman codes, subject to a length limit. * all possible valid and complete Huffman codes, subject to a length limit.
* Copyright (C) 2007, 2008, 2012 Mark Adler * Copyright (C) 2007, 2008, 2012, 2018 Mark Adler
* Version 1.4 18 August 2012 Mark Adler * Version 1.5 1 August 2018 Mark Adler
*/ */
/* Version history: /* Version history:
@ -17,86 +17,87 @@
1.4 18 Aug 2012 Avoid shifts more than bits in type (caused endless loop!) 1.4 18 Aug 2012 Avoid shifts more than bits in type (caused endless loop!)
Clean up comparisons of different types Clean up comparisons of different types
Clean up code indentation Clean up code indentation
1.5 1 Aug 2018 Clean up code style and formatting
*/ */
/* /*
Examine all possible Huffman codes for a given number of symbols and a Examine all possible Huffman codes for a given number of symbols and a
maximum code length in bits to determine the maximum table size for zilb's maximum code length in bits to determine the maximum table size for zlib's
inflate. Only complete Huffman codes are counted. inflate. Only complete Huffman codes are counted.
Two codes are considered distinct if the vectors of the number of codes per Two codes are considered distinct if the vectors of the number of codes per
length are not identical. So permutations of the symbol assignments result length are not identical. So permutations of the symbol assignments result
in the same code for the counting, as do permutations of the assignments of in the same code for the counting, as do permutations of the assignments of
the bit values to the codes (i.e. only canonical codes are counted). the bit values to the codes (i.e. only canonical codes are counted).
We build a code from shorter to longer lengths, determining how many symbols We build a code from shorter to longer lengths, determining how many symbols
are coded at each length. At each step, we have how many symbols remain to are coded at each length. At each step, we have how many symbols remain to
be coded, what the last code length used was, and how many bit patterns of be coded, what the last code length used was, and how many bit patterns of
that length remain unused. Then we add one to the code length and double the that length remain unused. Then we add one to the code length and double the
number of unused patterns to graduate to the next code length. We then number of unused patterns to graduate to the next code length. We then
assign all portions of the remaining symbols to that code length that assign all portions of the remaining symbols to that code length that
preserve the properties of a correct and eventually complete code. Those preserve the properties of a correct and eventually complete code. Those
properties are: we cannot use more bit patterns than are available; and when properties are: we cannot use more bit patterns than are available; and when
all the symbols are used, there are exactly zero possible bit patterns all the symbols are used, there are exactly zero possible bit patterns
remaining. remaining.
The inflate Huffman decoding algorithm uses two-level lookup tables for The inflate Huffman decoding algorithm uses two-level lookup tables for
speed. There is a single first-level table to decode codes up to root bits speed. There is a single first-level table to decode codes up to root bits
in length (root == 9 in the current inflate implementation). The table in length (root == 9 in the current inflate implementation). The table has 1
has 1 << root entries and is indexed by the next root bits of input. Codes << root entries and is indexed by the next root bits of input. Codes shorter
shorter than root bits have replicated table entries, so that the correct than root bits have replicated table entries, so that the correct entry is
entry is pointed to regardless of the bits that follow the short code. If pointed to regardless of the bits that follow the short code. If the code is
the code is longer than root bits, then the table entry points to a second- longer than root bits, then the table entry points to a second- level table.
level table. The size of that table is determined by the longest code with The size of that table is determined by the longest code with that root-bit
that root-bit prefix. If that longest code has length len, then the table prefix. If that longest code has length len, then the table has size 1 <<
has size 1 << (len - root), to index the remaining bits in that set of (len - root), to index the remaining bits in that set of codes. Each
codes. Each subsequent root-bit prefix then has its own sub-table. The subsequent root-bit prefix then has its own sub-table. The total number of
total number of table entries required by the code is calculated table entries required by the code is calculated incrementally as the number
incrementally as the number of codes at each bit length is populated. When of codes at each bit length is populated. When all of the codes are shorter
all of the codes are shorter than root bits, then root is reduced to the than root bits, then root is reduced to the longest code length, resulting
longest code length, resulting in a single, smaller, one-level table. in a single, smaller, one-level table.
The inflate algorithm also provides for small values of root (relative to The inflate algorithm also provides for small values of root (relative to
the log2 of the number of symbols), where the shortest code has more bits the log2 of the number of symbols), where the shortest code has more bits
than root. In that case, root is increased to the length of the shortest than root. In that case, root is increased to the length of the shortest
code. This program, by design, does not handle that case, so it is verified code. This program, by design, does not handle that case, so it is verified
that the number of symbols is less than 2^(root + 1). that the number of symbols is less than 2^(root + 1).
In order to speed up the examination (by about ten orders of magnitude for In order to speed up the examination (by about ten orders of magnitude for
the default arguments), the intermediate states in the build-up of a code the default arguments), the intermediate states in the build-up of a code
are remembered and previously visited branches are pruned. The memory are remembered and previously visited branches are pruned. The memory
required for this will increase rapidly with the total number of symbols and required for this will increase rapidly with the total number of symbols and
the maximum code length in bits. However this is a very small price to pay the maximum code length in bits. However this is a very small price to pay
for the vast speedup. for the vast speedup.
First, all of the possible Huffman codes are counted, and reachable First, all of the possible Huffman codes are counted, and reachable
intermediate states are noted by a non-zero count in a saved-results array. intermediate states are noted by a non-zero count in a saved-results array.
Second, the intermediate states that lead to (root + 1) bit or longer codes Second, the intermediate states that lead to (root + 1) bit or longer codes
are used to look at all sub-codes from those junctures for their inflate are used to look at all sub-codes from those junctures for their inflate
memory usage. (The amount of memory used is not affected by the number of memory usage. (The amount of memory used is not affected by the number of
codes of root bits or less in length.) Third, the visited states in the codes of root bits or less in length.) Third, the visited states in the
construction of those sub-codes and the associated calculation of the table construction of those sub-codes and the associated calculation of the table
size is recalled in order to avoid recalculating from the same juncture. size is recalled in order to avoid recalculating from the same juncture.
Beginning the code examination at (root + 1) bit codes, which is enabled by Beginning the code examination at (root + 1) bit codes, which is enabled by
identifying the reachable nodes, accounts for about six of the orders of identifying the reachable nodes, accounts for about six of the orders of
magnitude of improvement for the default arguments. About another four magnitude of improvement for the default arguments. About another four
orders of magnitude come from not revisiting previous states. Out of orders of magnitude come from not revisiting previous states. Out of
approximately 2x10^16 possible Huffman codes, only about 2x10^6 sub-codes approximately 2x10^16 possible Huffman codes, only about 2x10^6 sub-codes
need to be examined to cover all of the possible table memory usage cases need to be examined to cover all of the possible table memory usage cases
for the default arguments of 286 symbols limited to 15-bit codes. for the default arguments of 286 symbols limited to 15-bit codes.
Note that an unsigned long long type is used for counting. It is quite easy Note that an unsigned long long type is used for counting. It is quite easy
to exceed the capacity of an eight-byte integer with a large number of to exceed the capacity of an eight-byte integer with a large number of
symbols and a large maximum code length, so multiple-precision arithmetic symbols and a large maximum code length, so multiple-precision arithmetic
would need to replace the unsigned long long arithmetic in that case. This would need to replace the unsigned long long arithmetic in that case. This
program will abort if an overflow occurs. The big_t type identifies where program will abort if an overflow occurs. The big_t type identifies where
the counting takes place. the counting takes place.
An unsigned long long type is also used for calculating the number of An unsigned long long type is also used for calculating the number of
possible codes remaining at the maximum length. This limits the maximum possible codes remaining at the maximum length. This limits the maximum code
code length to the number of bits in a long long minus the number of bits length to the number of bits in a long long minus the number of bits needed
needed to represent the symbols in a flat code. The code_t type identifies to represent the symbols in a flat code. The code_t type identifies where
where the bit pattern counting takes place. the bit pattern counting takes place.
*/ */
#include <stdio.h> #include <stdio.h>
@ -106,13 +107,13 @@
#define local static #define local static
/* special data types */ // Special data types.
typedef unsigned long long big_t; /* type for code counting */ typedef unsigned long long big_t; // type for code counting
#define PRIbig "llu" /* printf format for big_t */ #define PRIbig "llu" // printf format for big_t
typedef unsigned long long code_t; /* type for bit pattern counting */ typedef unsigned long long code_t; // type for bit pattern counting
struct tab { /* type for been here check */ struct tab { // type for been here check
size_t len; /* length of bit vector in char's */ size_t len; // length of bit vector in char's
char *vec; /* allocated bit vector */ char *vec; // allocated bit vector
}; };
/* The array for saving results, num[], is indexed with this triplet: /* The array for saving results, num[], is indexed with this triplet:
@ -127,24 +128,23 @@ struct tab { /* type for been here check */
left: 2..syms - 1, but only the evens (so syms == 8 -> 2, 4, 6) left: 2..syms - 1, but only the evens (so syms == 8 -> 2, 4, 6)
len: 1..max - 1 (max == maximum code length in bits) len: 1..max - 1 (max == maximum code length in bits)
syms == 2 is not saved since that immediately leads to a single code. left syms == 2 is not saved since that immediately leads to a single code. left
must be even, since it represents the number of available bit patterns at must be even, since it represents the number of available bit patterns at
the current length, which is double the number at the previous length. the current length, which is double the number at the previous length. left
left ends at syms-1 since left == syms immediately results in a single code. ends at syms-1 since left == syms immediately results in a single code.
(left > sym is not allowed since that would result in an incomplete code.) (left > sym is not allowed since that would result in an incomplete code.)
len is less than max, since the code completes immediately when len == max. len is less than max, since the code completes immediately when len == max.
The offset into the array is calculated for the three indices with the The offset into the array is calculated for the three indices with the first
first one (syms) being outermost, and the last one (len) being innermost. one (syms) being outermost, and the last one (len) being innermost. We build
We build the array with length max-1 lists for the len index, with syms-3 the array with length max-1 lists for the len index, with syms-3 of those
of those for each symbol. There are totsym-2 of those, with each one for each symbol. There are totsym-2 of those, with each one varying in
varying in length as a function of sym. See the calculation of index in length as a function of sym. See the calculation of index in map() for the
count() for the index, and the calculation of size in main() for the size index, and the calculation of size in main() for the size of the array.
of the array.
For the deflate example of 286 symbols limited to 15-bit codes, the array For the deflate example of 286 symbols limited to 15-bit codes, the array
has 284,284 entries, taking up 2.17 MB for an 8-byte big_t. More than has 284,284 entries, taking up 2.17 MB for an 8-byte big_t. More than half
half of the space allocated for saved results is actually used -- not all of the space allocated for saved results is actually used -- not all
possible triplets are reached in the generation of valid Huffman codes. possible triplets are reached in the generation of valid Huffman codes.
*/ */
@ -152,14 +152,14 @@ struct tab { /* type for been here check */
to the num[] array as described above for the (syms, left, len) triplet. to the num[] array as described above for the (syms, left, len) triplet.
Each element in the array is further indexed by the (mem, rem) doublet, Each element in the array is further indexed by the (mem, rem) doublet,
where mem is the amount of inflate table space used so far, and rem is the where mem is the amount of inflate table space used so far, and rem is the
remaining unused entries in the current inflate sub-table. Each indexed remaining unused entries in the current inflate sub-table. Each indexed
element is simply one bit indicating whether the state has been visited or element is simply one bit indicating whether the state has been visited or
not. Since the ranges for mem and rem are not known a priori, each bit not. Since the ranges for mem and rem are not known a priori, each bit
vector is of a variable size, and grows as needed to accommodate the visited vector is of a variable size, and grows as needed to accommodate the visited
states. mem and rem are used to calculate a single index in a triangular states. mem and rem are used to calculate a single index in a triangular
array. Since the range of mem is expected in the default case to be about array. Since the range of mem is expected in the default case to be about
ten times larger than the range of rem, the array is skewed to reduce the ten times larger than the range of rem, the array is skewed to reduce the
memory usage, with eight times the range for mem than for rem. See the memory usage, with eight times the range for mem than for rem. See the
calculations for offset and bit in beenhere() for the details. calculations for offset and bit in beenhere() for the details.
For the deflate example of 286 symbols limited to 15-bit codes, the bit For the deflate example of 286 symbols limited to 15-bit codes, the bit
@ -167,23 +167,22 @@ struct tab { /* type for been here check */
array itself. array itself.
*/ */
/* Globals to avoid propagating constants or constant pointers recursively */ // Globals to avoid propagating constants or constant pointers recursively.
struct { struct {
int max; /* maximum allowed bit length for the codes */ int max; // maximum allowed bit length for the codes
int root; /* size of base code table in bits */ int root; // size of base code table in bits
int large; /* largest code table so far */ int large; // largest code table so far
size_t size; /* number of elements in num and done */ size_t size; // number of elements in num and done
int *code; /* number of symbols assigned to each bit length */ int *code; // number of symbols assigned to each bit length
big_t *num; /* saved results array for code counting */ big_t *num; // saved results array for code counting
struct tab *done; /* states already evaluated array */ struct tab *done; // states already evaluated array
} g; } g;
/* Index function for num[] and done[] */ // Index function for num[] and done[].
#define INDEX(i,j,k) (((size_t)((i-1)>>1)*((i-2)>>1)+(j>>1)-1)*(g.max-1)+k-1) #define INDEX(i,j,k) (((size_t)((i-1)>>1)*((i-2)>>1)+(j>>1)-1)*(g.max-1)+k-1)
/* Free allocated space. Uses globals code, num, and done. */ // Free allocated space. Uses globals code, num, and done.
local void cleanup(void) local void cleanup(void) {
{
size_t n; size_t n;
if (g.done != NULL) { if (g.done != NULL) {
@ -198,91 +197,89 @@ local void cleanup(void)
free(g.code); free(g.code);
} }
/* Return the number of possible Huffman codes using bit patterns of lengths // Return the number of possible Huffman codes using bit patterns of lengths
len through max inclusive, coding syms symbols, with left bit patterns of // len through max inclusive, coding syms symbols, with left bit patterns of
length len unused -- return -1 if there is an overflow in the counting. // length len unused -- return -1 if there is an overflow in the counting. Keep
Keep a record of previous results in num to prevent repeating the same // a record of previous results in num to prevent repeating the same
calculation. Uses the globals max and num. */ // calculation. Uses the globals max and num.
local big_t count(int syms, int len, int left) local big_t count(int syms, int len, int left) {
{ big_t sum; // number of possible codes from this juncture
big_t sum; /* number of possible codes from this juncture */ big_t got; // value returned from count()
big_t got; /* value returned from count() */ int least; // least number of syms to use at this juncture
int least; /* least number of syms to use at this juncture */ int most; // most number of syms to use at this juncture
int most; /* most number of syms to use at this juncture */ int use; // number of bit patterns to use in next call
int use; /* number of bit patterns to use in next call */ size_t index; // index of this case in *num
size_t index; /* index of this case in *num */
/* see if only one possible code */ // see if only one possible code
if (syms == left) if (syms == left)
return 1; return 1;
/* note and verify the expected state */ // note and verify the expected state
assert(syms > left && left > 0 && len < g.max); assert(syms > left && left > 0 && len < g.max);
/* see if we've done this one already */ // see if we've done this one already
index = INDEX(syms, left, len); index = INDEX(syms, left, len);
got = g.num[index]; got = g.num[index];
if (got) if (got)
return got; /* we have -- return the saved result */ return got; // we have -- return the saved result
/* we need to use at least this many bit patterns so that the code won't be // we need to use at least this many bit patterns so that the code won't be
incomplete at the next length (more bit patterns than symbols) */ // incomplete at the next length (more bit patterns than symbols)
least = (left << 1) - syms; least = (left << 1) - syms;
if (least < 0) if (least < 0)
least = 0; least = 0;
/* we can use at most this many bit patterns, lest there not be enough // we can use at most this many bit patterns, lest there not be enough
available for the remaining symbols at the maximum length (if there were // available for the remaining symbols at the maximum length (if there were
no limit to the code length, this would become: most = left - 1) */ // no limit to the code length, this would become: most = left - 1)
most = (((code_t)left << (g.max - len)) - syms) / most = (((code_t)left << (g.max - len)) - syms) /
(((code_t)1 << (g.max - len)) - 1); (((code_t)1 << (g.max - len)) - 1);
/* count all possible codes from this juncture and add them up */ // count all possible codes from this juncture and add them up
sum = 0; sum = 0;
for (use = least; use <= most; use++) { for (use = least; use <= most; use++) {
got = count(syms - use, len + 1, (left - use) << 1); got = count(syms - use, len + 1, (left - use) << 1);
sum += got; sum += got;
if (got == (big_t)0 - 1 || sum < got) /* overflow */ if (got == (big_t)0 - 1 || sum < got) // overflow
return (big_t)0 - 1; return (big_t)0 - 1;
} }
/* verify that all recursive calls are productive */ // verify that all recursive calls are productive
assert(sum != 0); assert(sum != 0);
/* save the result and return it */ // save the result and return it
g.num[index] = sum; g.num[index] = sum;
return sum; return sum;
} }
/* Return true if we've been here before, set to true if not. Set a bit in a // Return true if we've been here before, set to true if not. Set a bit in a
bit vector to indicate visiting this state. Each (syms,len,left) state // bit vector to indicate visiting this state. Each (syms,len,left) state has a
has a variable size bit vector indexed by (mem,rem). The bit vector is // variable size bit vector indexed by (mem,rem). The bit vector is lengthened
lengthened if needed to allow setting the (mem,rem) bit. */ // if needed to allow setting the (mem,rem) bit.
local int beenhere(int syms, int len, int left, int mem, int rem) local int beenhere(int syms, int len, int left, int mem, int rem) {
{ size_t index; // index for this state's bit vector
size_t index; /* index for this state's bit vector */ size_t offset; // offset in this state's bit vector
size_t offset; /* offset in this state's bit vector */ int bit; // mask for this state's bit
int bit; /* mask for this state's bit */ size_t length; // length of the bit vector in bytes
size_t length; /* length of the bit vector in bytes */ char *vector; // new or enlarged bit vector
char *vector; /* new or enlarged bit vector */
/* point to vector for (syms,left,len), bit in vector for (mem,rem) */ // point to vector for (syms,left,len), bit in vector for (mem,rem)
index = INDEX(syms, left, len); index = INDEX(syms, left, len);
mem -= 1 << g.root; mem -= 1 << g.root;
offset = (mem >> 3) + rem; offset = (mem >> 3) + rem;
offset = ((offset * (offset + 1)) >> 1) + rem; offset = ((offset * (offset + 1)) >> 1) + rem;
bit = 1 << (mem & 7); bit = 1 << (mem & 7);
/* see if we've been here */ // see if we've been here
length = g.done[index].len; length = g.done[index].len;
if (offset < length && (g.done[index].vec[offset] & bit) != 0) if (offset < length && (g.done[index].vec[offset] & bit) != 0)
return 1; /* done this! */ return 1; // done this!
/* we haven't been here before -- set the bit to show we have now */ // we haven't been here before -- set the bit to show we have now
/* see if we need to lengthen the vector in order to set the bit */ // see if we need to lengthen the vector in order to set the bit
if (length <= offset) { if (length <= offset) {
/* if we have one already, enlarge it, zero out the appended space */ // if we have one already, enlarge it, zero out the appended space
if (length) { if (length) {
do { do {
length <<= 1; length <<= 1;
@ -293,7 +290,7 @@ local int beenhere(int syms, int len, int left, int mem, int rem)
length - g.done[index].len); length - g.done[index].len);
} }
/* otherwise we need to make a new vector and zero it out */ // otherwise we need to make a new vector and zero it out
else { else {
length = 1 << (len - g.root); length = 1 << (len - g.root);
while (length <= offset) while (length <= offset)
@ -301,40 +298,39 @@ local int beenhere(int syms, int len, int left, int mem, int rem)
vector = calloc(length, sizeof(char)); vector = calloc(length, sizeof(char));
} }
/* in either case, bail if we can't get the memory */ // in either case, bail if we can't get the memory
if (vector == NULL) { if (vector == NULL) {
fputs("abort: unable to allocate enough memory\n", stderr); fputs("abort: unable to allocate enough memory\n", stderr);
cleanup(); cleanup();
exit(1); exit(1);
} }
/* install the new vector */ // install the new vector
g.done[index].len = length; g.done[index].len = length;
g.done[index].vec = vector; g.done[index].vec = vector;
} }
/* set the bit */ // set the bit
g.done[index].vec[offset] |= bit; g.done[index].vec[offset] |= bit;
return 0; return 0;
} }
/* Examine all possible codes from the given node (syms, len, left). Compute // Examine all possible codes from the given node (syms, len, left). Compute
the amount of memory required to build inflate's decoding tables, where the // the amount of memory required to build inflate's decoding tables, where the
number of code structures used so far is mem, and the number remaining in // number of code structures used so far is mem, and the number remaining in
the current sub-table is rem. Uses the globals max, code, root, large, and // the current sub-table is rem. Uses the globals max, code, root, large, and
done. */ // done.
local void examine(int syms, int len, int left, int mem, int rem) local void examine(int syms, int len, int left, int mem, int rem) {
{ int least; // least number of syms to use at this juncture
int least; /* least number of syms to use at this juncture */ int most; // most number of syms to use at this juncture
int most; /* most number of syms to use at this juncture */ int use; // number of bit patterns to use in next call
int use; /* number of bit patterns to use in next call */
/* see if we have a complete code */ // see if we have a complete code
if (syms == left) { if (syms == left) {
/* set the last code entry */ // set the last code entry
g.code[len] = left; g.code[len] = left;
/* complete computation of memory used by this code */ // complete computation of memory used by this code
while (rem < left) { while (rem < left) {
left -= rem; left -= rem;
rem = 1 << (len - g.root); rem = 1 << (len - g.root);
@ -342,7 +338,7 @@ local void examine(int syms, int len, int left, int mem, int rem)
} }
assert(rem == left); assert(rem == left);
/* if this is a new maximum, show the entries used and the sub-code */ // if this is a new maximum, show the entries used and the sub-code
if (mem > g.large) { if (mem > g.large) {
g.large = mem; g.large = mem;
printf("max %d: ", mem); printf("max %d: ", mem);
@ -353,28 +349,28 @@ local void examine(int syms, int len, int left, int mem, int rem)
fflush(stdout); fflush(stdout);
} }
/* remove entries as we drop back down in the recursion */ // remove entries as we drop back down in the recursion
g.code[len] = 0; g.code[len] = 0;
return; return;
} }
/* prune the tree if we can */ // prune the tree if we can
if (beenhere(syms, len, left, mem, rem)) if (beenhere(syms, len, left, mem, rem))
return; return;
/* we need to use at least this many bit patterns so that the code won't be // we need to use at least this many bit patterns so that the code won't be
incomplete at the next length (more bit patterns than symbols) */ // incomplete at the next length (more bit patterns than symbols)
least = (left << 1) - syms; least = (left << 1) - syms;
if (least < 0) if (least < 0)
least = 0; least = 0;
/* we can use at most this many bit patterns, lest there not be enough // we can use at most this many bit patterns, lest there not be enough
available for the remaining symbols at the maximum length (if there were // available for the remaining symbols at the maximum length (if there were
no limit to the code length, this would become: most = left - 1) */ // no limit to the code length, this would become: most = left - 1)
most = (((code_t)left << (g.max - len)) - syms) / most = (((code_t)left << (g.max - len)) - syms) /
(((code_t)1 << (g.max - len)) - 1); (((code_t)1 << (g.max - len)) - 1);
/* occupy least table spaces, creating new sub-tables as needed */ // occupy least table spaces, creating new sub-tables as needed
use = least; use = least;
while (rem < use) { while (rem < use) {
use -= rem; use -= rem;
@ -383,7 +379,7 @@ local void examine(int syms, int len, int left, int mem, int rem)
} }
rem -= use; rem -= use;
/* examine codes from here, updating table space as we go */ // examine codes from here, updating table space as we go
for (use = least; use <= most; use++) { for (use = least; use <= most; use++) {
g.code[len] = use; g.code[len] = use;
examine(syms - use, len + 1, (left - use) << 1, examine(syms - use, len + 1, (left - use) << 1,
@ -395,79 +391,74 @@ local void examine(int syms, int len, int left, int mem, int rem)
rem--; rem--;
} }
/* remove entries as we drop back down in the recursion */ // remove entries as we drop back down in the recursion
g.code[len] = 0; g.code[len] = 0;
} }
/* Look at all sub-codes starting with root + 1 bits. Look at only the valid // Look at all sub-codes starting with root + 1 bits. Look at only the valid
intermediate code states (syms, left, len). For each completed code, // intermediate code states (syms, left, len). For each completed code,
calculate the amount of memory required by inflate to build the decoding // calculate the amount of memory required by inflate to build the decoding
tables. Find the maximum amount of memory required and show the code that // tables. Find the maximum amount of memory required and show the code that
requires that maximum. Uses the globals max, root, and num. */ // requires that maximum. Uses the globals max, root, and num.
local void enough(int syms) local void enough(int syms) {
{ int n; // number of remaing symbols for this node
int n; /* number of remaing symbols for this node */ int left; // number of unused bit patterns at this length
int left; /* number of unused bit patterns at this length */ size_t index; // index of this case in *num
size_t index; /* index of this case in *num */
/* clear code */ // clear code
for (n = 0; n <= g.max; n++) for (n = 0; n <= g.max; n++)
g.code[n] = 0; g.code[n] = 0;
/* look at all (root + 1) bit and longer codes */ // look at all (root + 1) bit and longer codes
g.large = 1 << g.root; /* base table */ g.large = 1 << g.root; // base table
if (g.root < g.max) /* otherwise, there's only a base table */ if (g.root < g.max) // otherwise, there's only a base table
for (n = 3; n <= syms; n++) for (n = 3; n <= syms; n++)
for (left = 2; left < n; left += 2) for (left = 2; left < n; left += 2) {
{ // look at all reachable (root + 1) bit nodes, and the
/* look at all reachable (root + 1) bit nodes, and the // resulting codes (complete at root + 2 or more)
resulting codes (complete at root + 2 or more) */
index = INDEX(n, left, g.root + 1); index = INDEX(n, left, g.root + 1);
if (g.root + 1 < g.max && g.num[index]) /* reachable node */ if (g.root + 1 < g.max && g.num[index]) // reachable node
examine(n, g.root + 1, left, 1 << g.root, 0); examine(n, g.root + 1, left, 1 << g.root, 0);
/* also look at root bit codes with completions at root + 1 // also look at root bit codes with completions at root + 1
bits (not saved in num, since complete), just in case */ // bits (not saved in num, since complete), just in case
if (g.num[index - 1] && n <= left << 1) if (g.num[index - 1] && n <= left << 1)
examine((n - left) << 1, g.root + 1, (n - left) << 1, examine((n - left) << 1, g.root + 1, (n - left) << 1,
1 << g.root, 0); 1 << g.root, 0);
} }
/* done */ // done
printf("done: maximum of %d table entries\n", g.large); printf("done: maximum of %d table entries\n", g.large);
} }
/* // Examine and show the total number of possible Huffman codes for a given
Examine and show the total number of possible Huffman codes for a given // maximum number of symbols, initial root table size, and maximum code length
maximum number of symbols, initial root table size, and maximum code length // in bits -- those are the command arguments in that order. The default values
in bits -- those are the command arguments in that order. The default // are 286, 9, and 15 respectively, for the deflate literal/length code. The
values are 286, 9, and 15 respectively, for the deflate literal/length code. // possible codes are counted for each number of coded symbols from two to the
The possible codes are counted for each number of coded symbols from two to // maximum. The counts for each of those and the total number of codes are
the maximum. The counts for each of those and the total number of codes are // shown. The maximum number of inflate table entires is then calculated across
shown. The maximum number of inflate table entires is then calculated // all possible codes. Each new maximum number of table entries and the
across all possible codes. Each new maximum number of table entries and the // associated sub-code (starting at root + 1 == 10 bits) is shown.
associated sub-code (starting at root + 1 == 10 bits) is shown. //
// To count and examine Huffman codes that are not length-limited, provide a
// maximum length equal to the number of symbols minus one.
//
// For the deflate literal/length code, use "enough". For the deflate distance
// code, use "enough 30 6".
int main(int argc, char **argv) {
int syms; // total number of symbols to code
int n; // number of symbols to code for this run
big_t got; // return value of count()
big_t sum; // accumulated number of codes over n
code_t word; // for counting bits in code_t
To count and examine Huffman codes that are not length-limited, provide a // set up globals for cleanup()
maximum length equal to the number of symbols minus one.
For the deflate literal/length code, use "enough". For the deflate distance
code, use "enough 30 6".
*/
int main(int argc, char **argv)
{
int syms; /* total number of symbols to code */
int n; /* number of symbols to code for this run */
big_t got; /* return value of count() */
big_t sum; /* accumulated number of codes over n */
code_t word; /* for counting bits in code_t */
/* set up globals for cleanup() */
g.code = NULL; g.code = NULL;
g.num = NULL; g.num = NULL;
g.done = NULL; g.done = NULL;
/* get arguments -- default to the deflate literal/length code */ // get arguments -- default to the deflate literal/length code
syms = 286; syms = 286;
g.root = 9; g.root = 9;
g.max = 15; g.max = 15;
@ -485,38 +476,38 @@ int main(int argc, char **argv)
return 1; return 1;
} }
/* if not restricting the code length, the longest is syms - 1 */ // if not restricting the code length, the longest is syms - 1
if (g.max > syms - 1) if (g.max > syms - 1)
g.max = syms - 1; g.max = syms - 1;
/* determine the number of bits in a code_t */ // determine the number of bits in a code_t
for (n = 0, word = 1; word; n++, word <<= 1) for (n = 0, word = 1; word; n++, word <<= 1)
; ;
/* make sure that the calculation of most will not overflow */ // make sure that the calculation of most will not overflow
if (g.max > n || (code_t)(syms - 2) >= (((code_t)0 - 1) >> (g.max - 1))) { if (g.max > n || (code_t)(syms - 2) >= (((code_t)0 - 1) >> (g.max - 1))) {
fputs("abort: code length too long for internal types\n", stderr); fputs("abort: code length too long for internal types\n", stderr);
return 1; return 1;
} }
/* reject impossible code requests */ // reject impossible code requests
if ((code_t)(syms - 1) > ((code_t)1 << g.max) - 1) { if ((code_t)(syms - 1) > ((code_t)1 << g.max) - 1) {
fprintf(stderr, "%d symbols cannot be coded in %d bits\n", fprintf(stderr, "%d symbols cannot be coded in %d bits\n",
syms, g.max); syms, g.max);
return 1; return 1;
} }
/* allocate code vector */ // allocate code vector
g.code = calloc(g.max + 1, sizeof(int)); g.code = calloc(g.max + 1, sizeof(int));
if (g.code == NULL) { if (g.code == NULL) {
fputs("abort: unable to allocate enough memory\n", stderr); fputs("abort: unable to allocate enough memory\n", stderr);
return 1; return 1;
} }
/* determine size of saved results array, checking for overflows, // determine size of saved results array, checking for overflows,
allocate and clear the array (set all to zero with calloc()) */ // allocate and clear the array (set all to zero with calloc())
if (syms == 2) /* iff max == 1 */ if (syms == 2) // iff max == 1
g.num = NULL; /* won't be saving any results */ g.num = NULL; // won't be saving any results
else { else {
g.size = syms >> 1; g.size = syms >> 1;
if (g.size > ((size_t)0 - 1) / (n = (syms - 1) >> 1) || if (g.size > ((size_t)0 - 1) / (n = (syms - 1) >> 1) ||
@ -529,12 +520,12 @@ int main(int argc, char **argv)
} }
} }
/* count possible codes for all numbers of symbols, add up counts */ // count possible codes for all numbers of symbols, add up counts
sum = 0; sum = 0;
for (n = 2; n <= syms; n++) { for (n = 2; n <= syms; n++) {
got = count(n, 1, 2); got = count(n, 1, 2);
sum += got; sum += got;
if (got == (big_t)0 - 1 || sum < got) { /* overflow */ if (got == (big_t)0 - 1 || sum < got) { // overflow
fputs("abort: can't count that high!\n", stderr); fputs("abort: can't count that high!\n", stderr);
cleanup(); cleanup();
return 1; return 1;
@ -547,7 +538,7 @@ int main(int argc, char **argv)
else else
puts(" (no length limit)"); puts(" (no length limit)");
/* allocate and clear done array for beenhere() */ // allocate and clear done array for beenhere()
if (syms == 2) if (syms == 2)
g.done = NULL; g.done = NULL;
else if (g.size > ((size_t)0 - 1) / sizeof(struct tab) || else if (g.size > ((size_t)0 - 1) / sizeof(struct tab) ||
@ -557,15 +548,15 @@ int main(int argc, char **argv)
return 1; return 1;
} }
/* find and show maximum inflate table usage */ // find and show maximum inflate table usage
if (g.root > g.max) /* reduce root to max length */ if (g.root > g.max) // reduce root to max length
g.root = g.max; g.root = g.max;
if ((code_t)syms < ((code_t)1 << (g.root + 1))) if ((code_t)syms < ((code_t)1 << (g.root + 1)))
enough(syms); enough(syms);
else else
puts("cannot handle minimum code lengths > root"); puts("cannot handle minimum code lengths > root");
/* done */ // done
cleanup(); cleanup();
return 0; return 0;
} }