diff options
| author | Tavian Barnes <tavianator@gmail.com> | 2009-12-17 03:12:34 -0500 |
|---|---|---|
| committer | Tavian Barnes <tavianator@gmail.com> | 2009-12-17 03:12:34 -0500 |
| commit | c66148484cad66972348ffe850fd23bc9dca60f9 (patch) | |
| tree | d4616778e980773a2ddb4e606f85193d51f4f71c /dimension/tokenize.h | |
| parent | fb8fb378dcce0564085eb6222053a0db57eb480c (diff) | |
| download | dimension-c66148484cad66972348ffe850fd23bc9dca60f9.tar.xz | |
Tokenize on-the-fly.
No more manual tokenization and parsing, giving a simpler interface. Also,
this brings us closer to a dmnsn_realize_string() interface.
Diffstat (limited to 'dimension/tokenize.h')
| -rw-r--r-- | dimension/tokenize.h | 7 |
1 files changed, 4 insertions, 3 deletions
diff --git a/dimension/tokenize.h b/dimension/tokenize.h index 67a386c..541c604 100644 --- a/dimension/tokenize.h +++ b/dimension/tokenize.h @@ -36,10 +36,11 @@ struct dmnsn_token { unsigned int line, col; }; -/* The workhorse */ -dmnsn_array *dmnsn_tokenize(const char *filename, FILE *file); +/* For debugging */ +dmnsn_array *dmnsn_tokenize(FILE *file, const char *filename); -/* Free an array of tokens - use this rather than dmnsn_delete_array() */ +/* Token destruction */ +void dmnsn_delete_token(dmnsn_token token); void dmnsn_delete_tokens(dmnsn_array *tokens); /* Print an S-expression of a list of tokens to `file' */ |