summaryrefslogtreecommitdiffstats
path: root/dimension/tokenize.c
Commit message (Collapse)AuthorAgeFilesLines
* Even more tokenizer work.Tavian Barnes2009-10-291-102/+73
| | | | | | The tokenizer is now able to tokenize all of POV-Ray 3.6's include files. It turns out that some #includes may be dynamic (ex. with #writes beforehand), so the tokenizer can't directly perform includes - the executor must do this.
* Prefix tokens with DMNSN_T_.Tavian Barnes2009-10-291-96/+96
|
* Tokenize all language directives.Tavian Barnes2009-10-291-4/+52
|
* Complete string parsing.Tavian Barnes2009-10-291-6/+69
|
* Set the locale to `C' during tokenization.Tavian Barnes2009-10-281-5/+27
|
* Search in the same path as the current file for #includes.Tavian Barnes2009-10-281-8/+27
|
* Split tokenizer.sh into multiple tests.Tavian Barnes2009-10-281-1/+1
|
* Store source filename in tokens, give better diagnostics.Tavian Barnes2009-10-281-46/+72
|
* Make '#include' work.Tavian Barnes2009-10-281-0/+33
|
* Tokenize strings and language directives.Tavian Barnes2009-10-281-3/+140
|
* Tokenize keywords.Tavian Barnes2009-10-281-11/+38
|
* Tokenize identifiers.Tavian Barnes2009-10-281-6/+41
|
* Fix line calculation in comment tokenizer.Tavian Barnes2009-10-281-0/+1
|
* Split some tokenizing work out into helper functions.Tavian Barnes2009-10-281-56/+82
|
* Ignore comments in tokenizer.Tavian Barnes2009-10-271-4/+35
|
* Add line and column numbers to tokens.Tavian Barnes2009-10-271-1/+7
|
* Tokenize numeric values.Tavian Barnes2009-10-271-2/+59
|
* Support more symbols in tokenizer.Tavian Barnes2009-10-271-18/+46
|
* Make tokenizer a bit more robust.Tavian Barnes2009-10-271-3/+19
|
* New tokenizer test.Tavian Barnes2009-10-271-4/+4
|
* New mmap-based tokenizer.Tavian Barnes2009-10-271-13/+33
|
* Begin tokenizer.Tavian Barnes2009-10-261-0/+114