A listing of supported lexer types follows.
下面是一个支持lexer类型的清单。
These directives are used to define the tokens the lexer can return.
这些指令用来定义lexer可以返回的记号。
A good lexer example can help a lot with learning how to write a tokenizer.
一个好的lexer例子会非常有助于学习如何编写断词器（tokenizer）。
Whenever possible, try to develop separate tests for your lexer and your parser.
只要有可能，请尝试为您的lexer和解析器开发单独的测试。
The grammar shown above in typographify.def provides guidance for the design of a Spark lexer/scanner.
上面的 typographify.def中所示的语法提供了Spark 词法分析程序／扫描程序的设计指南。
How to specify TAB width for Alex lexer?
如何指定亚历克斯词法分析器的标签宽度？
Why does the lexer rule for strings takes precedence over all my other rules?
为什么字符串的词法规则优先于我的所有其他的规则？
This does mean that the lexer and the parser both have to know that an empty line separates a header from a body.
这确实意味着lexer和解析器都必须明白，是一个空行将标题从主体中分离了出来。
The Gang advocates abandoning your core language and building an entirely new one atop it, creating your own lexer, parser, grammar, and so on.
四人组主张放弃核心语言并在其上构建全新的语言，创建您自己的lexer、解析器、语法等等。
For lexing purposes, this means that the lexer definition of an integer number, for example, can be used to build the lexer definition of a real number and a fraction.
例如，对于词法分析目的而言，这意味着可以用整数的词法分析器定义来构建实数和分数的词法分析器定义。
These Numbers will be outside the range of possible valid characters; thus, the lexer can return individual characters as themselves, or special tokens using these defined values.
这些数字将会在可能的合法字符范围之外；这样，lexer可以原样返回单个字符本身，或者返回使用这些定义值的记号。
There is no completely standardized way to do this; the most portable solution is to open a temporary file, write data to it, and hand it to the lexer. Here's a sample bit of code to do this.
对此没有完全标准化的方法；最轻便的方法是打开一个临时文件，将数据写入到这个临时文件，然后将它交给lexer。
There is no completely standardized way to do this; the most portable solution is to open a temporary file, write data to it, and hand it to the lexer. Here's a sample bit of code to do this.
对此没有完全标准化的方法；最轻便的方法是打开一个临时文件，将数据写入到这个临时文件，然后将它交给lexer。
应用推荐