Critical severity9.0NVD Advisory· Published Jan 8, 2016· Updated May 6, 2026
CVE-2015-8557
CVE-2015-8557
Description
The FontManager._get_nix_font_path function in formatters/img.py in Pygments 1.2.2 through 2.0.2 allows remote attackers to execute arbitrary commands via shell metacharacters in a font name.
Affected packages
Versions sourced from the GitHub Security Advisory.
| Package | Affected versions | Patched versions |
|---|---|---|
PygmentsPyPI | >= 1.2.2, < 2.1 | 2.1 |
Affected products
14cpe:2.3:a:pygments:pygments:1.2.2:*:*:*:*:*:*:*+ 9 more
- cpe:2.3:a:pygments:pygments:1.2.2:*:*:*:*:*:*:*
- cpe:2.3:a:pygments:pygments:1.3:*:*:*:*:*:*:*
- cpe:2.3:a:pygments:pygments:1.3.1:*:*:*:*:*:*:*
- cpe:2.3:a:pygments:pygments:1.4:*:*:*:*:*:*:*
- cpe:2.3:a:pygments:pygments:1.5:*:*:*:*:*:*:*
- cpe:2.3:a:pygments:pygments:1.6:*:*:*:*:*:*:*
- cpe:2.3:a:pygments:pygments:1.6:rc1:*:*:*:*:*:*
- cpe:2.3:a:pygments:pygments:2.0:*:*:*:*:*:*:*
- cpe:2.3:a:pygments:pygments:2.0.1:*:*:*:*:*:*:*
- cpe:2.3:a:pygments:pygments:2.0:rc1:*:*:*:*:*:*
cpe:2.3:o:canonical:ubuntu_linux:12.04:*:*:*:lts:*:*:*+ 3 more
- cpe:2.3:o:canonical:ubuntu_linux:12.04:*:*:*:lts:*:*:*
- cpe:2.3:o:canonical:ubuntu_linux:14.04:*:*:*:lts:*:*:*
- cpe:2.3:o:canonical:ubuntu_linux:15.04:*:*:*:*:*:*:*
- cpe:2.3:o:canonical:ubuntu_linux:15.10:*:*:*:*:*:*:*
Patches
1db6dd826f862Merged in Javantea/pygments-main (pull request #501)
90 files changed · +9161 −424
AUTHORS+18 −1 modified@@ -7,12 +7,13 @@ Other contributors, listed alphabetically, are: * Sam Aaron -- Ioke lexer * Ali Afshar -- image formatter -* Thomas Aglassinger -- Rexx lexer +* Thomas Aglassinger -- Easytrieve, JCL and Rexx lexers * Kumar Appaiah -- Debian control lexer * Andreas Amann -- AppleScript lexer * Timothy Armstrong -- Dart lexer fixes * Jeffrey Arnold -- R/S, Rd, BUGS, Jags, and Stan lexers * Jeremy Ashkenas -- CoffeeScript lexer +* José Joaquín Atria -- Praat lexer * Stefan Matthias Aust -- Smalltalk lexer * Lucas Bajolet -- Nit lexer * Ben Bangert -- Mako lexers @@ -23,6 +24,8 @@ Other contributors, listed alphabetically, are: * Michael Bayer -- Myghty lexers * Thomas Beale -- Archetype lexers * John Benediktsson -- Factor lexer +* Trevor Bergeron -- mIRC formatter +* Vincent Bernat -- LessCSS lexer * Christopher Bertels -- Fancy lexer * Jarrett Billingsley -- MiniD lexer * Adam Blinkinsop -- Haskell, Redcode lexers @@ -36,11 +39,14 @@ Other contributors, listed alphabetically, are: * Daniël W. Crompton - Pike lexer * Pete Curry -- bugfixes * Bryan Davis -- EBNF lexer +* Bruno Deferrari -- Shen lexer * Owen Durni -- Haxe lexer * Alexander Dutton, Oxford University Computing Services -- SPARQL lexer +* James Edwards -- Terraform lexer * Nick Efford -- Python 3 lexer * Sven Efftinge -- Xtend lexer * Artem Egorkine -- terminal256 formatter +* Michael Ficarra -- CPSA lexer * James H. Fisher -- PostScript lexer * William S. Fulton -- SWIG lexer * Carlos Galdino -- Elixir and Elixir Console lexers @@ -55,14 +61,18 @@ Other contributors, listed alphabetically, are: * Krzysiek Goj -- Scala lexer * Matt Good -- Genshi, Cheetah lexers * Michał Górny -- vim modeline support +* Alex Gosse -- TrafficScript lexer * Patrick Gotthardt -- PHP namespaces support * Olivier Guibe -- Asymptote lexer * Jordi Gutiérrez Hermoso -- Octave lexer +* Florian Hahn -- Boogie lexer * Martin Harriman -- SNOBOL lexer * Matthew Harrison -- SVG formatter * Steven Hazel -- Tcl lexer +* Dan Michael Heggø -- Turtle lexer * Aslak Hellesøy -- Gherkin lexer * Greg Hendershott -- Racket lexer +* Justin Hendrick -- ParaSail lexer * David Hess, Fish Software, Inc. -- Objective-J lexer * Varun Hiremath -- Debian control lexer * Rob Hoelz -- Perl 6 lexer @@ -73,6 +83,7 @@ Other contributors, listed alphabetically, are: * Alastair Houghton -- Lexer inheritance facility * Tim Howard -- BlitzMax lexer * Ivan Inozemtsev -- Fantom lexer +* Hiroaki Itoh -- Shell console rewrite * Brian R. Jackson -- Tea lexer * Christian Jann -- ShellSession lexer * Dennis Kaarsemaker -- sources.list lexer @@ -93,10 +104,12 @@ Other contributors, listed alphabetically, are: * Gerd Kurzbach -- Modelica lexer * Jon Larimer, Google Inc. -- Smali lexer * Olov Lassus -- Dart lexer +* Matt Layman -- TAP lexer * Sylvestre Ledru -- Scilab lexer * Mark Lee -- Vala lexer * Ben Mabey -- Gherkin lexer * Angus MacArthur -- QML lexer +* Louis Mandel -- X10 lexer * Louis Marchand -- Eiffel lexer * Simone Margaritelli -- Hybris lexer * Kirk McDonald -- D lexer @@ -114,6 +127,7 @@ Other contributors, listed alphabetically, are: * Jon Morton -- Rust lexer * Paulo Moura -- Logtalk lexer * Mher Movsisyan -- DTD lexer +* Dejan Muhamedagic -- Crmsh lexer * Ana Nelson -- Ragel, ANTLR, R console lexers * Nam T. Nguyen -- Monokai style * Jesper Noehr -- HTML formatter "anchorlinenos" @@ -153,6 +167,8 @@ Other contributors, listed alphabetically, are: * Jerome St-Louis -- eC lexer * James Strachan -- Kotlin lexer * Tom Stuart -- Treetop lexer +* Colin Sullivan -- SuperCollider lexer +* Edoardo Tenani -- Arduino lexer * Tiberius Teng -- default style overhaul * Jeremy Thurgood -- Erlang, Squid config lexers * Brian Tiffin -- OpenCOBOL lexer @@ -174,5 +190,6 @@ Other contributors, listed alphabetically, are: * Enrique Zamudio -- Ceylon lexer * Alex Zimin -- Nemerle lexer * Rob Zimmerman -- Kal lexer +* Vincent Zurczak -- Roboconf lexer Many thanks for all contributions!
CHANGES+34 −0 modified@@ -18,18 +18,52 @@ Version 2.1 * Modula-2 with multi-dialect support (#1090) * Fortran fixed format (PR#213) * Archetype Definition language (PR#483) + * Terraform (PR#432) + * Jcl, Easytrieve (PR#208) + * ParaSail (PR#381) + * Boogie (PR#420) + * Turtle (PR#425) + * Fish Shell (PR#422) + * Roboconf (PR#449) + * Test Anything Protocol (PR#428) + * Shen (PR#385) + * Component Pascal (PR#437) + * SuperCollider (PR#472) + * Shell consoles (Tcsh, PowerShell, MSDOS) (PR#479) + * Elm and J (PR#452) + * Crmsh (PR#440) + * Praat (PR#492) + - Added styles: * Lovelace (PR#456) * Algol and Algol-nu (#1090) + +- Added formatters: + + * IRC (PR#458) + + - Updated autopygmentize script (PR#445) - Fixed style inheritance for non-standard token types in HTML output. - Added support for async/await to Python 3 lexer. +- Rewrote linenos option for TerminalFormatter (it's better, but slightly + different output than before). (#1147) + +- Javascript lexer now supports most of ES6. (#1100) + +- Cocoa builtins updated for iOS 8.1 (PR#433) + +- Combined BashSessionLexer and ShellSessionLexer, new version should support + the prompt styles of either. + +- Added option to pygmentize to show a full traceback on exceptions. + Version 2.0.3 -------------
doc/docs/lexerdevelopment.rst+3 −4 modified@@ -145,7 +145,7 @@ Regex Flags You can either define regex flags locally in the regex (``r'(?x)foo bar'``) or globally by adding a `flags` attribute to your lexer class. If no attribute is -defined, it defaults to `re.MULTILINE`. For more informations about regular +defined, it defaults to `re.MULTILINE`. For more information about regular expression flags see the page about `regular expressions`_ in the Python documentation. @@ -345,15 +345,14 @@ There are a few more things you can do with states: `PythonLexer`'s string literal processing. - If you want your lexer to start lexing in a different state you can modify the - stack by overloading the `get_tokens_unprocessed()` method:: + stack by overriding the `get_tokens_unprocessed()` method:: from pygments.lexer import RegexLexer class ExampleLexer(RegexLexer): tokens = {...} - def get_tokens_unprocessed(self, text): - stack = ['root', 'otherstate'] + def get_tokens_unprocessed(self, text, stack=('root', 'otherstate')): for item in RegexLexer.get_tokens_unprocessed(text, stack): yield item
.hgignore+3 −0 modified@@ -2,15 +2,18 @@ syntax: glob *.egg *.pyc *.pyo +.*.sw[op] .idea/ .ropeproject +.project .tags .tox Pygments.egg-info/* TAGS build/* dist/* doc/_build +TAGS tests/.coverage tests/cover tests/examplefiles/output
pygments/cmdline.py+6 −5 modified@@ -19,11 +19,12 @@ from pygments.util import ClassNotFound, OptionError, docstring_headline, \ guess_decode, guess_decode_from_terminal, terminal_encoding from pygments.lexers import get_all_lexers, get_lexer_by_name, guess_lexer, \ - get_lexer_for_filename, find_lexer_class_for_filename, TextLexer + get_lexer_for_filename, find_lexer_class_for_filename +from pygments.lexers.special import TextLexer from pygments.formatters.latex import LatexEmbeddedLexer, LatexFormatter from pygments.formatters import get_all_formatters, get_formatter_by_name, \ - get_formatter_for_filename, find_formatter_class, \ - TerminalFormatter # pylint:disable-msg=E0611 + get_formatter_for_filename, find_formatter_class +from pygments.formatters.terminal import TerminalFormatter from pygments.filters import get_all_filters, find_filter_class from pygments.styles import get_all_styles, get_style_by_name @@ -247,7 +248,7 @@ def main_inner(popts, args, usage): print(usage, file=sys.stderr) return 2 - what, name = args + what, name = args # pylint: disable=unbalanced-tuple-unpacking if what not in ('lexer', 'formatter', 'filter'): print(usage, file=sys.stderr) return 2 @@ -269,7 +270,7 @@ def main_inner(popts, args, usage): opts.pop('-P', None) # encodings - inencoding = parsed_opts.get('inencoding', parsed_opts.get('encoding')) + inencoding = parsed_opts.get('inencoding', parsed_opts.get('encoding')) outencoding = parsed_opts.get('outencoding', parsed_opts.get('encoding')) # handle ``pygmentize -N``
pygments/filter.py+1 −1 modified@@ -69,6 +69,6 @@ def __init__(self, **options): Filter.__init__(self, **options) def filter(self, lexer, stream): - # pylint: disable-msg=E1102 + # pylint: disable=not-callable for ttype, value in self.function(lexer, stream, self.options): yield ttype, value
pygments/formatters/html.py+1 −1 modified@@ -140,7 +140,7 @@ class HtmlFormatter(Formatter): When `tagsfile` is set to the path of a ctags index file, it is used to generate hyperlinks from names to their definition. You must enable - `anchorlines` and run ctags with the `-n` option for this to work. The + `lineanchors` and run ctags with the `-n` option for this to work. The `python-ctags` module from PyPI must be installed to use this feature; otherwise a `RuntimeError` will be raised.
pygments/formatters/irc.py+182 −0 added@@ -0,0 +1,182 @@ +# -*- coding: utf-8 -*- +""" + pygments.formatters.irc + ~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + + Formatter for IRC output + + :copyright: Copyright 2006-2015 by the Pygments team, see AUTHORS. + :license: BSD, see LICENSE for details. +""" + +import sys + +from pygments.formatter import Formatter +from pygments.token import Keyword, Name, Comment, String, Error, \ + Number, Operator, Generic, Token, Whitespace +from pygments.util import get_choice_opt + + +__all__ = ['IRCFormatter'] + + +#: Map token types to a tuple of color values for light and dark +#: backgrounds. +IRC_COLORS = { + Token: ('', ''), + + Whitespace: ('lightgray', 'darkgray'), + Comment: ('lightgray', 'darkgray'), + Comment.Preproc: ('teal', 'turquoise'), + Keyword: ('darkblue', 'blue'), + Keyword.Type: ('teal', 'turquoise'), + Operator.Word: ('purple', 'fuchsia'), + Name.Builtin: ('teal', 'turquoise'), + Name.Function: ('darkgreen', 'green'), + Name.Namespace: ('_teal_', '_turquoise_'), + Name.Class: ('_darkgreen_', '_green_'), + Name.Exception: ('teal', 'turquoise'), + Name.Decorator: ('darkgray', 'lightgray'), + Name.Variable: ('darkred', 'red'), + Name.Constant: ('darkred', 'red'), + Name.Attribute: ('teal', 'turquoise'), + Name.Tag: ('blue', 'blue'), + String: ('brown', 'brown'), + Number: ('darkblue', 'blue'), + + Generic.Deleted: ('red', 'red'), + Generic.Inserted: ('darkgreen', 'green'), + Generic.Heading: ('**', '**'), + Generic.Subheading: ('*purple*', '*fuchsia*'), + Generic.Error: ('red', 'red'), + + Error: ('_red_', '_red_'), +} + + +IRC_COLOR_MAP = { + 'white': 0, + 'black': 1, + 'darkblue': 2, + 'green': 3, + 'red': 4, + 'brown': 5, + 'purple': 6, + 'orange': 7, + 'darkgreen': 7, #compat w/ ansi + 'yellow': 8, + 'lightgreen': 9, + 'turquoise': 9, # compat w/ ansi + 'teal': 10, + 'lightblue': 11, + 'darkred': 11, # compat w/ ansi + 'blue': 12, + 'fuchsia': 13, + 'darkgray': 14, + 'lightgray': 15, +} + +def ircformat(color, text): + if len(color) < 1: + return text + add = sub = '' + if '_' in color: # italic + add += '\x1D' + sub = '\x1D' + sub + color = color.strip('_') + if '*' in color: # bold + add += '\x02' + sub = '\x02' + sub + color = color.strip('*') + # underline (\x1F) not supported + # backgrounds (\x03FF,BB) not supported + if len(color) > 0: # actual color - may have issues with ircformat("red", "blah")+"10" type stuff + add += '\x03' + str(IRC_COLOR_MAP[color]).zfill(2) + sub = '\x03' + sub + return add + text + sub + return '<'+add+'>'+text+'</'+sub+'>' + + +class IRCFormatter(Formatter): + r""" + Format tokens with IRC color sequences + + The `get_style_defs()` method doesn't do anything special since there is + no support for common styles. + + Options accepted: + + `bg` + Set to ``"light"`` or ``"dark"`` depending on the terminal's background + (default: ``"light"``). + + `colorscheme` + A dictionary mapping token types to (lightbg, darkbg) color names or + ``None`` (default: ``None`` = use builtin colorscheme). + + `linenos` + Set to ``True`` to have line numbers in the output as well + (default: ``False`` = no line numbers). + """ + name = 'IRC' + aliases = ['irc', 'IRC'] + filenames = [] + + def __init__(self, **options): + Formatter.__init__(self, **options) + self.darkbg = get_choice_opt(options, 'bg', + ['light', 'dark'], 'light') == 'dark' + self.colorscheme = options.get('colorscheme', None) or IRC_COLORS + self.linenos = options.get('linenos', False) + self._lineno = 0 + + def _write_lineno(self, outfile): + self._lineno += 1 + outfile.write("\n%04d: " % self._lineno) + + def _format_unencoded_with_lineno(self, tokensource, outfile): + self._write_lineno(outfile) + + for ttype, value in tokensource: + if value.endswith("\n"): + self._write_lineno(outfile) + value = value[:-1] + color = self.colorscheme.get(ttype) + while color is None: + ttype = ttype[:-1] + color = self.colorscheme.get(ttype) + if color: + color = color[self.darkbg] + spl = value.split('\n') + for line in spl[:-1]: + self._write_lineno(outfile) + if line: + outfile.write(ircformat(color, line[:-1])) + if spl[-1]: + outfile.write(ircformat(color, spl[-1])) + else: + outfile.write(value) + + outfile.write("\n") + + def format_unencoded(self, tokensource, outfile): + if self.linenos: + self._format_unencoded_with_lineno(tokensource, outfile) + return + + for ttype, value in tokensource: + color = self.colorscheme.get(ttype) + while color is None: + ttype = ttype[:-1] + color = self.colorscheme.get(ttype) + if color: + color = color[self.darkbg] + spl = value.split('\n') + for line in spl[:-1]: + if line: + outfile.write(ircformat(color, line)) + outfile.write('\n') + if spl[-1]: + outfile.write(ircformat(color, spl[-1])) + else: + outfile.write(value)
pygments/formatters/_mapping.py+1 −0 modified@@ -20,6 +20,7 @@ 'BmpImageFormatter': ('pygments.formatters.img', 'img_bmp', ('bmp', 'bitmap'), ('*.bmp',), 'Create a bitmap image from source code. This uses the Python Imaging Library to generate a pixmap from the source code.'), 'GifImageFormatter': ('pygments.formatters.img', 'img_gif', ('gif',), ('*.gif',), 'Create a GIF image from source code. This uses the Python Imaging Library to generate a pixmap from the source code.'), 'HtmlFormatter': ('pygments.formatters.html', 'HTML', ('html',), ('*.html', '*.htm'), "Format tokens as HTML 4 ``<span>`` tags within a ``<pre>`` tag, wrapped in a ``<div>`` tag. The ``<div>``'s CSS class can be set by the `cssclass` option."), + 'IRCFormatter': ('pygments.formatters.irc', 'IRC', ('irc', 'IRC'), (), 'Format tokens with IRC color sequences'), 'ImageFormatter': ('pygments.formatters.img', 'img', ('img', 'IMG', 'png'), ('*.png',), 'Create a PNG image from source code. This uses the Python Imaging Library to generate a pixmap from the source code.'), 'JpgImageFormatter': ('pygments.formatters.img', 'img_jpg', ('jpg', 'jpeg'), ('*.jpg',), 'Create a JPEG image from source code. This uses the Python Imaging Library to generate a pixmap from the source code.'), 'LatexFormatter': ('pygments.formatters.latex', 'LaTeX', ('latex', 'tex'), ('*.tex',), 'Format tokens as LaTeX code. This needs the `fancyvrb` and `color` standard packages.'),
pygments/formatters/terminal.py+28 −43 modified@@ -49,6 +49,7 @@ Generic.Inserted: ('darkgreen', 'green'), Generic.Heading: ('**', '**'), Generic.Subheading: ('*purple*', '*fuchsia*'), + Generic.Prompt: ('**', '**'), Generic.Error: ('red', 'red'), Error: ('_red_', '_red_'), @@ -101,51 +102,35 @@ def format(self, tokensource, outfile): def _write_lineno(self, outfile): self._lineno += 1 - outfile.write("\n%04d: " % self._lineno) - - def _format_unencoded_with_lineno(self, tokensource, outfile): - self._write_lineno(outfile) - - for ttype, value in tokensource: - if value.endswith("\n"): - self._write_lineno(outfile) - value = value[:-1] - color = self.colorscheme.get(ttype) - while color is None: - ttype = ttype[:-1] - color = self.colorscheme.get(ttype) - if color: - color = color[self.darkbg] - spl = value.split('\n') - for line in spl[:-1]: - self._write_lineno(outfile) - if line: - outfile.write(ansiformat(color, line[:-1])) - if spl[-1]: - outfile.write(ansiformat(color, spl[-1])) - else: - outfile.write(value) - - outfile.write("\n") + outfile.write("%s%04d: " % (self._lineno != 1 and '\n' or '', self._lineno)) + + def _get_color(self, ttype): + # self.colorscheme is a dict containing usually generic types, so we + # have to walk the tree of dots. The base Token type must be a key, + # even if it's empty string, as in the default above. + colors = self.colorscheme.get(ttype) + while colors is None: + ttype = ttype.parent + colors = self.colorscheme.get(ttype) + return colors[self.darkbg] def format_unencoded(self, tokensource, outfile): if self.linenos: - self._format_unencoded_with_lineno(tokensource, outfile) - return + self._write_lineno(outfile) for ttype, value in tokensource: - color = self.colorscheme.get(ttype) - while color is None: - ttype = ttype[:-1] - color = self.colorscheme.get(ttype) - if color: - color = color[self.darkbg] - spl = value.split('\n') - for line in spl[:-1]: - if line: - outfile.write(ansiformat(color, line)) - outfile.write('\n') - if spl[-1]: - outfile.write(ansiformat(color, spl[-1])) - else: - outfile.write(value) + color = self._get_color(ttype) + + for line in value.splitlines(True): + if color: + outfile.write(ansiformat(color, line.rstrip('\n'))) + else: + outfile.write(line.rstrip('\n')) + if line.endswith('\n'): + if self.linenos: + self._write_lineno(outfile) + else: + outfile.write('\n') + + if self.linenos: + outfile.write("\n")
pygments/__init__.py+3 −3 modified@@ -46,13 +46,13 @@ def lex(code, lexer): except TypeError as err: if isinstance(err.args[0], str) and \ ('unbound method get_tokens' in err.args[0] or - 'missing 1 required positional argument' in err.args[0]): + 'missing 1 required positional argument' in err.args[0]): raise TypeError('lex() argument must be a lexer instance, ' 'not a class') raise -def format(tokens, formatter, outfile=None): +def format(tokens, formatter, outfile=None): # pylint: disable=redefined-builtin """ Format a tokenlist ``tokens`` with the formatter ``formatter``. @@ -70,7 +70,7 @@ def format(tokens, formatter, outfile=None): except TypeError as err: if isinstance(err.args[0], str) and \ ('unbound method format' in err.args[0] or - 'missing 1 required positional argument' in err.args[0]): + 'missing 1 required positional argument' in err.args[0]): raise TypeError('format() argument must be a formatter instance, ' 'not a class') raise
pygments/lexer.py+8 −9 modified@@ -14,7 +14,6 @@ import re import sys import time -import itertools from pygments.filter import apply_filters, Filter from pygments.filters import get_filter_by_name @@ -43,10 +42,10 @@ class LexerMeta(type): static methods which always return float values. """ - def __new__(cls, name, bases, d): + def __new__(mcs, name, bases, d): if 'analyse_text' in d: d['analyse_text'] = make_analysator(d['analyse_text']) - return type.__new__(cls, name, bases, d) + return type.__new__(mcs, name, bases, d) @add_metaclass(LexerMeta) @@ -189,7 +188,7 @@ def get_tokens(self, text, unfiltered=False): text += '\n' def streamer(): - for i, t, v in self.get_tokens_unprocessed(text): + for _, t, v in self.get_tokens_unprocessed(text): yield t, v stream = streamer() if not unfiltered: @@ -246,7 +245,7 @@ def get_tokens_unprocessed(self, text): # -class include(str): +class include(str): # pylint: disable=invalid-name """ Indicates that a state should include rules from another state. """ @@ -260,10 +259,10 @@ class _inherit(object): def __repr__(self): return 'inherit' -inherit = _inherit() +inherit = _inherit() # pylint: disable=invalid-name -class combined(tuple): +class combined(tuple): # pylint: disable=invalid-name """ Indicates a state combined from multiple states. """ @@ -320,8 +319,8 @@ def callback(lexer, match, ctx=None): if data is not None: if ctx: ctx.pos = match.start(i + 1) - for item in action(lexer, _PseudoMatch(match.start(i + 1), - data), ctx): + for item in action( + lexer, _PseudoMatch(match.start(i + 1), data), ctx): if item: yield item if ctx:
pygments/lexers/archetype.py+3 −1 modified@@ -274,6 +274,8 @@ class AdlLexer(AtomsLexer): (r'^(definition)[ \t]*\n', Generic.Heading, 'cadl_section'), (r'^([ \t]*|[ \t]+.*)\n', using(OdinLexer)), (r'^([^"]*")(>[ \t]*\n)', bygroups(String, Punctuation)), + # template overlay delimiter + (r'^----------*\n', Text, '#pop'), (r'^.*\n', String), default('#pop'), ], @@ -300,7 +302,7 @@ class AdlLexer(AtomsLexer): default('#pop'), ], 'root': [ - (r'^(archetype|template|template_overlay|operational_template|' + (r'^(archetype|template_overlay|operational_template|template|' r'speciali[sz]e)', Generic.Heading), (r'^(language|description|ontology|terminology|annotations|' r'component_terminologies|revision_history)[ \t]*\n',
pygments/lexers/asm.py+2 −1 modified@@ -286,7 +286,8 @@ class LlvmLexer(RegexLexer): r'|lshr|ashr|and|or|xor|icmp|fcmp' r'|phi|call|trunc|zext|sext|fptrunc|fpext|uitofp|sitofp|fptoui' - r'|fptosi|inttoptr|ptrtoint|bitcast|select|va_arg|ret|br|switch' + r'|fptosi|inttoptr|ptrtoint|bitcast|addrspacecast' + r'|select|va_arg|ret|br|switch' r'|invoke|unwind|unreachable' r'|indirectbr|landingpad|resume'
pygments/lexers/chapel.py+4 −4 modified@@ -46,10 +46,10 @@ class ChapelLexer(RegexLexer): 'continue', 'delete', 'dmapped', 'do', 'domain', 'else', 'enum', 'export', 'extern', 'for', 'forall', 'if', 'index', 'inline', 'iter', 'label', 'lambda', 'let', 'local', 'new', 'noinit', 'on', - 'otherwise', 'pragma', 'private', 'public', 'reduce', 'return', - 'scan', 'select', 'serial', 'single', 'sparse', 'subdomain', - 'sync', 'then', 'use', 'when', 'where', 'while', 'with', 'yield', - 'zip'), suffix=r'\b'), + 'otherwise', 'pragma', 'private', 'public', 'reduce', + 'require', 'return', 'scan', 'select', 'serial', 'single', + 'sparse', 'subdomain', 'sync', 'then', 'use', 'when', 'where', + 'while', 'with', 'yield', 'zip'), suffix=r'\b'), Keyword), (r'(proc)((?:\s|\\\s)+)', bygroups(Keyword, Text), 'procname'), (r'(class|module|record|union)(\s+)', bygroups(Keyword, Text),
pygments/lexers/c_like.py+83 −81 modified@@ -418,6 +418,8 @@ class ArduinoLexer(CppLexer): This is an extension of the CppLexer, as the Arduino® Language is a superset of C++ + + .. versionadded:: 2.1 """ name = 'Arduino' @@ -426,93 +428,93 @@ class ArduinoLexer(CppLexer): mimetypes = ['text/x-arduino'] # Language constants - constants = set(( 'DIGITAL_MESSAGE', 'FIRMATA_STRING', 'ANALOG_MESSAGE', - 'REPORT_DIGITAL', 'REPORT_ANALOG', 'INPUT_PULLUP', + constants = set(('DIGITAL_MESSAGE', 'FIRMATA_STRING', 'ANALOG_MESSAGE', + 'REPORT_DIGITAL', 'REPORT_ANALOG', 'INPUT_PULLUP', 'SET_PIN_MODE', 'INTERNAL2V56', 'SYSTEM_RESET', 'LED_BUILTIN', - 'INTERNAL1V1', 'SYSEX_START', 'INTERNAL', 'EXTERNAL', - 'DEFAULT', 'OUTPUT', 'INPUT', 'HIGH', 'LOW' )) + 'INTERNAL1V1', 'SYSEX_START', 'INTERNAL', 'EXTERNAL', + 'DEFAULT', 'OUTPUT', 'INPUT', 'HIGH', 'LOW')) # Language sketch main structure functions - structure = set(( 'setup', 'loop' )) + structure = set(('setup', 'loop')) # Language variable types - storage = set(( 'boolean', 'const', 'byte', 'word', 'string', 'String', 'array' )) + storage = set(('boolean', 'const', 'byte', 'word', 'string', 'String', 'array')) # Language shipped functions and class ( ) - functions = set(( 'KeyboardController', 'MouseController', 'SoftwareSerial', - 'EthernetServer', 'EthernetClient', 'LiquidCrystal', - 'RobotControl', 'GSMVoiceCall', 'EthernetUDP', 'EsploraTFT', - 'HttpClient', 'RobotMotor', 'WiFiClient', 'GSMScanner', - 'FileSystem', 'Scheduler', 'GSMServer', 'YunClient', 'YunServer', - 'IPAddress', 'GSMClient', 'GSMModem', 'Keyboard', 'Ethernet', - 'Console', 'GSMBand', 'Esplora', 'Stepper', 'Process', - 'WiFiUDP', 'GSM_SMS', 'Mailbox', 'USBHost', 'Firmata', 'PImage', - 'Client', 'Server', 'GSMPIN', 'FileIO', 'Bridge', 'Serial', - 'EEPROM', 'Stream', 'Mouse', 'Audio', 'Servo', 'File', 'Task', - 'GPRS', 'WiFi', 'Wire', 'TFT', 'GSM', 'SPI', 'SD', - 'runShellCommandAsynchronously', 'analogWriteResolution', - 'retrieveCallingNumber', 'printFirmwareVersion', - 'analogReadResolution', 'sendDigitalPortPair', - 'noListenOnLocalhost', 'readJoystickButton', 'setFirmwareVersion', - 'readJoystickSwitch', 'scrollDisplayRight', 'getVoiceCallStatus', - 'scrollDisplayLeft', 'writeMicroseconds', 'delayMicroseconds', - 'beginTransmission', 'getSignalStrength', 'runAsynchronously', - 'getAsynchronously', 'listenOnLocalhost', 'getCurrentCarrier', - 'readAccelerometer', 'messageAvailable', 'sendDigitalPorts', - 'lineFollowConfig', 'countryNameWrite', 'runShellCommand', - 'readStringUntil', 'rewindDirectory', 'readTemperature', - 'setClockDivider', 'readLightSensor', 'endTransmission', - 'analogReference', 'detachInterrupt', 'countryNameRead', - 'attachInterrupt', 'encryptionType', 'readBytesUntil', - 'robotNameWrite', 'readMicrophone', 'robotNameRead', 'cityNameWrite', - 'userNameWrite', 'readJoystickY', 'readJoystickX', 'mouseReleased', - 'openNextFile', 'scanNetworks', 'noInterrupts', 'digitalWrite', - 'beginSpeaker', 'mousePressed', 'isActionDone', 'mouseDragged', - 'displayLogos', 'noAutoscroll', 'addParameter', 'remoteNumber', - 'getModifiers', 'keyboardRead', 'userNameRead', 'waitContinue', - 'processInput', 'parseCommand', 'printVersion', 'readNetworks', - 'writeMessage', 'blinkVersion', 'cityNameRead', 'readMessage', - 'setDataMode', 'parsePacket', 'isListening', 'setBitOrder', - 'beginPacket', 'isDirectory', 'motorsWrite', 'drawCompass', - 'digitalRead', 'clearScreen', 'serialEvent', 'rightToLeft', - 'setTextSize', 'leftToRight', 'requestFrom', 'keyReleased', - 'compassRead', 'analogWrite', 'interrupts', 'WiFiServer', - 'disconnect', 'playMelody', 'parseFloat', 'autoscroll', - 'getPINUsed', 'setPINUsed', 'setTimeout', 'sendAnalog', - 'readSlider', 'analogRead', 'beginWrite', 'createChar', - 'motorsStop', 'keyPressed', 'tempoWrite', 'readButton', - 'subnetMask', 'debugPrint', 'macAddress', 'writeGreen', - 'randomSeed', 'attachGPRS', 'readString', 'sendString', - 'remotePort', 'releaseAll', 'mouseMoved', 'background', - 'getXChange', 'getYChange', 'answerCall', 'getResult', - 'voiceCall', 'endPacket', 'constrain', 'getSocket', 'writeJSON', - 'getButton', 'available', 'connected', 'findUntil', 'readBytes', - 'exitValue', 'readGreen', 'writeBlue', 'startLoop', 'IPAddress', - 'isPressed', 'sendSysex', 'pauseMode', 'gatewayIP', 'setCursor', - 'getOemKey', 'tuneWrite', 'noDisplay', 'loadImage', 'switchPIN', - 'onRequest', 'onReceive', 'changePIN', 'playFile', 'noBuffer', - 'parseInt', 'overflow', 'checkPIN', 'knobRead', 'beginTFT', - 'bitClear', 'updateIR', 'bitWrite', 'position', 'writeRGB', - 'highByte', 'writeRed', 'setSpeed', 'readBlue', 'noStroke', - 'remoteIP', 'transfer', 'shutdown', 'hangCall', 'beginSMS', - 'endWrite', 'attached', 'maintain', 'noCursor', 'checkReg', - 'checkPUK', 'shiftOut', 'isValid', 'shiftIn', 'pulseIn', - 'connect', 'println', 'localIP', 'pinMode', 'getIMEI', - 'display', 'noBlink', 'process', 'getBand', 'running', 'beginSD', - 'drawBMP', 'lowByte', 'setBand', 'release', 'bitRead', 'prepare', - 'pointTo', 'readRed', 'setMode', 'noFill', 'remove', 'listen', - 'stroke', 'detach', 'attach', 'noTone', 'exists', 'buffer', - 'height', 'bitSet', 'circle', 'config', 'cursor', 'random', - 'IRread', 'sizeof', 'setDNS', 'endSMS', 'getKey', 'micros', - 'millis', 'begin', 'print', 'write', 'ready', 'flush', 'width', - 'isPIN', 'blink', 'clear', 'press', 'mkdir', 'rmdir', 'close', - 'point', 'yield', 'image', 'float', 'BSSID', 'click', 'delay', - 'read', 'text', 'move', 'peek', 'beep', 'rect', 'line', 'open', - 'seek', 'fill', 'size', 'turn', 'stop', 'home', 'find', 'char', - 'byte', 'step', 'word', 'long', 'tone', 'sqrt', 'RSSI', 'SSID', - 'end', 'bit', 'tan', 'cos', 'sin', 'pow', 'map', 'abs', 'max', - 'min', 'int', 'get', 'run', 'put' )) - + functions = set(('KeyboardController', 'MouseController', 'SoftwareSerial', + 'EthernetServer', 'EthernetClient', 'LiquidCrystal', + 'RobotControl', 'GSMVoiceCall', 'EthernetUDP', 'EsploraTFT', + 'HttpClient', 'RobotMotor', 'WiFiClient', 'GSMScanner', + 'FileSystem', 'Scheduler', 'GSMServer', 'YunClient', 'YunServer', + 'IPAddress', 'GSMClient', 'GSMModem', 'Keyboard', 'Ethernet', + 'Console', 'GSMBand', 'Esplora', 'Stepper', 'Process', + 'WiFiUDP', 'GSM_SMS', 'Mailbox', 'USBHost', 'Firmata', 'PImage', + 'Client', 'Server', 'GSMPIN', 'FileIO', 'Bridge', 'Serial', + 'EEPROM', 'Stream', 'Mouse', 'Audio', 'Servo', 'File', 'Task', + 'GPRS', 'WiFi', 'Wire', 'TFT', 'GSM', 'SPI', 'SD', + 'runShellCommandAsynchronously', 'analogWriteResolution', + 'retrieveCallingNumber', 'printFirmwareVersion', + 'analogReadResolution', 'sendDigitalPortPair', + 'noListenOnLocalhost', 'readJoystickButton', 'setFirmwareVersion', + 'readJoystickSwitch', 'scrollDisplayRight', 'getVoiceCallStatus', + 'scrollDisplayLeft', 'writeMicroseconds', 'delayMicroseconds', + 'beginTransmission', 'getSignalStrength', 'runAsynchronously', + 'getAsynchronously', 'listenOnLocalhost', 'getCurrentCarrier', + 'readAccelerometer', 'messageAvailable', 'sendDigitalPorts', + 'lineFollowConfig', 'countryNameWrite', 'runShellCommand', + 'readStringUntil', 'rewindDirectory', 'readTemperature', + 'setClockDivider', 'readLightSensor', 'endTransmission', + 'analogReference', 'detachInterrupt', 'countryNameRead', + 'attachInterrupt', 'encryptionType', 'readBytesUntil', + 'robotNameWrite', 'readMicrophone', 'robotNameRead', 'cityNameWrite', + 'userNameWrite', 'readJoystickY', 'readJoystickX', 'mouseReleased', + 'openNextFile', 'scanNetworks', 'noInterrupts', 'digitalWrite', + 'beginSpeaker', 'mousePressed', 'isActionDone', 'mouseDragged', + 'displayLogos', 'noAutoscroll', 'addParameter', 'remoteNumber', + 'getModifiers', 'keyboardRead', 'userNameRead', 'waitContinue', + 'processInput', 'parseCommand', 'printVersion', 'readNetworks', + 'writeMessage', 'blinkVersion', 'cityNameRead', 'readMessage', + 'setDataMode', 'parsePacket', 'isListening', 'setBitOrder', + 'beginPacket', 'isDirectory', 'motorsWrite', 'drawCompass', + 'digitalRead', 'clearScreen', 'serialEvent', 'rightToLeft', + 'setTextSize', 'leftToRight', 'requestFrom', 'keyReleased', + 'compassRead', 'analogWrite', 'interrupts', 'WiFiServer', + 'disconnect', 'playMelody', 'parseFloat', 'autoscroll', + 'getPINUsed', 'setPINUsed', 'setTimeout', 'sendAnalog', + 'readSlider', 'analogRead', 'beginWrite', 'createChar', + 'motorsStop', 'keyPressed', 'tempoWrite', 'readButton', + 'subnetMask', 'debugPrint', 'macAddress', 'writeGreen', + 'randomSeed', 'attachGPRS', 'readString', 'sendString', + 'remotePort', 'releaseAll', 'mouseMoved', 'background', + 'getXChange', 'getYChange', 'answerCall', 'getResult', + 'voiceCall', 'endPacket', 'constrain', 'getSocket', 'writeJSON', + 'getButton', 'available', 'connected', 'findUntil', 'readBytes', + 'exitValue', 'readGreen', 'writeBlue', 'startLoop', 'IPAddress', + 'isPressed', 'sendSysex', 'pauseMode', 'gatewayIP', 'setCursor', + 'getOemKey', 'tuneWrite', 'noDisplay', 'loadImage', 'switchPIN', + 'onRequest', 'onReceive', 'changePIN', 'playFile', 'noBuffer', + 'parseInt', 'overflow', 'checkPIN', 'knobRead', 'beginTFT', + 'bitClear', 'updateIR', 'bitWrite', 'position', 'writeRGB', + 'highByte', 'writeRed', 'setSpeed', 'readBlue', 'noStroke', + 'remoteIP', 'transfer', 'shutdown', 'hangCall', 'beginSMS', + 'endWrite', 'attached', 'maintain', 'noCursor', 'checkReg', + 'checkPUK', 'shiftOut', 'isValid', 'shiftIn', 'pulseIn', + 'connect', 'println', 'localIP', 'pinMode', 'getIMEI', + 'display', 'noBlink', 'process', 'getBand', 'running', 'beginSD', + 'drawBMP', 'lowByte', 'setBand', 'release', 'bitRead', 'prepare', + 'pointTo', 'readRed', 'setMode', 'noFill', 'remove', 'listen', + 'stroke', 'detach', 'attach', 'noTone', 'exists', 'buffer', + 'height', 'bitSet', 'circle', 'config', 'cursor', 'random', + 'IRread', 'sizeof', 'setDNS', 'endSMS', 'getKey', 'micros', + 'millis', 'begin', 'print', 'write', 'ready', 'flush', 'width', + 'isPIN', 'blink', 'clear', 'press', 'mkdir', 'rmdir', 'close', + 'point', 'yield', 'image', 'float', 'BSSID', 'click', 'delay', + 'read', 'text', 'move', 'peek', 'beep', 'rect', 'line', 'open', + 'seek', 'fill', 'size', 'turn', 'stop', 'home', 'find', 'char', + 'byte', 'step', 'word', 'long', 'tone', 'sqrt', 'RSSI', 'SSID', + 'end', 'bit', 'tan', 'cos', 'sin', 'pow', 'map', 'abs', 'max', + 'min', 'int', 'get', 'run', 'put')) + def get_tokens_unprocessed(self, text): for index, token, value in CppLexer.get_tokens_unprocessed(self, text): @@ -528,7 +530,7 @@ def get_tokens_unprocessed(self, text): elif token is Name.Function: if value in self.structure: yield index, Name.Other, value - else: + else: yield index, token, value elif token is Keyword: if value in self.storage:
pygments/lexers/_cocoa_builtins.py+4 −5 modified@@ -14,16 +14,15 @@ from __future__ import print_function -COCOA_INTERFACES = set(['UITableViewCell', 'HKCorrelationQuery', 'NSURLSessionDataTask', 'PHFetchOptions', 'NSLinguisticTagger', 'NSStream', 'AVAudioUnitDelay', 'GCMotion', 'SKPhysicsWorld', 'NSString', 'CMAttitude', 'AVAudioEnvironmentDistanceAttenuationParameters', 'HKStatisticsCollection', 'SCNPlane', 'CBPeer', 'JSContext', 'SCNTransaction', 'SCNTorus', 'AVAudioUnitEffect', 'UICollectionReusableView', 'MTLSamplerDescriptor', 'AVAssetReaderSampleReferenceOutput', 'AVMutableCompositionTrack', 'GKLeaderboard', 'NSFetchedResultsController', 'SKRange', 'MKTileOverlayRenderer', 'MIDINetworkSession', 'UIVisualEffectView', 'CIWarpKernel', 'PKObject', 'MKRoute', 'MPVolumeView', 'UIPrintInfo', 'SCNText', 'ADClient', 'UIKeyCommand', 'AVMutableAudioMix', 'GLKEffectPropertyLight', 'WKScriptMessage', 'AVMIDIPlayer', 'PHCollectionListChangeRequest', 'UICollectionViewLayout', 'NSMutableCharacterSet', 'SKPaymentTransaction', 'NEOnDemandRuleConnect', 'NSShadow', 'SCNView', 'NSURLSessionConfiguration', 'MTLVertexAttributeDescriptor', 'CBCharacteristic', 'HKQuantityType', 'CKLocationSortDescriptor', 'NEVPNIKEv2SecurityAssociationParameters', 'CMStepCounter', 'NSNetService', 'AVAssetWriterInputMetadataAdaptor', 'UICollectionView', 'UIViewPrintFormatter', 'SCNLevelOfDetail', 'CAShapeLayer', 'MCPeerID', 'MPRatingCommand', 'WKNavigation', 'NSDictionary', 'NSFileVersion', 'CMGyroData', 'AVAudioUnitDistortion', 'CKFetchRecordsOperation', 'SKPhysicsJointSpring', 'SCNHitTestResult', 'AVAudioTime', 'CIFilter', 'UIView', 'SCNConstraint', 'CAPropertyAnimation', 'MKMapItem', 'MPRemoteCommandCenter', 'UICollectionViewFlowLayoutInvalidationContext', 'UIInputViewController', 'PKPass', 'SCNPhysicsBehavior', 'MTLRenderPassColorAttachmentDescriptor', 'MKPolygonRenderer', 'CKNotification', 'JSValue', 'PHCollectionList', 'CLGeocoder', 'NSByteCountFormatter', 'AVCaptureScreenInput', 'MPFeedbackCommand', 'CAAnimation', 'MKOverlayPathView', 'UIActionSheet', 'UIMotionEffectGroup', 'NSLengthFormatter', 'UIBarItem', 'SKProduct', 'AVAssetExportSession', 'NSKeyedUnarchiver', 'NSMutableSet', 'SCNPyramid', 'PHAssetCollection', 'MKMapView', 'HMHomeManager', 'CATransition', 'MTLCompileOptions', 'UIVibrancyEffect', 'CLCircularRegion', 'MKTileOverlay', 'SCNShape', 'ACAccountCredential', 'SKPhysicsJointLimit', 'MKMapSnapshotter', 'AVMediaSelectionGroup', 'NSIndexSet', 'CBPeripheralManager', 'CKRecordZone', 'AVAudioRecorder', 'NSURL', 'CBCentral', 'NSNumber', 'AVAudioOutputNode', 'MTLVertexAttributeDescriptorArray', 'MKETAResponse', 'SKTransition', 'SSReadingList', 'HKSourceQuery', 'UITableViewRowAction', 'UITableView', 'SCNParticlePropertyController', 'AVCaptureStillImageOutput', 'GCController', 'AVAudioPlayerNode', 'AVAudioSessionPortDescription', 'NSHTTPURLResponse', 'NEOnDemandRuleEvaluateConnection', 'SKEffectNode', 'HKQuantity', 'GCControllerElement', 'AVPlayerItemAccessLogEvent', 'SCNBox', 'NSExtensionContext', 'MKOverlayRenderer', 'SCNPhysicsVehicle', 'NSDecimalNumber', 'EKReminder', 'MKPolylineView', 'CKQuery', 'AVAudioMixerNode', 'GKAchievementDescription', 'EKParticipant', 'NSBlockOperation', 'UIActivityItemProvider', 'CLLocation', 'NSBatchUpdateRequest', 'PHContentEditingOutput', 'PHObjectChangeDetails', 'MPMoviePlayerController', 'AVAudioFormat', 'HMTrigger', 'MTLRenderPassDepthAttachmentDescriptor', 'SCNRenderer', 'GKScore', 'UISplitViewController', 'HKSource', 'NSURLConnection', 'ABUnknownPersonViewController', 'SCNTechnique', 'UIMenuController', 'NSEvent', 'SKTextureAtlas', 'NSKeyedArchiver', 'GKLeaderboardSet', 'NSSimpleCString', 'AVAudioPCMBuffer', 'CBATTRequest', 'GKMatchRequest', 'AVMetadataObject', 'SKProductsRequest', 'UIAlertView', 'NSIncrementalStore', 'MFMailComposeViewController', 'SCNFloor', 'NSSortDescriptor', 'CKFetchNotificationChangesOperation', 'MPMovieAccessLog', 'NSManagedObjectContext', 'AVAudioUnitGenerator', 'WKBackForwardList', 'SKMutableTexture', 'AVCaptureAudioDataOutput', 'ACAccount', 'AVMetadataItem', 'MPRatingCommandEvent', 'AVCaptureDeviceInputSource', 'CLLocationManager', 'MPRemoteCommand', 'AVCaptureSession', 'UIStepper', 'UIRefreshControl', 'NEEvaluateConnectionRule', 'CKModifyRecordsOperation', 'UICollectionViewTransitionLayout', 'CBCentralManager', 'NSPurgeableData', 'SLComposeViewController', 'NSHashTable', 'MKUserTrackingBarButtonItem', 'UILexiconEntry', 'CMMotionActivity', 'SKAction', 'SKShader', 'AVPlayerItemOutput', 'MTLRenderPassAttachmentDescriptor', 'UIDocumentInteractionController', 'UIDynamicItemBehavior', 'NSMutableDictionary', 'UILabel', 'AVCaptureInputPort', 'NSExpression', 'CAInterAppAudioTransportView', 'SKMutablePayment', 'UIImage', 'PHCachingImageManager', 'SCNTransformConstraint', 'UIColor', 'SCNGeometrySource', 'AVCaptureAutoExposureBracketedStillImageSettings', 'UIPopoverBackgroundView', 'UIToolbar', 'NSNotificationCenter', 'AVAssetReaderOutputMetadataAdaptor', 'NSEntityMigrationPolicy', 'NSLocale', 'NSURLSession', 'SCNCamera', 'NSTimeZone', 'UIManagedDocument', 'AVMutableVideoCompositionLayerInstruction', 'AVAssetTrackGroup', 'NSInvocationOperation', 'ALAssetRepresentation', 'AVQueuePlayer', 'HMServiceGroup', 'UIPasteboard', 'PHContentEditingInput', 'NSLayoutManager', 'EKCalendarChooser', 'EKObject', 'CATiledLayer', 'GLKReflectionMapEffect', 'NSManagedObjectID', 'NSEnergyFormatter', 'SLRequest', 'HMCharacteristic', 'AVPlayerLayer', 'MTLRenderPassDescriptor', 'SKPayment', 'NSPointerArray', 'AVAudioMix', 'SCNLight', 'MCAdvertiserAssistant', 'MKMapSnapshotOptions', 'HKCategorySample', 'AVAudioEnvironmentReverbParameters', 'SCNMorpher', 'AVTimedMetadataGroup', 'CBMutableCharacteristic', 'NSFetchRequest', 'UIDevice', 'NSManagedObject', 'NKAssetDownload', 'AVOutputSettingsAssistant', 'SKPhysicsJointPin', 'UITabBar', 'UITextInputMode', 'NSFetchRequestExpression', 'HMActionSet', 'CTSubscriber', 'PHAssetChangeRequest', 'NSPersistentStoreRequest', 'UITabBarController', 'HKQuantitySample', 'AVPlayerItem', 'AVSynchronizedLayer', 'MKDirectionsRequest', 'NSMetadataItem', 'UIPresentationController', 'UINavigationItem', 'PHFetchResultChangeDetails', 'PHImageManager', 'AVCaptureManualExposureBracketedStillImageSettings', 'UIStoryboardPopoverSegue', 'SCNLookAtConstraint', 'UIGravityBehavior', 'UIWindow', 'CBMutableDescriptor', 'NEOnDemandRuleDisconnect', 'UIBezierPath', 'UINavigationController', 'ABPeoplePickerNavigationController', 'EKSource', 'AVAssetWriterInput', 'AVPlayerItemTrack', 'GLKEffectPropertyTexture', 'NSURLResponse', 'SKPaymentQueue', 'NSAssertionHandler', 'MKReverseGeocoder', 'GCControllerAxisInput', 'NSArray', 'NSOrthography', 'NSURLSessionUploadTask', 'NSCharacterSet', 'AVMutableVideoCompositionInstruction', 'AVAssetReaderOutput', 'EAGLContext', 'WKFrameInfo', 'CMPedometer', 'MyClass', 'CKModifyBadgeOperation', 'AVCaptureAudioFileOutput', 'SKEmitterNode', 'NSMachPort', 'AVVideoCompositionCoreAnimationTool', 'PHCollection', 'SCNPhysicsWorld', 'NSURLRequest', 'CMAccelerometerData', 'NSNetServiceBrowser', 'CLFloor', 'AVAsynchronousVideoCompositionRequest', 'SCNGeometry', 'SCNIKConstraint', 'CIKernel', 'CAGradientLayer', 'HKCharacteristicType', 'NSFormatter', 'SCNAction', 'CATransaction', 'CBUUID', 'UIStoryboard', 'MPMediaLibrary', 'UITapGestureRecognizer', 'MPMediaItemArtwork', 'NSURLSessionTask', 'AVAudioUnit', 'MCBrowserViewController', 'NSRelationshipDescription', 'HKSample', 'WKWebView', 'NSMutableAttributedString', 'NSPersistentStoreAsynchronousResult', 'MPNowPlayingInfoCenter', 'MKLocalSearch', 'EAAccessory', 'HKCorrelation', 'CATextLayer', 'NSNotificationQueue', 'UINib', 'GLKTextureLoader', 'HKObjectType', 'NSValue', 'NSMutableIndexSet', 'SKPhysicsContact', 'NSProgress', 'AVPlayerViewController', 'CAScrollLayer', 'GKSavedGame', 'NSTextCheckingResult', 'PHObjectPlaceholder', 'SKConstraint', 'EKEventEditViewController', 'NSEntityDescription', 'NSURLCredentialStorage', 'UIApplication', 'SKDownload', 'SCNNode', 'MKLocalSearchRequest', 'SKScene', 'UISearchDisplayController', 'NEOnDemandRule', 'MTLRenderPassStencilAttachmentDescriptor', 'CAReplicatorLayer', 'UIPrintPageRenderer', 'EKCalendarItem', 'NSUUID', 'EAAccessoryManager', 'NEOnDemandRuleIgnore', 'SKRegion', 'AVAssetResourceLoader', 'EAWiFiUnconfiguredAccessoryBrowser', 'NSUserActivity', 'CTCall', 'UIPrinterPickerController', 'CIVector', 'UINavigationBar', 'UIPanGestureRecognizer', 'MPMediaQuery', 'ABNewPersonViewController', 'CKRecordZoneID', 'HKAnchoredObjectQuery', 'CKFetchRecordZonesOperation', 'UIStoryboardSegue', 'ACAccountType', 'GKSession', 'SKVideoNode', 'PHChange', 'SKReceiptRefreshRequest', 'GCExtendedGamepadSnapshot', 'MPSeekCommandEvent', 'GCExtendedGamepad', 'CAValueFunction', 'SCNCylinder', 'NSNotification', 'NSBatchUpdateResult', 'PKPushCredentials', 'SCNPhysicsSliderJoint', 'AVCaptureDeviceFormat', 'AVPlayerItemErrorLog', 'NSMapTable', 'NSSet', 'CMMotionManager', 'GKVoiceChatService', 'UIPageControl', 'UILexicon', 'MTLArrayType', 'AVAudioUnitReverb', 'MKGeodesicPolyline', 'AVMutableComposition', 'NSLayoutConstraint', 'UIPrinter', 'NSOrderedSet', 'CBAttribute', 'PKPushPayload', 'NSIncrementalStoreNode', 'EKEventStore', 'MPRemoteCommandEvent', 'UISlider', 'UIBlurEffect', 'CKAsset', 'AVCaptureInput', 'AVAudioEngine', 'MTLVertexDescriptor', 'SKPhysicsBody', 'NSOperation', 'UIImageAsset', 'MKMapCamera', 'SKProductsResponse', 'GLKEffectPropertyMaterial', 'AVCaptureDevice', 'CTCallCenter', 'CABTMIDILocalPeripheralViewController', 'NEVPNManager', 'HKQuery', 'SCNPhysicsContact', 'CBMutableService', 'AVSampleBufferDisplayLayer', 'SCNSceneSource', 'SKLightNode', 'CKDiscoveredUserInfo', 'NSMutableArray', 'MTLDepthStencilDescriptor', 'MTLArgument', 'NSMassFormatter', 'CIRectangleFeature', 'PKPushRegistry', 'NEVPNConnection', 'MCNearbyServiceBrowser', 'NSOperationQueue', 'MKPolylineRenderer', 'UICollectionViewLayoutAttributes', 'NSValueTransformer', 'UICollectionViewFlowLayout', 'CIBarcodeFeature', 'MPChangePlaybackRateCommandEvent', 'NSEntityMapping', 'SKTexture', 'NSMergePolicy', 'UITextInputStringTokenizer', 'NSRecursiveLock', 'AVAsset', 'NSUndoManager', 'AVAudioUnitSampler', 'NSItemProvider', 'SKUniform', 'MPMediaPickerController', 'CKOperation', 'MTLRenderPipelineDescriptor', 'EAWiFiUnconfiguredAccessory', 'NSFileCoordinator', 'SKRequest', 'NSFileHandle', 'NSConditionLock', 'UISegmentedControl', 'NSManagedObjectModel', 'UITabBarItem', 'SCNCone', 'MPMediaItem', 'SCNMaterial', 'EKRecurrenceRule', 'UIEvent', 'UITouch', 'UIPrintInteractionController', 'CMDeviceMotion', 'NEVPNProtocol', 'NSCompoundPredicate', 'HKHealthStore', 'MKMultiPoint', 'HKSampleType', 'UIPrintFormatter', 'AVAudioUnitEQFilterParameters', 'SKView', 'NSConstantString', 'UIPopoverController', 'CKDatabase', 'AVMetadataFaceObject', 'UIAccelerometer', 'EKEventViewController', 'CMAltitudeData', 'MTLStencilDescriptor', 'UISwipeGestureRecognizer', 'NSPort', 'MKCircleRenderer', 'AVCompositionTrack', 'NSAsynchronousFetchRequest', 'NSUbiquitousKeyValueStore', 'NSMetadataQueryResultGroup', 'AVAssetResourceLoadingDataRequest', 'UITableViewHeaderFooterView', 'CKNotificationID', 'AVAudioSession', 'HKUnit', 'NSNull', 'NSPersistentStoreResult', 'MKCircleView', 'AVAudioChannelLayout', 'NEVPNProtocolIKEv2', 'WKProcessPool', 'UIAttachmentBehavior', 'CLBeacon', 'NSInputStream', 'NSURLCache', 'GKPlayer', 'NSMappingModel', 'NSHTTPCookie', 'AVMutableVideoComposition', 'PHFetchResult', 'NSAttributeDescription', 'AVPlayer', 'MKAnnotationView', 'UIFontDescriptor', 'NSTimer', 'CBDescriptor', 'MKOverlayView', 'AVAudioUnitTimePitch', 'NSSaveChangesRequest', 'UIReferenceLibraryViewController', 'SKPhysicsJointFixed', 'UILocalizedIndexedCollation', 'UIInterpolatingMotionEffect', 'UIDocumentPickerViewController', 'AVAssetWriter', 'NSBundle', 'SKStoreProductViewController', 'GLKViewController', 'NSMetadataQueryAttributeValueTuple', 'GKTurnBasedMatch', 'AVAudioFile', 'UIActivity', 'NSPipe', 'MKShape', 'NSMergeConflict', 'CIImage', 'HKObject', 'UIRotationGestureRecognizer', 'AVPlayerItemLegibleOutput', 'AVAssetImageGenerator', 'GCControllerButtonInput', 'CKMarkNotificationsReadOperation', 'CKSubscription', 'MPTimedMetadata', 'NKIssue', 'UIScreenMode', 'HMAccessoryBrowser', 'GKTurnBasedEventHandler', 'UIWebView', 'MKPolyline', 'JSVirtualMachine', 'AVAssetReader', 'NSAttributedString', 'GKMatchmakerViewController', 'NSCountedSet', 'UIButton', 'WKNavigationResponse', 'GKLocalPlayer', 'MPMovieErrorLog', 'AVSpeechUtterance', 'HKStatistics', 'UILocalNotification', 'HKBiologicalSexObject', 'AVURLAsset', 'CBPeripheral', 'NSDateComponentsFormatter', 'SKSpriteNode', 'UIAccessibilityElement', 'AVAssetWriterInputGroup', 'HMZone', 'AVAssetReaderAudioMixOutput', 'NSEnumerator', 'UIDocument', 'MKLocalSearchResponse', 'UISimpleTextPrintFormatter', 'PHPhotoLibrary', 'CBService', 'UIDocumentMenuViewController', 'MCSession', 'QLPreviewController', 'CAMediaTimingFunction', 'UITextPosition', 'ASIdentifierManager', 'AVAssetResourceLoadingRequest', 'SLComposeServiceViewController', 'UIPinchGestureRecognizer', 'PHObject', 'NSExtensionItem', 'HKSampleQuery', 'MTLRenderPipelineColorAttachmentDescriptorArray', 'MKRouteStep', 'SCNCapsule', 'NSMetadataQuery', 'AVAssetResourceLoadingContentInformationRequest', 'UITraitCollection', 'CTCarrier', 'NSFileSecurity', 'UIAcceleration', 'UIMotionEffect', 'MTLRenderPipelineReflection', 'CLHeading', 'CLVisit', 'MKDirectionsResponse', 'HMAccessory', 'MTLStructType', 'UITextView', 'CMMagnetometerData', 'UICollisionBehavior', 'UIProgressView', 'CKServerChangeToken', 'UISearchBar', 'MKPlacemark', 'AVCaptureConnection', 'NSPropertyMapping', 'ALAssetsFilter', 'SK3DNode', 'AVPlayerItemErrorLogEvent', 'NSJSONSerialization', 'AVAssetReaderVideoCompositionOutput', 'ABPersonViewController', 'CIDetector', 'GKTurnBasedMatchmakerViewController', 'MPMediaItemCollection', 'SCNSphere', 'NSCondition', 'NSURLCredential', 'MIDINetworkConnection', 'NSFileProviderExtension', 'NSDecimalNumberHandler', 'NSAtomicStoreCacheNode', 'NSAtomicStore', 'EKAlarm', 'CKNotificationInfo', 'AVAudioUnitEQ', 'UIPercentDrivenInteractiveTransition', 'MKPolygon', 'AVAssetTrackSegment', 'MTLVertexAttribute', 'NSExpressionDescription', 'HKStatisticsCollectionQuery', 'NSURLAuthenticationChallenge', 'NSDirectoryEnumerator', 'MKDistanceFormatter', 'UIAlertAction', 'NSPropertyListSerialization', 'GKPeerPickerController', 'UIUserNotificationSettings', 'UITableViewController', 'GKNotificationBanner', 'MKPointAnnotation', 'MTLRenderPassColorAttachmentDescriptorArray', 'NSCache', 'SKPhysicsJoint', 'NSXMLParser', 'UIViewController', 'MFMessageComposeViewController', 'AVAudioInputNode', 'NSDataDetector', 'CABTMIDICentralViewController', 'AVAudioUnitMIDIInstrument', 'AVCaptureVideoPreviewLayer', 'AVAssetWriterInputPassDescription', 'MPChangePlaybackRateCommand', 'NSURLComponents', 'CAMetalLayer', 'UISnapBehavior', 'AVMetadataMachineReadableCodeObject', 'CKDiscoverUserInfosOperation', 'NSTextAttachment', 'NSException', 'UIMenuItem', 'CMMotionActivityManager', 'SCNGeometryElement', 'NCWidgetController', 'CAEmitterLayer', 'MKUserLocation', 'UIImagePickerController', 'CIFeature', 'AVCaptureDeviceInput', 'ALAsset', 'NSURLSessionDownloadTask', 'SCNPhysicsHingeJoint', 'MPMoviePlayerViewController', 'NSMutableOrderedSet', 'SCNMaterialProperty', 'UIFont', 'AVCaptureVideoDataOutput', 'NSCachedURLResponse', 'ALAssetsLibrary', 'NSInvocation', 'UILongPressGestureRecognizer', 'NSTextStorage', 'WKWebViewConfiguration', 'CIFaceFeature', 'MKMapSnapshot', 'GLKEffectPropertyFog', 'AVComposition', 'CKDiscoverAllContactsOperation', 'AVAudioMixInputParameters', 'CAEmitterBehavior', 'PKPassLibrary', 'UIMutableUserNotificationCategory', 'NSLock', 'NEVPNProtocolIPSec', 'ADBannerView', 'UIDocumentPickerExtensionViewController', 'UIActivityIndicatorView', 'AVPlayerMediaSelectionCriteria', 'CALayer', 'UIAccessibilityCustomAction', 'UIBarButtonItem', 'AVAudioSessionRouteDescription', 'CLBeaconRegion', 'HKBloodTypeObject', 'MTLVertexBufferLayoutDescriptorArray', 'CABasicAnimation', 'AVVideoCompositionInstruction', 'AVMutableTimedMetadataGroup', 'EKRecurrenceEnd', 'NSTextContainer', 'TWTweetComposeViewController', 'UIScrollView', 'WKNavigationAction', 'AVPlayerItemMetadataOutput', 'EKRecurrenceDayOfWeek', 'NSNumberFormatter', 'MTLComputePipelineReflection', 'UIScreen', 'CLRegion', 'NSProcessInfo', 'GLKTextureInfo', 'SCNSkinner', 'AVCaptureMetadataOutput', 'SCNAnimationEvent', 'NSTextTab', 'JSManagedValue', 'NSDate', 'UITextChecker', 'WKBackForwardListItem', 'NSData', 'NSParagraphStyle', 'AVMutableMetadataItem', 'EKCalendar', 'NSMutableURLRequest', 'UIVideoEditorController', 'HMTimerTrigger', 'AVAudioUnitVarispeed', 'UIDynamicAnimator', 'AVCompositionTrackSegment', 'GCGamepadSnapshot', 'MPMediaEntity', 'GLKSkyboxEffect', 'UISwitch', 'EKStructuredLocation', 'UIGestureRecognizer', 'NSProxy', 'GLKBaseEffect', 'UIPushBehavior', 'GKScoreChallenge', 'NSCoder', 'MPMediaPlaylist', 'NSDateComponents', 'WKUserScript', 'EKEvent', 'NSDateFormatter', 'NSAsynchronousFetchResult', 'AVAssetWriterInputPixelBufferAdaptor', 'UIVisualEffect', 'UICollectionViewCell', 'UITextField', 'CLPlacemark', 'MPPlayableContentManager', 'AVCaptureOutput', 'HMCharacteristicWriteAction', 'CKModifySubscriptionsOperation', 'NSPropertyDescription', 'GCGamepad', 'UIMarkupTextPrintFormatter', 'SCNTube', 'NSPersistentStoreCoordinator', 'AVAudioEnvironmentNode', 'GKMatchmaker', 'CIContext', 'NSThread', 'SLComposeSheetConfigurationItem', 'SKPhysicsJointSliding', 'NSPredicate', 'GKVoiceChat', 'SKCropNode', 'AVCaptureAudioPreviewOutput', 'NSStringDrawingContext', 'GKGameCenterViewController', 'UIPrintPaper', 'SCNPhysicsBallSocketJoint', 'UICollectionViewLayoutInvalidationContext', 'GLKEffectPropertyTransform', 'AVAudioIONode', 'UIDatePicker', 'MKDirections', 'ALAssetsGroup', 'CKRecordZoneNotification', 'SCNScene', 'MPMovieAccessLogEvent', 'CKFetchSubscriptionsOperation', 'CAEmitterCell', 'AVAudioUnitTimeEffect', 'HMCharacteristicMetadata', 'MKPinAnnotationView', 'UIPickerView', 'UIImageView', 'UIUserNotificationCategory', 'SCNPhysicsVehicleWheel', 'HKCategoryType', 'MPMediaQuerySection', 'GKFriendRequestComposeViewController', 'NSError', 'MTLRenderPipelineColorAttachmentDescriptor', 'SCNPhysicsShape', 'UISearchController', 'SCNPhysicsBody', 'CTSubscriberInfo', 'AVPlayerItemAccessLog', 'MPMediaPropertyPredicate', 'CMLogItem', 'NSAutoreleasePool', 'NSSocketPort', 'AVAssetReaderTrackOutput', 'SKNode', 'UIMutableUserNotificationAction', 'SCNProgram', 'AVSpeechSynthesisVoice', 'CMAltimeter', 'AVCaptureAudioChannel', 'GKTurnBasedExchangeReply', 'AVVideoCompositionLayerInstruction', 'AVSpeechSynthesizer', 'GKChallengeEventHandler', 'AVCaptureFileOutput', 'UIControl', 'SCNPhysicsField', 'CKReference', 'LAContext', 'CKRecordID', 'ADInterstitialAd', 'AVAudioSessionDataSourceDescription', 'AVAudioBuffer', 'CIColorKernel', 'GCControllerDirectionPad', 'NSFileManager', 'AVMutableAudioMixInputParameters', 'UIScreenEdgePanGestureRecognizer', 'CAKeyframeAnimation', 'CKQueryNotification', 'PHAdjustmentData', 'EASession', 'AVAssetResourceRenewalRequest', 'UIInputView', 'NSFileWrapper', 'UIResponder', 'NSPointerFunctions', 'NSHTTPCookieStorage', 'AVMediaSelectionOption', 'NSRunLoop', 'NSFileAccessIntent', 'CAAnimationGroup', 'MKCircle', 'UIAlertController', 'NSMigrationManager', 'NSDateIntervalFormatter', 'UICollectionViewUpdateItem', 'CKDatabaseOperation', 'PHImageRequestOptions', 'SKReachConstraints', 'CKRecord', 'CAInterAppAudioSwitcherView', 'WKWindowFeatures', 'GKInvite', 'NSMutableData', 'PHAssetCollectionChangeRequest', 'NSMutableParagraphStyle', 'UIDynamicBehavior', 'GLKEffectProperty', 'CKFetchRecordChangesOperation', 'SKShapeNode', 'MPMovieErrorLogEvent', 'MKPolygonView', 'MPContentItem', 'HMAction', 'NSScanner', 'GKAchievementChallenge', 'AVAudioPlayer', 'CKContainer', 'AVVideoComposition', 'NKLibrary', 'NSPersistentStore', 'AVCaptureMovieFileOutput', 'HMRoom', 'GKChallenge', 'UITextRange', 'NSURLProtectionSpace', 'ACAccountStore', 'MPSkipIntervalCommand', 'NSComparisonPredicate', 'HMHome', 'PHVideoRequestOptions', 'NSOutputStream', 'MPSkipIntervalCommandEvent', 'PKAddPassesViewController', 'UITextSelectionRect', 'CTTelephonyNetworkInfo', 'AVTextStyleRule', 'NSFetchedPropertyDescription', 'UIPageViewController', 'CATransformLayer', 'UICollectionViewController', 'AVAudioNode', 'MCNearbyServiceAdvertiser', 'NSObject', 'PHAsset', 'GKLeaderboardViewController', 'CKQueryCursor', 'MPMusicPlayerController', 'MKOverlayPathRenderer', 'CMPedometerData', 'HMService', 'SKFieldNode', 'GKAchievement', 'WKUserContentController', 'AVAssetTrack', 'TWRequest', 'SKLabelNode', 'AVCaptureBracketedStillImageSettings', 'MIDINetworkHost', 'MPMediaPredicate', 'AVFrameRateRange', 'MTLTextureDescriptor', 'MTLVertexBufferLayoutDescriptor', 'MPFeedbackCommandEvent', 'UIUserNotificationAction', 'HKStatisticsQuery', 'SCNParticleSystem', 'NSIndexPath', 'AVVideoCompositionRenderContext', 'CADisplayLink', 'HKObserverQuery', 'UIPopoverPresentationController', 'CKQueryOperation', 'CAEAGLLayer', 'NSMutableString', 'NSMessagePort', 'NSURLQueryItem', 'MTLStructMember', 'AVAudioSessionChannelDescription', 'GLKView', 'UIActivityViewController', 'GKAchievementViewController', 'GKTurnBasedParticipant', 'NSURLProtocol', 'NSUserDefaults', 'NSCalendar', 'SKKeyframeSequence', 'AVMetadataItemFilter', 'CKModifyRecordZonesOperation', 'WKPreferences', 'NSMethodSignature', 'NSRegularExpression', 'EAGLSharegroup', 'AVPlayerItemVideoOutput', 'PHContentEditingInputRequestOptions', 'GKMatch', 'CIColor', 'UIDictationPhrase']) -COCOA_PROTOCOLS = set(['SKStoreProductViewControllerDelegate', 'AVVideoCompositionInstruction', 'AVAudioSessionDelegate', 'GKMatchDelegate', 'NSFileManagerDelegate', 'UILayoutSupport', 'NSCopying', 'UIPrintInteractionControllerDelegate', 'QLPreviewControllerDataSource', 'SKProductsRequestDelegate', 'NSTextStorageDelegate', 'MCBrowserViewControllerDelegate', 'MTLComputeCommandEncoder', 'SCNSceneExportDelegate', 'UISearchResultsUpdating', 'MFMailComposeViewControllerDelegate', 'MTLBlitCommandEncoder', 'NSDecimalNumberBehaviors', 'PHContentEditingController', 'NSMutableCopying', 'UIActionSheetDelegate', 'UIViewControllerTransitioningDelegate', 'UIAlertViewDelegate', 'AVAudioPlayerDelegate', 'MKReverseGeocoderDelegate', 'NSCoding', 'UITextInputTokenizer', 'GKFriendRequestComposeViewControllerDelegate', 'UIActivityItemSource', 'NSCacheDelegate', 'UIAdaptivePresentationControllerDelegate', 'GKAchievementViewControllerDelegate', 'UIViewControllerTransitionCoordinator', 'EKEventEditViewDelegate', 'NSURLConnectionDelegate', 'UITableViewDelegate', 'GKPeerPickerControllerDelegate', 'UIGuidedAccessRestrictionDelegate', 'AVSpeechSynthesizerDelegate', 'AVAudio3DMixing', 'AVPlayerItemLegibleOutputPushDelegate', 'ADInterstitialAdDelegate', 'HMAccessoryBrowserDelegate', 'AVAssetResourceLoaderDelegate', 'UITabBarControllerDelegate', 'CKRecordValue', 'SKPaymentTransactionObserver', 'AVCaptureAudioDataOutputSampleBufferDelegate', 'UIInputViewAudioFeedback', 'GKChallengeListener', 'SKSceneDelegate', 'UIPickerViewDelegate', 'UIWebViewDelegate', 'UIApplicationDelegate', 'GKInviteEventListener', 'MPMediaPlayback', 'MyClassJavaScriptMethods', 'AVAsynchronousKeyValueLoading', 'QLPreviewItem', 'SCNBoundingVolume', 'NSPortDelegate', 'UIContentContainer', 'SCNNodeRendererDelegate', 'SKRequestDelegate', 'SKPhysicsContactDelegate', 'HMAccessoryDelegate', 'UIPageViewControllerDataSource', 'SCNSceneRendererDelegate', 'SCNPhysicsContactDelegate', 'MKMapViewDelegate', 'AVPlayerItemOutputPushDelegate', 'UICollectionViewDelegate', 'UIImagePickerControllerDelegate', 'MTLRenderCommandEncoder', 'UIToolbarDelegate', 'WKUIDelegate', 'SCNActionable', 'NSURLConnectionDataDelegate', 'MKOverlay', 'CBCentralManagerDelegate', 'JSExport', 'NSTextLayoutOrientationProvider', 'UIPickerViewDataSource', 'PKPushRegistryDelegate', 'UIViewControllerTransitionCoordinatorContext', 'NSLayoutManagerDelegate', 'MTLLibrary', 'NSFetchedResultsControllerDelegate', 'ABPeoplePickerNavigationControllerDelegate', 'MTLResource', 'NSDiscardableContent', 'UITextFieldDelegate', 'MTLBuffer', 'MTLSamplerState', 'GKGameCenterControllerDelegate', 'MPMediaPickerControllerDelegate', 'UISplitViewControllerDelegate', 'UIAppearance', 'UIPickerViewAccessibilityDelegate', 'UITraitEnvironment', 'UIScrollViewAccessibilityDelegate', 'ADBannerViewDelegate', 'MPPlayableContentDataSource', 'MTLComputePipelineState', 'NSURLSessionDelegate', 'MTLCommandBuffer', 'NSXMLParserDelegate', 'UIViewControllerRestoration', 'UISearchBarDelegate', 'UIBarPositioning', 'CBPeripheralDelegate', 'UISearchDisplayDelegate', 'CAAction', 'PKAddPassesViewControllerDelegate', 'MCNearbyServiceAdvertiserDelegate', 'MTLDepthStencilState', 'GKTurnBasedMatchmakerViewControllerDelegate', 'MPPlayableContentDelegate', 'AVCaptureVideoDataOutputSampleBufferDelegate', 'UIAppearanceContainer', 'UIStateRestoring', 'UITextDocumentProxy', 'MTLDrawable', 'NSURLSessionTaskDelegate', 'NSFilePresenter', 'AVAudioStereoMixing', 'UIViewControllerContextTransitioning', 'UITextInput', 'CBPeripheralManagerDelegate', 'UITextInputDelegate', 'NSFastEnumeration', 'NSURLAuthenticationChallengeSender', 'SCNProgramDelegate', 'AVVideoCompositing', 'SCNAnimatable', 'NSSecureCoding', 'MCAdvertiserAssistantDelegate', 'GKLocalPlayerListener', 'GLKNamedEffect', 'UIPopoverControllerDelegate', 'AVCaptureMetadataOutputObjectsDelegate', 'NSExtensionRequestHandling', 'UITextSelecting', 'UIPrinterPickerControllerDelegate', 'NCWidgetProviding', 'MTLCommandEncoder', 'NSURLProtocolClient', 'MFMessageComposeViewControllerDelegate', 'UIVideoEditorControllerDelegate', 'WKNavigationDelegate', 'GKSavedGameListener', 'UITableViewDataSource', 'MTLFunction', 'EKCalendarChooserDelegate', 'NSUserActivityDelegate', 'UICollisionBehaviorDelegate', 'NSStreamDelegate', 'MCNearbyServiceBrowserDelegate', 'HMHomeDelegate', 'UINavigationControllerDelegate', 'MCSessionDelegate', 'UIDocumentPickerDelegate', 'UIViewControllerInteractiveTransitioning', 'GKTurnBasedEventListener', 'SCNSceneRenderer', 'MTLTexture', 'GLKViewDelegate', 'EAAccessoryDelegate', 'WKScriptMessageHandler', 'PHPhotoLibraryChangeObserver', 'NSKeyedUnarchiverDelegate', 'AVPlayerItemMetadataOutputPushDelegate', 'NSMachPortDelegate', 'SCNShadable', 'UIPopoverBackgroundViewMethods', 'UIDocumentMenuDelegate', 'UIBarPositioningDelegate', 'ABPersonViewControllerDelegate', 'NSNetServiceBrowserDelegate', 'EKEventViewDelegate', 'UIScrollViewDelegate', 'NSURLConnectionDownloadDelegate', 'UIGestureRecognizerDelegate', 'UINavigationBarDelegate', 'AVAudioMixing', 'NSFetchedResultsSectionInfo', 'UIDocumentInteractionControllerDelegate', 'MTLParallelRenderCommandEncoder', 'QLPreviewControllerDelegate', 'UIAccessibilityReadingContent', 'ABUnknownPersonViewControllerDelegate', 'GLKViewControllerDelegate', 'UICollectionViewDelegateFlowLayout', 'UIPopoverPresentationControllerDelegate', 'UIDynamicAnimatorDelegate', 'NSTextAttachmentContainer', 'MKAnnotation', 'UIAccessibilityIdentification', 'UICoordinateSpace', 'ABNewPersonViewControllerDelegate', 'MTLDevice', 'CAMediaTiming', 'AVCaptureFileOutputRecordingDelegate', 'HMHomeManagerDelegate', 'UITextViewDelegate', 'UITabBarDelegate', 'GKLeaderboardViewControllerDelegate', 'UISearchControllerDelegate', 'EAWiFiUnconfiguredAccessoryBrowserDelegate', 'UITextInputTraits', 'MTLRenderPipelineState', 'GKVoiceChatClient', 'UIKeyInput', 'UICollectionViewDataSource', 'SCNTechniqueSupport', 'NSLocking', 'AVCaptureFileOutputDelegate', 'GKChallengeEventHandlerDelegate', 'UIObjectRestoration', 'CIFilterConstructor', 'AVPlayerItemOutputPullDelegate', 'EAGLDrawable', 'AVVideoCompositionValidationHandling', 'UIViewControllerAnimatedTransitioning', 'NSURLSessionDownloadDelegate', 'UIAccelerometerDelegate', 'UIPageViewControllerDelegate', 'MTLCommandQueue', 'UIDataSourceModelAssociation', 'AVAudioRecorderDelegate', 'GKSessionDelegate', 'NSKeyedArchiverDelegate', 'CAMetalDrawable', 'UIDynamicItem', 'CLLocationManagerDelegate', 'NSMetadataQueryDelegate', 'NSNetServiceDelegate', 'GKMatchmakerViewControllerDelegate', 'NSURLSessionDataDelegate']) -COCOA_PRIMITIVES = set(['ROTAHeader', '__CFBundle', 'MortSubtable', 'AudioFilePacketTableInfo', 'CGPDFOperatorTable', 'KerxStateEntry', 'ExtendedTempoEvent', 'CTParagraphStyleSetting', 'OpaqueMIDIPort', '_GLKMatrix3', '_GLKMatrix2', '_GLKMatrix4', 'ExtendedControlEvent', 'CAFAudioDescription', 'OpaqueCMBlockBuffer', 'CGTextDrawingMode', 'EKErrorCode', 'GCAcceleration', 'AudioUnitParameterInfo', '__SCPreferences', '__CTFrame', '__CTLine', 'AudioFile_SMPTE_Time', 'gss_krb5_lucid_context_v1', 'OpaqueJSValue', 'TrakTableEntry', 'AudioFramePacketTranslation', 'CGImageSource', 'OpaqueJSPropertyNameAccumulator', 'JustPCGlyphRepeatAddAction', '__CFBinaryHeap', 'OpaqueMIDIThruConnection', 'opaqueCMBufferQueue', 'OpaqueMusicSequence', 'MortRearrangementSubtable', 'MixerDistanceParams', 'MorxSubtable', 'MIDIObjectPropertyChangeNotification', 'SFNTLookupSegment', 'CGImageMetadataErrors', 'CGPath', 'OpaqueMIDIEndpoint', 'AudioComponentPlugInInterface', 'gss_ctx_id_t_desc_struct', 'sfntFontFeatureSetting', 'OpaqueJSContextGroup', '__SCNetworkConnection', 'AudioUnitParameterValueTranslation', 'CGImageMetadataType', 'CGPattern', 'AudioFileTypeAndFormatID', 'CGContext', 'AUNodeInteraction', 'SFNTLookupTable', 'JustPCDecompositionAction', 'KerxControlPointHeader', 'AudioStreamPacketDescription', 'KernSubtableHeader', '__SecCertificate', 'AUMIDIOutputCallbackStruct', 'MIDIMetaEvent', 'AudioQueueChannelAssignment', 'AnchorPoint', 'JustTable', '__CFNetService', 'CF_BRIDGED_TYPE', 'gss_krb5_lucid_key', 'CGPDFDictionary', 'KerxSubtableHeader', 'CAF_UUID_ChunkHeader', 'gss_krb5_cfx_keydata', 'OpaqueJSClass', 'CGGradient', 'OpaqueMIDISetup', 'JustPostcompTable', '__CTParagraphStyle', 'AudioUnitParameterHistoryInfo', 'OpaqueJSContext', 'CGShading', 'MIDIThruConnectionParams', 'BslnFormat0Part', 'SFNTLookupSingle', '__CFHost', '__SecRandom', '__CTFontDescriptor', '_NSRange', 'sfntDirectory', 'AudioQueueLevelMeterState', 'CAFPositionPeak', 'PropLookupSegment', '__CVOpenGLESTextureCache', 'sfntInstance', '_GLKQuaternion', 'AnkrTable', '__SCNetworkProtocol', 'gss_buffer_desc_struct', 'CAFFileHeader', 'KerxOrderedListHeader', 'CGBlendMode', 'STXEntryOne', 'CAFRegion', 'SFNTLookupTrimmedArrayHeader', 'SCNMatrix4', 'KerxControlPointEntry', 'OpaqueMusicTrack', '_GLKVector4', 'gss_OID_set_desc_struct', 'OpaqueMusicPlayer', '_CFHTTPAuthentication', 'CGAffineTransform', 'CAFMarkerChunk', 'AUHostIdentifier', 'ROTAGlyphEntry', 'BslnTable', 'gss_krb5_lucid_context_version', '_GLKMatrixStack', 'CGImage', 'KernStateEntry', 'SFNTLookupSingleHeader', 'MortLigatureSubtable', 'CAFUMIDChunk', 'SMPTETime', 'CAFDataChunk', 'CGPDFStream', 'AudioFileRegionList', 'STEntryTwo', 'SFNTLookupBinarySearchHeader', 'OpbdTable', '__CTGlyphInfo', 'BslnFormat2Part', 'KerxIndexArrayHeader', 'TrakTable', 'KerxKerningPair', '__CFBitVector', 'KernVersion0SubtableHeader', 'OpaqueAudioComponentInstance', 'AudioChannelLayout', '__CFUUID', 'MIDISysexSendRequest', '__CFNumberFormatter', 'CGImageSourceStatus', 'AudioFileMarkerList', 'AUSamplerBankPresetData', 'CGDataProvider', 'AudioFormatInfo', '__SecIdentity', 'sfntCMapExtendedSubHeader', 'MIDIChannelMessage', 'KernOffsetTable', 'CGColorSpaceModel', 'MFMailComposeErrorCode', 'CGFunction', '__SecTrust', 'AVAudio3DAngularOrientation', 'CGFontPostScriptFormat', 'KernStateHeader', 'AudioUnitCocoaViewInfo', 'CGDataConsumer', 'OpaqueMIDIDevice', 'KernVersion0Header', 'AnchorPointTable', 'CGImageDestination', 'CAFInstrumentChunk', 'AudioUnitMeterClipping', 'MorxChain', '__CTFontCollection', 'STEntryOne', 'STXEntryTwo', 'ExtendedNoteOnEvent', 'CGColorRenderingIntent', 'KerxSimpleArrayHeader', 'MorxTable', '_GLKVector3', '_GLKVector2', 'MortTable', 'CGPDFBox', 'AudioUnitParameterValueFromString', '__CFSocket', 'ALCdevice_struct', 'MIDINoteMessage', 'sfntFeatureHeader', 'CGRect', '__SCNetworkInterface', '__CFTree', 'MusicEventUserData', 'TrakTableData', 'GCQuaternion', 'MortContextualSubtable', '__CTRun', 'AudioUnitFrequencyResponseBin', 'MortChain', 'MorxInsertionSubtable', 'CGImageMetadata', 'gss_auth_identity', 'AudioUnitMIDIControlMapping', 'CAFChunkHeader', 'CGImagePropertyOrientation', 'CGPDFScanner', 'OpaqueMusicEventIterator', 'sfntDescriptorHeader', 'AudioUnitNodeConnection', 'OpaqueMIDIDeviceList', 'ExtendedAudioFormatInfo', 'BslnFormat1Part', 'sfntFontDescriptor', 'KernSimpleArrayHeader', '__CFRunLoopObserver', 'CGPatternTiling', 'MIDINotification', 'MorxLigatureSubtable', 'MessageComposeResult', 'MIDIThruConnectionEndpoint', 'MusicDeviceStdNoteParams', 'opaqueCMSimpleQueue', 'ALCcontext_struct', 'OpaqueAudioQueue', 'PropLookupSingle', 'CGInterpolationQuality', 'CGColor', 'AudioOutputUnitStartAtTimeParams', 'gss_name_t_desc_struct', 'CGFunctionCallbacks', 'CAFPacketTableHeader', 'AudioChannelDescription', 'sfntFeatureName', 'MorxContextualSubtable', 'CVSMPTETime', 'AudioValueRange', 'CGTextEncoding', 'AudioStreamBasicDescription', 'AUNodeRenderCallback', 'AudioPanningInfo', 'KerxOrderedListEntry', '__CFAllocator', 'OpaqueJSPropertyNameArray', '__SCDynamicStore', 'OpaqueMIDIEntity', '__CTRubyAnnotation', 'SCNVector4', 'CFHostClientContext', 'CFNetServiceClientContext', 'AudioUnitPresetMAS_SettingData', 'opaqueCMBufferQueueTriggerToken', 'AudioUnitProperty', 'CAFRegionChunk', 'CGPDFString', '__GLsync', '__CFStringTokenizer', 'JustWidthDeltaEntry', 'sfntVariationAxis', '__CFNetDiagnostic', 'CAFOverviewSample', 'sfntCMapEncoding', 'CGVector', '__SCNetworkService', 'opaqueCMSampleBuffer', 'AUHostVersionIdentifier', 'AudioBalanceFade', 'sfntFontRunFeature', 'KerxCoordinateAction', 'sfntCMapSubHeader', 'CVPlanarPixelBufferInfo', 'AUNumVersion', 'AUSamplerInstrumentData', 'AUPreset', '__CTRunDelegate', 'OpaqueAudioQueueProcessingTap', 'KerxTableHeader', '_NSZone', 'OpaqueExtAudioFile', '__CFRunLoopSource', '__CVMetalTextureCache', 'KerxAnchorPointAction', 'OpaqueJSString', 'AudioQueueParameterEvent', '__CFHTTPMessage', 'OpaqueCMClock', 'ScheduledAudioFileRegion', 'STEntryZero', 'AVAudio3DPoint', 'gss_channel_bindings_struct', 'sfntVariationHeader', 'AUChannelInfo', 'UIOffset', 'GLKEffectPropertyPrv', 'KerxStateHeader', 'CGLineJoin', 'CGPDFDocument', '__CFBag', 'KernOrderedListHeader', '__SCNetworkSet', '__SecKey', 'MIDIObjectAddRemoveNotification', 'AudioUnitParameter', 'JustPCActionSubrecord', 'AudioComponentDescription', 'AudioUnitParameterValueName', 'AudioUnitParameterEvent', 'KerxControlPointAction', 'AudioTimeStamp', 'KernKerningPair', 'gss_buffer_set_desc_struct', 'MortFeatureEntry', 'FontVariation', 'CAFStringID', 'LcarCaretClassEntry', 'AudioUnitParameterStringFromValue', 'ACErrorCode', 'ALMXGlyphEntry', 'LtagTable', '__CTTypesetter', 'AuthorizationOpaqueRef', 'UIEdgeInsets', 'CGPathElement', 'CAFMarker', 'KernTableHeader', 'NoteParamsControlValue', 'SSLContext', 'gss_cred_id_t_desc_struct', 'AudioUnitParameterNameInfo', 'CGDataConsumerCallbacks', 'ALMXHeader', 'CGLineCap', 'MIDIControlTransform', 'CGPDFArray', '__SecPolicy', 'AudioConverterPrimeInfo', '__CTTextTab', '__CFNetServiceMonitor', 'AUInputSamplesInOutputCallbackStruct', '__CTFramesetter', 'CGPDFDataFormat', 'STHeader', 'CVPlanarPixelBufferInfo_YCbCrPlanar', 'MIDIValueMap', 'JustDirectionTable', '__SCBondStatus', 'SFNTLookupSegmentHeader', 'OpaqueCMMemoryPool', 'CGPathDrawingMode', 'CGFont', '__SCNetworkReachability', 'AudioClassDescription', 'CGPoint', 'AVAudio3DVectorOrientation', 'CAFStrings', '__CFNetServiceBrowser', 'opaqueMTAudioProcessingTap', 'sfntNameRecord', 'CGPDFPage', 'CGLayer', 'ComponentInstanceRecord', 'CAFInfoStrings', 'HostCallbackInfo', 'MusicDeviceNoteParams', 'OpaqueVTCompressionSession', 'KernIndexArrayHeader', 'CVPlanarPixelBufferInfo_YCbCrBiPlanar', 'MusicTrackLoopInfo', 'opaqueCMFormatDescription', 'STClassTable', 'sfntDirectoryEntry', 'OpaqueCMTimebase', 'CGDataProviderDirectCallbacks', 'MIDIPacketList', 'CAFOverviewChunk', 'MIDIPacket', 'ScheduledAudioSlice', 'CGDataProviderSequentialCallbacks', 'AudioBuffer', 'MorxRearrangementSubtable', 'CGPatternCallbacks', 'AUDistanceAttenuationData', 'MIDIIOErrorNotification', 'CGPDFContentStream', 'IUnknownVTbl', 'MIDITransform', 'MortInsertionSubtable', 'CABarBeatTime', 'AudioBufferList', '__CVBuffer', 'AURenderCallbackStruct', 'STXEntryZero', 'JustPCDuctilityAction', 'OpaqueAudioQueueTimeline', 'VTDecompressionOutputCallbackRecord', 'OpaqueMIDIClient', '__CFPlugInInstance', 'AudioQueueBuffer', '__CFFileDescriptor', 'AudioUnitConnection', '_GKTurnBasedExchangeStatus', 'LcarCaretTable', 'CVPlanarComponentInfo', 'JustWidthDeltaGroup', 'OpaqueAudioComponent', 'ParameterEvent', '__CVPixelBufferPool', '__CTFont', 'CGColorSpace', 'CGSize', 'AUDependentParameter', 'MIDIDriverInterface', 'gss_krb5_rfc1964_keydata', '__CFDateFormatter', 'LtagStringRange', 'OpaqueVTDecompressionSession', 'gss_iov_buffer_desc_struct', 'AUPresetEvent', 'PropTable', 'KernOrderedListEntry', 'CF_BRIDGED_MUTABLE_TYPE', 'gss_OID_desc_struct', 'AudioUnitPresetMAS_Settings', 'AudioFileMarker', 'JustPCConditionalAddAction', 'BslnFormat3Part', '__CFNotificationCenter', 'MortSwashSubtable', 'AUParameterMIDIMapping', 'SCNVector3', 'OpaqueAudioConverter', 'MIDIRawData', 'sfntNameHeader', '__CFRunLoop', 'MFMailComposeResult', 'CATransform3D', 'OpbdSideValues', 'CAF_SMPTE_Time', '__SecAccessControl', 'JustPCAction', 'OpaqueVTFrameSilo', 'OpaqueVTMultiPassStorage', 'CGPathElementType', 'AudioFormatListItem', 'AudioUnitExternalBuffer', 'AudioFileRegion', 'AudioValueTranslation', 'CGImageMetadataTag', 'CAFPeakChunk', 'AudioBytePacketTranslation', 'sfntCMapHeader', '__CFURLEnumerator', 'STXHeader', 'CGPDFObjectType', 'SFNTLookupArrayHeader']) - +COCOA_INTERFACES = set(['UITableViewCell', 'HKCorrelationQuery', 'NSURLSessionDataTask', 'PHFetchOptions', 'NSLinguisticTagger', 'NSStream', 'AVAudioUnitDelay', 'GCMotion', 'SKPhysicsWorld', 'NSString', 'CMAttitude', 'AVAudioEnvironmentDistanceAttenuationParameters', 'HKStatisticsCollection', 'SCNPlane', 'CBPeer', 'JSContext', 'SCNTransaction', 'SCNTorus', 'AVAudioUnitEffect', 'UICollectionReusableView', 'MTLSamplerDescriptor', 'AVAssetReaderSampleReferenceOutput', 'AVMutableCompositionTrack', 'GKLeaderboard', 'NSFetchedResultsController', 'SKRange', 'MKTileOverlayRenderer', 'MIDINetworkSession', 'UIVisualEffectView', 'CIWarpKernel', 'PKObject', 'MKRoute', 'MPVolumeView', 'UIPrintInfo', 'SCNText', 'ADClient', 'PKPayment', 'AVMutableAudioMix', 'GLKEffectPropertyLight', 'WKScriptMessage', 'AVMIDIPlayer', 'PHCollectionListChangeRequest', 'UICollectionViewLayout', 'NSMutableCharacterSet', 'SKPaymentTransaction', 'NEOnDemandRuleConnect', 'NSShadow', 'SCNView', 'NSURLSessionConfiguration', 'MTLVertexAttributeDescriptor', 'CBCharacteristic', 'HKQuantityType', 'CKLocationSortDescriptor', 'NEVPNIKEv2SecurityAssociationParameters', 'CMStepCounter', 'NSNetService', 'AVAssetWriterInputMetadataAdaptor', 'UICollectionView', 'UIViewPrintFormatter', 'SCNLevelOfDetail', 'CAShapeLayer', 'MCPeerID', 'MPRatingCommand', 'WKNavigation', 'NSDictionary', 'NSFileVersion', 'CMGyroData', 'AVAudioUnitDistortion', 'CKFetchRecordsOperation', 'SKPhysicsJointSpring', 'SCNHitTestResult', 'AVAudioTime', 'CIFilter', 'UIView', 'SCNConstraint', 'CAPropertyAnimation', 'MKMapItem', 'MPRemoteCommandCenter', 'PKPaymentSummaryItem', 'UICollectionViewFlowLayoutInvalidationContext', 'UIInputViewController', 'PKPass', 'SCNPhysicsBehavior', 'MTLRenderPassColorAttachmentDescriptor', 'MKPolygonRenderer', 'CKNotification', 'JSValue', 'PHCollectionList', 'CLGeocoder', 'NSByteCountFormatter', 'AVCaptureScreenInput', 'MPFeedbackCommand', 'CAAnimation', 'MKOverlayPathView', 'UIActionSheet', 'UIMotionEffectGroup', 'NSLengthFormatter', 'UIBarItem', 'SKProduct', 'AVAssetExportSession', 'NSKeyedUnarchiver', 'NSMutableSet', 'SCNPyramid', 'PHAssetCollection', 'MKMapView', 'HMHomeManager', 'CATransition', 'MTLCompileOptions', 'UIVibrancyEffect', 'CLCircularRegion', 'MKTileOverlay', 'SCNShape', 'ACAccountCredential', 'SKPhysicsJointLimit', 'MKMapSnapshotter', 'AVMediaSelectionGroup', 'NSIndexSet', 'CBPeripheralManager', 'CKRecordZone', 'AVAudioRecorder', 'NSURL', 'CBCentral', 'NSNumber', 'AVAudioOutputNode', 'MTLVertexAttributeDescriptorArray', 'MKETAResponse', 'SKTransition', 'SSReadingList', 'HKSourceQuery', 'UITableViewRowAction', 'UITableView', 'SCNParticlePropertyController', 'AVCaptureStillImageOutput', 'GCController', 'AVAudioPlayerNode', 'AVAudioSessionPortDescription', 'NSHTTPURLResponse', 'NEOnDemandRuleEvaluateConnection', 'SKEffectNode', 'HKQuantity', 'GCControllerElement', 'AVPlayerItemAccessLogEvent', 'SCNBox', 'NSExtensionContext', 'MKOverlayRenderer', 'SCNPhysicsVehicle', 'NSDecimalNumber', 'EKReminder', 'MKPolylineView', 'CKQuery', 'AVAudioMixerNode', 'GKAchievementDescription', 'EKParticipant', 'NSBlockOperation', 'UIActivityItemProvider', 'CLLocation', 'NSBatchUpdateRequest', 'PHContentEditingOutput', 'PHObjectChangeDetails', 'HKWorkoutType', 'MPMoviePlayerController', 'AVAudioFormat', 'HMTrigger', 'MTLRenderPassDepthAttachmentDescriptor', 'SCNRenderer', 'GKScore', 'UISplitViewController', 'HKSource', 'NSURLConnection', 'ABUnknownPersonViewController', 'SCNTechnique', 'UIMenuController', 'NSEvent', 'SKTextureAtlas', 'NSKeyedArchiver', 'GKLeaderboardSet', 'NSSimpleCString', 'AVAudioPCMBuffer', 'CBATTRequest', 'GKMatchRequest', 'AVMetadataObject', 'SKProductsRequest', 'UIAlertView', 'NSIncrementalStore', 'MFMailComposeViewController', 'SCNFloor', 'NSSortDescriptor', 'CKFetchNotificationChangesOperation', 'MPMovieAccessLog', 'NSManagedObjectContext', 'AVAudioUnitGenerator', 'WKBackForwardList', 'SKMutableTexture', 'AVCaptureAudioDataOutput', 'ACAccount', 'AVMetadataItem', 'MPRatingCommandEvent', 'AVCaptureDeviceInputSource', 'CLLocationManager', 'MPRemoteCommand', 'AVCaptureSession', 'UIStepper', 'UIRefreshControl', 'NEEvaluateConnectionRule', 'CKModifyRecordsOperation', 'UICollectionViewTransitionLayout', 'CBCentralManager', 'NSPurgeableData', 'PKShippingMethod', 'SLComposeViewController', 'NSHashTable', 'MKUserTrackingBarButtonItem', 'UILexiconEntry', 'CMMotionActivity', 'SKAction', 'SKShader', 'AVPlayerItemOutput', 'MTLRenderPassAttachmentDescriptor', 'UIDocumentInteractionController', 'UIDynamicItemBehavior', 'NSMutableDictionary', 'UILabel', 'AVCaptureInputPort', 'NSExpression', 'CAInterAppAudioTransportView', 'SKMutablePayment', 'UIImage', 'PHCachingImageManager', 'SCNTransformConstraint', 'HKCorrelationType', 'UIColor', 'SCNGeometrySource', 'AVCaptureAutoExposureBracketedStillImageSettings', 'UIPopoverBackgroundView', 'UIToolbar', 'NSNotificationCenter', 'UICollectionViewLayoutAttributes', 'AVAssetReaderOutputMetadataAdaptor', 'NSEntityMigrationPolicy', 'HMUser', 'NSLocale', 'NSURLSession', 'SCNCamera', 'NSTimeZone', 'UIManagedDocument', 'AVMutableVideoCompositionLayerInstruction', 'AVAssetTrackGroup', 'NSInvocationOperation', 'ALAssetRepresentation', 'AVQueuePlayer', 'HMServiceGroup', 'UIPasteboard', 'PHContentEditingInput', 'NSLayoutManager', 'EKCalendarChooser', 'EKObject', 'CATiledLayer', 'GLKReflectionMapEffect', 'NSManagedObjectID', 'NSEnergyFormatter', 'SLRequest', 'HMCharacteristic', 'AVPlayerLayer', 'MTLRenderPassDescriptor', 'SKPayment', 'NSPointerArray', 'AVAudioMix', 'SCNLight', 'MCAdvertiserAssistant', 'MKMapSnapshotOptions', 'HKCategorySample', 'AVAudioEnvironmentReverbParameters', 'SCNMorpher', 'AVTimedMetadataGroup', 'CBMutableCharacteristic', 'NSFetchRequest', 'UIDevice', 'NSManagedObject', 'NKAssetDownload', 'AVOutputSettingsAssistant', 'SKPhysicsJointPin', 'UITabBar', 'UITextInputMode', 'NSFetchRequestExpression', 'HMActionSet', 'CTSubscriber', 'PHAssetChangeRequest', 'NSPersistentStoreRequest', 'UITabBarController', 'HKQuantitySample', 'AVPlayerItem', 'AVSynchronizedLayer', 'MKDirectionsRequest', 'NSMetadataItem', 'UIPresentationController', 'UINavigationItem', 'PHFetchResultChangeDetails', 'PHImageManager', 'AVCaptureManualExposureBracketedStillImageSettings', 'UIStoryboardPopoverSegue', 'SCNLookAtConstraint', 'UIGravityBehavior', 'UIWindow', 'CBMutableDescriptor', 'NEOnDemandRuleDisconnect', 'UIBezierPath', 'UINavigationController', 'ABPeoplePickerNavigationController', 'EKSource', 'AVAssetWriterInput', 'AVPlayerItemTrack', 'GLKEffectPropertyTexture', 'NSHTTPCookie', 'NSURLResponse', 'SKPaymentQueue', 'NSAssertionHandler', 'MKReverseGeocoder', 'GCControllerAxisInput', 'NSArray', 'NSOrthography', 'NSURLSessionUploadTask', 'NSCharacterSet', 'AVMutableVideoCompositionInstruction', 'AVAssetReaderOutput', 'EAGLContext', 'WKFrameInfo', 'CMPedometer', 'MyClass', 'CKModifyBadgeOperation', 'AVCaptureAudioFileOutput', 'SKEmitterNode', 'NSMachPort', 'AVVideoCompositionCoreAnimationTool', 'PHCollection', 'SCNPhysicsWorld', 'NSURLRequest', 'CMAccelerometerData', 'NSNetServiceBrowser', 'CLFloor', 'AVAsynchronousVideoCompositionRequest', 'SCNGeometry', 'SCNIKConstraint', 'CIKernel', 'CAGradientLayer', 'HKCharacteristicType', 'NSFormatter', 'SCNAction', 'CATransaction', 'CBUUID', 'UIStoryboard', 'MPMediaLibrary', 'UITapGestureRecognizer', 'MPMediaItemArtwork', 'NSURLSessionTask', 'AVAudioUnit', 'MCBrowserViewController', 'UIFontDescriptor', 'NSRelationshipDescription', 'HKSample', 'WKWebView', 'NSMutableAttributedString', 'NSPersistentStoreAsynchronousResult', 'MPNowPlayingInfoCenter', 'MKLocalSearch', 'EAAccessory', 'HKCorrelation', 'CATextLayer', 'NSNotificationQueue', 'UINib', 'GLKTextureLoader', 'HKObjectType', 'NSValue', 'NSMutableIndexSet', 'SKPhysicsContact', 'NSProgress', 'AVPlayerViewController', 'CAScrollLayer', 'GKSavedGame', 'NSTextCheckingResult', 'PHObjectPlaceholder', 'SKConstraint', 'EKEventEditViewController', 'NSEntityDescription', 'NSURLCredentialStorage', 'UIApplication', 'SKDownload', 'SCNNode', 'MKLocalSearchRequest', 'SKScene', 'UISearchDisplayController', 'NEOnDemandRule', 'MTLRenderPassStencilAttachmentDescriptor', 'CAReplicatorLayer', 'UIPrintPageRenderer', 'EKCalendarItem', 'NSUUID', 'EAAccessoryManager', 'NEOnDemandRuleIgnore', 'SKRegion', 'AVAssetResourceLoader', 'EAWiFiUnconfiguredAccessoryBrowser', 'NSUserActivity', 'CTCall', 'UIPrinterPickerController', 'CIVector', 'UINavigationBar', 'UIPanGestureRecognizer', 'MPMediaQuery', 'ABNewPersonViewController', 'CKRecordZoneID', 'HKAnchoredObjectQuery', 'CKFetchRecordZonesOperation', 'UIStoryboardSegue', 'ACAccountType', 'GKSession', 'SKVideoNode', 'PHChange', 'SKReceiptRefreshRequest', 'GCExtendedGamepadSnapshot', 'MPSeekCommandEvent', 'GCExtendedGamepad', 'CAValueFunction', 'SCNCylinder', 'NSNotification', 'NSBatchUpdateResult', 'PKPushCredentials', 'SCNPhysicsSliderJoint', 'AVCaptureDeviceFormat', 'AVPlayerItemErrorLog', 'NSMapTable', 'NSSet', 'CMMotionManager', 'GKVoiceChatService', 'UIPageControl', 'UILexicon', 'MTLArrayType', 'AVAudioUnitReverb', 'MKGeodesicPolyline', 'AVMutableComposition', 'NSLayoutConstraint', 'UIPrinter', 'NSOrderedSet', 'CBAttribute', 'PKPushPayload', 'NSIncrementalStoreNode', 'EKEventStore', 'MPRemoteCommandEvent', 'UISlider', 'UIBlurEffect', 'CKAsset', 'AVCaptureInput', 'AVAudioEngine', 'MTLVertexDescriptor', 'SKPhysicsBody', 'NSOperation', 'PKPaymentPass', 'UIImageAsset', 'MKMapCamera', 'SKProductsResponse', 'GLKEffectPropertyMaterial', 'AVCaptureDevice', 'CTCallCenter', 'CABTMIDILocalPeripheralViewController', 'NEVPNManager', 'HKQuery', 'SCNPhysicsContact', 'CBMutableService', 'AVSampleBufferDisplayLayer', 'SCNSceneSource', 'SKLightNode', 'CKDiscoveredUserInfo', 'NSMutableArray', 'MTLDepthStencilDescriptor', 'MTLArgument', 'NSMassFormatter', 'CIRectangleFeature', 'PKPushRegistry', 'NEVPNConnection', 'MCNearbyServiceBrowser', 'NSOperationQueue', 'MKPolylineRenderer', 'HKWorkout', 'NSValueTransformer', 'UICollectionViewFlowLayout', 'MPChangePlaybackRateCommandEvent', 'NSEntityMapping', 'SKTexture', 'NSMergePolicy', 'UITextInputStringTokenizer', 'NSRecursiveLock', 'AVAsset', 'NSUndoManager', 'AVAudioUnitSampler', 'NSItemProvider', 'SKUniform', 'MPMediaPickerController', 'CKOperation', 'MTLRenderPipelineDescriptor', 'EAWiFiUnconfiguredAccessory', 'NSFileCoordinator', 'SKRequest', 'NSFileHandle', 'NSConditionLock', 'UISegmentedControl', 'NSManagedObjectModel', 'UITabBarItem', 'SCNCone', 'MPMediaItem', 'SCNMaterial', 'EKRecurrenceRule', 'UIEvent', 'UITouch', 'UIPrintInteractionController', 'CMDeviceMotion', 'NEVPNProtocol', 'NSCompoundPredicate', 'HKHealthStore', 'MKMultiPoint', 'HKSampleType', 'UIPrintFormatter', 'AVAudioUnitEQFilterParameters', 'SKView', 'NSConstantString', 'UIPopoverController', 'CKDatabase', 'AVMetadataFaceObject', 'UIAccelerometer', 'EKEventViewController', 'CMAltitudeData', 'MTLStencilDescriptor', 'UISwipeGestureRecognizer', 'NSPort', 'MKCircleRenderer', 'AVCompositionTrack', 'NSAsynchronousFetchRequest', 'NSUbiquitousKeyValueStore', 'NSMetadataQueryResultGroup', 'AVAssetResourceLoadingDataRequest', 'UITableViewHeaderFooterView', 'CKNotificationID', 'AVAudioSession', 'HKUnit', 'NSNull', 'NSPersistentStoreResult', 'MKCircleView', 'AVAudioChannelLayout', 'NEVPNProtocolIKEv2', 'WKProcessPool', 'UIAttachmentBehavior', 'CLBeacon', 'NSInputStream', 'NSURLCache', 'GKPlayer', 'NSMappingModel', 'CIQRCodeFeature', 'AVMutableVideoComposition', 'PHFetchResult', 'NSAttributeDescription', 'AVPlayer', 'MKAnnotationView', 'PKPaymentRequest', 'NSTimer', 'CBDescriptor', 'MKOverlayView', 'AVAudioUnitTimePitch', 'NSSaveChangesRequest', 'UIReferenceLibraryViewController', 'SKPhysicsJointFixed', 'UILocalizedIndexedCollation', 'UIInterpolatingMotionEffect', 'UIDocumentPickerViewController', 'AVAssetWriter', 'NSBundle', 'SKStoreProductViewController', 'GLKViewController', 'NSMetadataQueryAttributeValueTuple', 'GKTurnBasedMatch', 'AVAudioFile', 'UIActivity', 'NSPipe', 'MKShape', 'NSMergeConflict', 'CIImage', 'HKObject', 'UIRotationGestureRecognizer', 'AVPlayerItemLegibleOutput', 'AVAssetImageGenerator', 'GCControllerButtonInput', 'CKMarkNotificationsReadOperation', 'CKSubscription', 'MPTimedMetadata', 'NKIssue', 'UIScreenMode', 'HMAccessoryBrowser', 'GKTurnBasedEventHandler', 'UIWebView', 'MKPolyline', 'JSVirtualMachine', 'AVAssetReader', 'NSAttributedString', 'GKMatchmakerViewController', 'NSCountedSet', 'UIButton', 'WKNavigationResponse', 'GKLocalPlayer', 'MPMovieErrorLog', 'AVSpeechUtterance', 'HKStatistics', 'UILocalNotification', 'HKBiologicalSexObject', 'AVURLAsset', 'CBPeripheral', 'NSDateComponentsFormatter', 'SKSpriteNode', 'UIAccessibilityElement', 'AVAssetWriterInputGroup', 'HMZone', 'AVAssetReaderAudioMixOutput', 'NSEnumerator', 'UIDocument', 'MKLocalSearchResponse', 'UISimpleTextPrintFormatter', 'PHPhotoLibrary', 'CBService', 'UIDocumentMenuViewController', 'MCSession', 'QLPreviewController', 'CAMediaTimingFunction', 'UITextPosition', 'ASIdentifierManager', 'AVAssetResourceLoadingRequest', 'SLComposeServiceViewController', 'UIPinchGestureRecognizer', 'PHObject', 'NSExtensionItem', 'HKSampleQuery', 'MTLRenderPipelineColorAttachmentDescriptorArray', 'MKRouteStep', 'SCNCapsule', 'NSMetadataQuery', 'AVAssetResourceLoadingContentInformationRequest', 'UITraitCollection', 'CTCarrier', 'NSFileSecurity', 'UIAcceleration', 'UIMotionEffect', 'MTLRenderPipelineReflection', 'CLHeading', 'CLVisit', 'MKDirectionsResponse', 'HMAccessory', 'MTLStructType', 'UITextView', 'CMMagnetometerData', 'UICollisionBehavior', 'UIProgressView', 'CKServerChangeToken', 'UISearchBar', 'MKPlacemark', 'AVCaptureConnection', 'NSPropertyMapping', 'ALAssetsFilter', 'SK3DNode', 'AVPlayerItemErrorLogEvent', 'NSJSONSerialization', 'AVAssetReaderVideoCompositionOutput', 'ABPersonViewController', 'CIDetector', 'GKTurnBasedMatchmakerViewController', 'MPMediaItemCollection', 'SCNSphere', 'NSCondition', 'NSURLCredential', 'MIDINetworkConnection', 'NSFileProviderExtension', 'NSDecimalNumberHandler', 'NSAtomicStoreCacheNode', 'NSAtomicStore', 'EKAlarm', 'CKNotificationInfo', 'AVAudioUnitEQ', 'UIPercentDrivenInteractiveTransition', 'MKPolygon', 'AVAssetTrackSegment', 'MTLVertexAttribute', 'NSExpressionDescription', 'HKStatisticsCollectionQuery', 'NSURLAuthenticationChallenge', 'NSDirectoryEnumerator', 'MKDistanceFormatter', 'UIAlertAction', 'NSPropertyListSerialization', 'GKPeerPickerController', 'UIUserNotificationSettings', 'UITableViewController', 'GKNotificationBanner', 'MKPointAnnotation', 'MTLRenderPassColorAttachmentDescriptorArray', 'NSCache', 'SKPhysicsJoint', 'NSXMLParser', 'UIViewController', 'PKPaymentToken', 'MFMessageComposeViewController', 'AVAudioInputNode', 'NSDataDetector', 'CABTMIDICentralViewController', 'AVAudioUnitMIDIInstrument', 'AVCaptureVideoPreviewLayer', 'AVAssetWriterInputPassDescription', 'MPChangePlaybackRateCommand', 'NSURLComponents', 'CAMetalLayer', 'UISnapBehavior', 'AVMetadataMachineReadableCodeObject', 'CKDiscoverUserInfosOperation', 'NSTextAttachment', 'NSException', 'UIMenuItem', 'CMMotionActivityManager', 'SCNGeometryElement', 'NCWidgetController', 'CAEmitterLayer', 'MKUserLocation', 'UIImagePickerController', 'CIFeature', 'AVCaptureDeviceInput', 'ALAsset', 'NSURLSessionDownloadTask', 'SCNPhysicsHingeJoint', 'MPMoviePlayerViewController', 'NSMutableOrderedSet', 'SCNMaterialProperty', 'UIFont', 'AVCaptureVideoDataOutput', 'NSCachedURLResponse', 'ALAssetsLibrary', 'NSInvocation', 'UILongPressGestureRecognizer', 'NSTextStorage', 'WKWebViewConfiguration', 'CIFaceFeature', 'MKMapSnapshot', 'GLKEffectPropertyFog', 'AVComposition', 'CKDiscoverAllContactsOperation', 'AVAudioMixInputParameters', 'CAEmitterBehavior', 'PKPassLibrary', 'UIMutableUserNotificationCategory', 'NSLock', 'NEVPNProtocolIPSec', 'ADBannerView', 'UIDocumentPickerExtensionViewController', 'UIActivityIndicatorView', 'AVPlayerMediaSelectionCriteria', 'CALayer', 'UIAccessibilityCustomAction', 'UIBarButtonItem', 'AVAudioSessionRouteDescription', 'CLBeaconRegion', 'HKBloodTypeObject', 'MTLVertexBufferLayoutDescriptorArray', 'CABasicAnimation', 'AVVideoCompositionInstruction', 'AVMutableTimedMetadataGroup', 'EKRecurrenceEnd', 'NSTextContainer', 'TWTweetComposeViewController', 'PKPaymentAuthorizationViewController', 'UIScrollView', 'WKNavigationAction', 'AVPlayerItemMetadataOutput', 'EKRecurrenceDayOfWeek', 'NSNumberFormatter', 'MTLComputePipelineReflection', 'UIScreen', 'CLRegion', 'NSProcessInfo', 'GLKTextureInfo', 'SCNSkinner', 'AVCaptureMetadataOutput', 'SCNAnimationEvent', 'NSTextTab', 'JSManagedValue', 'NSDate', 'UITextChecker', 'WKBackForwardListItem', 'NSData', 'NSParagraphStyle', 'AVMutableMetadataItem', 'EKCalendar', 'HKWorkoutEvent', 'NSMutableURLRequest', 'UIVideoEditorController', 'HMTimerTrigger', 'AVAudioUnitVarispeed', 'UIDynamicAnimator', 'AVCompositionTrackSegment', 'GCGamepadSnapshot', 'MPMediaEntity', 'GLKSkyboxEffect', 'UISwitch', 'EKStructuredLocation', 'UIGestureRecognizer', 'NSProxy', 'GLKBaseEffect', 'UIPushBehavior', 'GKScoreChallenge', 'NSCoder', 'MPMediaPlaylist', 'NSDateComponents', 'WKUserScript', 'EKEvent', 'NSDateFormatter', 'NSAsynchronousFetchResult', 'AVAssetWriterInputPixelBufferAdaptor', 'UIVisualEffect', 'UICollectionViewCell', 'UITextField', 'CLPlacemark', 'MPPlayableContentManager', 'AVCaptureOutput', 'HMCharacteristicWriteAction', 'CKModifySubscriptionsOperation', 'NSPropertyDescription', 'GCGamepad', 'UIMarkupTextPrintFormatter', 'SCNTube', 'NSPersistentStoreCoordinator', 'AVAudioEnvironmentNode', 'GKMatchmaker', 'CIContext', 'NSThread', 'SLComposeSheetConfigurationItem', 'SKPhysicsJointSliding', 'NSPredicate', 'GKVoiceChat', 'SKCropNode', 'AVCaptureAudioPreviewOutput', 'NSStringDrawingContext', 'GKGameCenterViewController', 'UIPrintPaper', 'SCNPhysicsBallSocketJoint', 'UICollectionViewLayoutInvalidationContext', 'GLKEffectPropertyTransform', 'AVAudioIONode', 'UIDatePicker', 'MKDirections', 'ALAssetsGroup', 'CKRecordZoneNotification', 'SCNScene', 'MPMovieAccessLogEvent', 'CKFetchSubscriptionsOperation', 'CAEmitterCell', 'AVAudioUnitTimeEffect', 'HMCharacteristicMetadata', 'MKPinAnnotationView', 'UIPickerView', 'UIImageView', 'UIUserNotificationCategory', 'SCNPhysicsVehicleWheel', 'HKCategoryType', 'MPMediaQuerySection', 'GKFriendRequestComposeViewController', 'NSError', 'MTLRenderPipelineColorAttachmentDescriptor', 'SCNPhysicsShape', 'UISearchController', 'SCNPhysicsBody', 'CTSubscriberInfo', 'AVPlayerItemAccessLog', 'MPMediaPropertyPredicate', 'CMLogItem', 'NSAutoreleasePool', 'NSSocketPort', 'AVAssetReaderTrackOutput', 'SKNode', 'UIMutableUserNotificationAction', 'SCNProgram', 'AVSpeechSynthesisVoice', 'CMAltimeter', 'AVCaptureAudioChannel', 'GKTurnBasedExchangeReply', 'AVVideoCompositionLayerInstruction', 'AVSpeechSynthesizer', 'GKChallengeEventHandler', 'AVCaptureFileOutput', 'UIControl', 'SCNPhysicsField', 'CKReference', 'LAContext', 'CKRecordID', 'ADInterstitialAd', 'AVAudioSessionDataSourceDescription', 'AVAudioBuffer', 'CIColorKernel', 'GCControllerDirectionPad', 'NSFileManager', 'AVMutableAudioMixInputParameters', 'UIScreenEdgePanGestureRecognizer', 'CAKeyframeAnimation', 'CKQueryNotification', 'PHAdjustmentData', 'EASession', 'AVAssetResourceRenewalRequest', 'UIInputView', 'NSFileWrapper', 'UIResponder', 'NSPointerFunctions', 'UIKeyCommand', 'NSHTTPCookieStorage', 'AVMediaSelectionOption', 'NSRunLoop', 'NSFileAccessIntent', 'CAAnimationGroup', 'MKCircle', 'UIAlertController', 'NSMigrationManager', 'NSDateIntervalFormatter', 'UICollectionViewUpdateItem', 'CKDatabaseOperation', 'PHImageRequestOptions', 'SKReachConstraints', 'CKRecord', 'CAInterAppAudioSwitcherView', 'WKWindowFeatures', 'GKInvite', 'NSMutableData', 'PHAssetCollectionChangeRequest', 'NSMutableParagraphStyle', 'UIDynamicBehavior', 'GLKEffectProperty', 'CKFetchRecordChangesOperation', 'SKShapeNode', 'MPMovieErrorLogEvent', 'MKPolygonView', 'MPContentItem', 'HMAction', 'NSScanner', 'GKAchievementChallenge', 'AVAudioPlayer', 'CKContainer', 'AVVideoComposition', 'NKLibrary', 'NSPersistentStore', 'AVCaptureMovieFileOutput', 'HMRoom', 'GKChallenge', 'UITextRange', 'NSURLProtectionSpace', 'ACAccountStore', 'MPSkipIntervalCommand', 'NSComparisonPredicate', 'HMHome', 'PHVideoRequestOptions', 'NSOutputStream', 'MPSkipIntervalCommandEvent', 'PKAddPassesViewController', 'UITextSelectionRect', 'CTTelephonyNetworkInfo', 'AVTextStyleRule', 'NSFetchedPropertyDescription', 'UIPageViewController', 'CATransformLayer', 'UICollectionViewController', 'AVAudioNode', 'MCNearbyServiceAdvertiser', 'NSObject', 'PHAsset', 'GKLeaderboardViewController', 'CKQueryCursor', 'MPMusicPlayerController', 'MKOverlayPathRenderer', 'CMPedometerData', 'HMService', 'SKFieldNode', 'GKAchievement', 'WKUserContentController', 'AVAssetTrack', 'TWRequest', 'SKLabelNode', 'AVCaptureBracketedStillImageSettings', 'MIDINetworkHost', 'MPMediaPredicate', 'AVFrameRateRange', 'MTLTextureDescriptor', 'MTLVertexBufferLayoutDescriptor', 'MPFeedbackCommandEvent', 'UIUserNotificationAction', 'HKStatisticsQuery', 'SCNParticleSystem', 'NSIndexPath', 'AVVideoCompositionRenderContext', 'CADisplayLink', 'HKObserverQuery', 'UIPopoverPresentationController', 'CKQueryOperation', 'CAEAGLLayer', 'NSMutableString', 'NSMessagePort', 'NSURLQueryItem', 'MTLStructMember', 'AVAudioSessionChannelDescription', 'GLKView', 'UIActivityViewController', 'GKAchievementViewController', 'GKTurnBasedParticipant', 'NSURLProtocol', 'NSUserDefaults', 'NSCalendar', 'SKKeyframeSequence', 'AVMetadataItemFilter', 'CKModifyRecordZonesOperation', 'WKPreferences', 'NSMethodSignature', 'NSRegularExpression', 'EAGLSharegroup', 'AVPlayerItemVideoOutput', 'PHContentEditingInputRequestOptions', 'GKMatch', 'CIColor', 'UIDictationPhrase']) +COCOA_PROTOCOLS = set(['SKStoreProductViewControllerDelegate', 'AVVideoCompositionInstruction', 'AVAudioSessionDelegate', 'GKMatchDelegate', 'NSFileManagerDelegate', 'UILayoutSupport', 'NSCopying', 'UIPrintInteractionControllerDelegate', 'QLPreviewControllerDataSource', 'SKProductsRequestDelegate', 'NSTextStorageDelegate', 'MCBrowserViewControllerDelegate', 'MTLComputeCommandEncoder', 'SCNSceneExportDelegate', 'UISearchResultsUpdating', 'MFMailComposeViewControllerDelegate', 'MTLBlitCommandEncoder', 'NSDecimalNumberBehaviors', 'PHContentEditingController', 'NSMutableCopying', 'UIActionSheetDelegate', 'UIViewControllerTransitioningDelegate', 'UIAlertViewDelegate', 'AVAudioPlayerDelegate', 'MKReverseGeocoderDelegate', 'NSCoding', 'UITextInputTokenizer', 'GKFriendRequestComposeViewControllerDelegate', 'UIActivityItemSource', 'NSCacheDelegate', 'UIAdaptivePresentationControllerDelegate', 'GKAchievementViewControllerDelegate', 'UIViewControllerTransitionCoordinator', 'EKEventEditViewDelegate', 'NSURLConnectionDelegate', 'UITableViewDelegate', 'GKPeerPickerControllerDelegate', 'UIGuidedAccessRestrictionDelegate', 'AVSpeechSynthesizerDelegate', 'AVAudio3DMixing', 'AVPlayerItemLegibleOutputPushDelegate', 'ADInterstitialAdDelegate', 'HMAccessoryBrowserDelegate', 'AVAssetResourceLoaderDelegate', 'UITabBarControllerDelegate', 'CKRecordValue', 'SKPaymentTransactionObserver', 'AVCaptureAudioDataOutputSampleBufferDelegate', 'UIInputViewAudioFeedback', 'GKChallengeListener', 'SKSceneDelegate', 'UIPickerViewDelegate', 'UIWebViewDelegate', 'UIApplicationDelegate', 'GKInviteEventListener', 'MPMediaPlayback', 'MyClassJavaScriptMethods', 'AVAsynchronousKeyValueLoading', 'QLPreviewItem', 'SCNBoundingVolume', 'NSPortDelegate', 'UIContentContainer', 'SCNNodeRendererDelegate', 'SKRequestDelegate', 'SKPhysicsContactDelegate', 'HMAccessoryDelegate', 'UIPageViewControllerDataSource', 'SCNSceneRendererDelegate', 'SCNPhysicsContactDelegate', 'MKMapViewDelegate', 'AVPlayerItemOutputPushDelegate', 'UICollectionViewDelegate', 'UIImagePickerControllerDelegate', 'MTLRenderCommandEncoder', 'PKPaymentAuthorizationViewControllerDelegate', 'UIToolbarDelegate', 'WKUIDelegate', 'SCNActionable', 'NSURLConnectionDataDelegate', 'MKOverlay', 'CBCentralManagerDelegate', 'JSExport', 'NSTextLayoutOrientationProvider', 'UIPickerViewDataSource', 'PKPushRegistryDelegate', 'UIViewControllerTransitionCoordinatorContext', 'NSLayoutManagerDelegate', 'MTLLibrary', 'NSFetchedResultsControllerDelegate', 'ABPeoplePickerNavigationControllerDelegate', 'MTLResource', 'NSDiscardableContent', 'UITextFieldDelegate', 'MTLBuffer', 'MTLSamplerState', 'GKGameCenterControllerDelegate', 'MPMediaPickerControllerDelegate', 'UISplitViewControllerDelegate', 'UIAppearance', 'UIPickerViewAccessibilityDelegate', 'UITraitEnvironment', 'UIScrollViewAccessibilityDelegate', 'ADBannerViewDelegate', 'MPPlayableContentDataSource', 'MTLComputePipelineState', 'NSURLSessionDelegate', 'MTLCommandBuffer', 'NSXMLParserDelegate', 'UIViewControllerRestoration', 'UISearchBarDelegate', 'UIBarPositioning', 'CBPeripheralDelegate', 'UISearchDisplayDelegate', 'CAAction', 'PKAddPassesViewControllerDelegate', 'MCNearbyServiceAdvertiserDelegate', 'MTLDepthStencilState', 'GKTurnBasedMatchmakerViewControllerDelegate', 'MPPlayableContentDelegate', 'AVCaptureVideoDataOutputSampleBufferDelegate', 'UIAppearanceContainer', 'UIStateRestoring', 'UITextDocumentProxy', 'MTLDrawable', 'NSURLSessionTaskDelegate', 'NSFilePresenter', 'AVAudioStereoMixing', 'UIViewControllerContextTransitioning', 'UITextInput', 'CBPeripheralManagerDelegate', 'UITextInputDelegate', 'NSFastEnumeration', 'NSURLAuthenticationChallengeSender', 'SCNProgramDelegate', 'AVVideoCompositing', 'SCNAnimatable', 'NSSecureCoding', 'MCAdvertiserAssistantDelegate', 'GKLocalPlayerListener', 'GLKNamedEffect', 'UIPopoverControllerDelegate', 'AVCaptureMetadataOutputObjectsDelegate', 'NSExtensionRequestHandling', 'UITextSelecting', 'UIPrinterPickerControllerDelegate', 'NCWidgetProviding', 'MTLCommandEncoder', 'NSURLProtocolClient', 'MFMessageComposeViewControllerDelegate', 'UIVideoEditorControllerDelegate', 'WKNavigationDelegate', 'GKSavedGameListener', 'UITableViewDataSource', 'MTLFunction', 'EKCalendarChooserDelegate', 'NSUserActivityDelegate', 'UICollisionBehaviorDelegate', 'NSStreamDelegate', 'MCNearbyServiceBrowserDelegate', 'HMHomeDelegate', 'UINavigationControllerDelegate', 'MCSessionDelegate', 'UIDocumentPickerDelegate', 'UIViewControllerInteractiveTransitioning', 'GKTurnBasedEventListener', 'SCNSceneRenderer', 'MTLTexture', 'GLKViewDelegate', 'EAAccessoryDelegate', 'WKScriptMessageHandler', 'PHPhotoLibraryChangeObserver', 'NSKeyedUnarchiverDelegate', 'AVPlayerItemMetadataOutputPushDelegate', 'NSMachPortDelegate', 'SCNShadable', 'UIPopoverBackgroundViewMethods', 'UIDocumentMenuDelegate', 'UIBarPositioningDelegate', 'ABPersonViewControllerDelegate', 'NSNetServiceBrowserDelegate', 'EKEventViewDelegate', 'UIScrollViewDelegate', 'NSURLConnectionDownloadDelegate', 'UIGestureRecognizerDelegate', 'UINavigationBarDelegate', 'AVAudioMixing', 'NSFetchedResultsSectionInfo', 'UIDocumentInteractionControllerDelegate', 'MTLParallelRenderCommandEncoder', 'QLPreviewControllerDelegate', 'UIAccessibilityReadingContent', 'ABUnknownPersonViewControllerDelegate', 'GLKViewControllerDelegate', 'UICollectionViewDelegateFlowLayout', 'UIPopoverPresentationControllerDelegate', 'UIDynamicAnimatorDelegate', 'NSTextAttachmentContainer', 'MKAnnotation', 'UIAccessibilityIdentification', 'UICoordinateSpace', 'ABNewPersonViewControllerDelegate', 'MTLDevice', 'CAMediaTiming', 'AVCaptureFileOutputRecordingDelegate', 'HMHomeManagerDelegate', 'UITextViewDelegate', 'UITabBarDelegate', 'GKLeaderboardViewControllerDelegate', 'UISearchControllerDelegate', 'EAWiFiUnconfiguredAccessoryBrowserDelegate', 'UITextInputTraits', 'MTLRenderPipelineState', 'GKVoiceChatClient', 'UIKeyInput', 'UICollectionViewDataSource', 'SCNTechniqueSupport', 'NSLocking', 'AVCaptureFileOutputDelegate', 'GKChallengeEventHandlerDelegate', 'UIObjectRestoration', 'CIFilterConstructor', 'AVPlayerItemOutputPullDelegate', 'EAGLDrawable', 'AVVideoComposi ... [truncated]
pygments/lexers/configs.py+76 −3 modified@@ -18,7 +18,8 @@ __all__ = ['IniLexer', 'RegeditLexer', 'PropertiesLexer', 'KconfigLexer', 'Cfengine3Lexer', 'ApacheConfLexer', 'SquidConfLexer', - 'NginxConfLexer', 'LighttpdConfLexer', 'DockerLexer'] + 'NginxConfLexer', 'LighttpdConfLexer', 'DockerLexer', + 'TerraformLexer'] class IniLexer(RegexLexer): @@ -28,8 +29,8 @@ class IniLexer(RegexLexer): name = 'INI' aliases = ['ini', 'cfg', 'dosini'] - filenames = ['*.ini', '*.cfg'] - mimetypes = ['text/x-ini'] + filenames = ['*.ini', '*.cfg', '*.inf'] + mimetypes = ['text/x-ini', 'text/inf'] tokens = { 'root': [ @@ -544,3 +545,75 @@ class DockerLexer(RegexLexer): (r'(.*\\\n)*.+', using(BashLexer)), ], } + + +class TerraformLexer(RegexLexer): + """ + Lexer for `terraformi .tf files <https://www.terraform.io/>`_ + + .. versionadded:: 2.1 + """ + + name = 'Terraform' + aliases = ['terraform', 'tf'] + filenames = ['*.tf'] + mimetypes = ['application/x-tf', 'application/x-terraform'] + + tokens = { + 'root': [ + include('string'), + include('punctuation'), + include('curly'), + include('basic'), + include('whitespace'), + (r'[0-9]+', Number), + ], + 'basic': [ + (words(('true', 'false'), prefix=r'\b', suffix=r'\b'), Keyword.Type), + (r'\s*/\*', Comment.Multiline, 'comment'), + (r'\s*#.*\n', Comment.Single), + (r'(.*?)(\s*)(=)', bygroups(Name.Attribute, Text, Operator)), + (words(('variable', 'resource', 'provider', 'provisioner', 'module'), + prefix=r'\b', suffix=r'\b'), Keyword.Reserved, 'function'), + (words(('ingress', 'egress', 'listener', 'default', 'connection'), + prefix=r'\b', suffix=r'\b'), Keyword.Declaration), + ('\$\{', String.Interpol, 'var_builtin'), + ], + 'function': [ + (r'(\s+)(".*")(\s+)', bygroups(Text, String, Text)), + include('punctuation'), + include('curly'), + ], + 'var_builtin': [ + (r'\$\{', String.Interpol, '#push'), + (words(('concat', 'file', 'join', 'lookup', 'element'), + prefix=r'\b', suffix=r'\b'), Name.Builtin), + include('string'), + include('punctuation'), + (r'\s+', Text), + (r'\}', String.Interpol, '#pop'), + ], + 'string':[ + (r'(".*")', bygroups(String.Double)), + ], + 'punctuation':[ + (r'[\[\]\(\),.]', Punctuation), + ], + # Keep this seperate from punctuation - we sometimes want to use different + # Tokens for { } + 'curly':[ + (r'\{', Text.Punctuation), + (r'\}', Text.Punctuation), + ], + 'comment': [ + (r'[^*/]', Comment.Multiline), + (r'/\*', Comment.Multiline, '#push'), + (r'\*/', Comment.Multiline, '#pop'), + (r'[*/]', Comment.Multiline) + ], + 'whitespace': [ + (r'\n', Text), + (r'\s+', Text), + (r'\\\n', Text), + ], + }
pygments/lexers/_csound_builtins.py+1338 −0 added@@ -0,0 +1,1338 @@ +# -*- coding: utf-8 -*- + +# Opcodes in Csound 6.05 from +# csound --list-opcodes +# except +# cggoto <http://www.csounds.com/manual/html/cggoto.html> +# cigoto <http://www.csounds.com/manual/html/cigoto.html> +# cingoto (undocumented) +# ckgoto <http://www.csounds.com/manual/html/ckgoto.html> +# cngoto <http://www.csounds.com/manual/html/cngoto.html> +# endin <http://www.csounds.com/manual/html/endin.html +# endop <http://www.csounds.com/manual/html/endop.html +# goto <http://www.csounds.com/manual/html/goto.html> +# igoto <http://www.csounds.com/manual/html/igoto.html> +# kgoto <http://www.csounds.com/manual/html/kgoto.html> +# loop_ge <http://www.csounds.com/manual/html/loop_ge.html> +# loop_gt <http://www.csounds.com/manual/html/loop_gt.html> +# loop_le <http://www.csounds.com/manual/html/loop_le.html> +# loop_lt <http://www.csounds.com/manual/html/loop_lt.html> +# opcode <http://www.csounds.com/manual/html/opcode.html> +# return <http://www.csounds.com/manual/html/return.html> +# rigoto <http://www.csounds.com/manual/html/rigoto.html> +# tigoto <http://www.csounds.com/manual/html/tigoto.html> +# timout <http://www.csounds.com/manual/html/timout.html> +# which are treated as keywords; the scoreline opcodes +# scoreline <http://www.csounds.com/manual/html/scoreline.html> +# scoreline_i <http://www.csounds.com/manual/html/scoreline_i.html> +# which allow Csound Score highlighting; the pyrun opcodes +# <http://www.csounds.com/manual/html/pyrun.html> +# pylrun +# pylruni +# pylrunt +# pyrun +# pyruni +# pyrunt +# which allow Python highlighting; and the Lua opcodes +# lua_exec <http://www.csounds.com/manual/html/lua_exec.html> +# lua_opdef <http://www.csounds.com/manual/html/lua_opdef.html> +# which allow Lua highlighting. +OPCODES = set(( + 'ATSadd', + 'ATSaddnz', + 'ATSbufread', + 'ATScross', + 'ATSinfo', + 'ATSinterpread', + 'ATSpartialtap', + 'ATSread', + 'ATSreadnz', + 'ATSsinnoi', + 'FLbox', + 'FLbutBank', + 'FLbutton', + 'FLcloseButton', + 'FLcolor', + 'FLcolor2', + 'FLcount', + 'FLexecButton', + 'FLgetsnap', + 'FLgroup', + 'FLgroupEnd', + 'FLgroup_end', + 'FLhide', + 'FLhvsBox', + 'FLhvsBoxSetValue', + 'FLjoy', + 'FLkeyIn', + 'FLknob', + 'FLlabel', + 'FLloadsnap', + 'FLmouse', + 'FLpack', + 'FLpackEnd', + 'FLpack_end', + 'FLpanel', + 'FLpanelEnd', + 'FLpanel_end', + 'FLprintk', + 'FLprintk2', + 'FLroller', + 'FLrun', + 'FLsavesnap', + 'FLscroll', + 'FLscrollEnd', + 'FLscroll_end', + 'FLsetAlign', + 'FLsetBox', + 'FLsetColor', + 'FLsetColor2', + 'FLsetFont', + 'FLsetPosition', + 'FLsetSize', + 'FLsetSnapGroup', + 'FLsetText', + 'FLsetTextColor', + 'FLsetTextSize', + 'FLsetTextType', + 'FLsetVal', + 'FLsetVal_i', + 'FLsetVali', + 'FLsetsnap', + 'FLshow', + 'FLslidBnk', + 'FLslidBnk2', + 'FLslidBnk2Set', + 'FLslidBnk2Setk', + 'FLslidBnkGetHandle', + 'FLslidBnkSet', + 'FLslidBnkSetk', + 'FLslider', + 'FLtabs', + 'FLtabsEnd', + 'FLtabs_end', + 'FLtext', + 'FLupdate', + 'FLvalue', + 'FLvkeybd', + 'FLvslidBnk', + 'FLvslidBnk2', + 'FLxyin', + 'MixerClear', + 'MixerGetLevel', + 'MixerReceive', + 'MixerSend', + 'MixerSetLevel', + 'MixerSetLevel_i', + 'OSCinit', + 'OSClisten', + 'OSCsend', + 'a', + 'abs', + 'active', + 'adsr', + 'adsyn', + 'adsynt', + 'adsynt2', + 'aftouch', + 'alpass', + 'alwayson', + 'ampdb', + 'ampdbfs', + 'ampmidi', + 'ampmidid', + 'areson', + 'aresonk', + 'array', + 'atone', + 'atonek', + 'atonex', + 'babo', + 'balance', + 'bamboo', + 'barmodel', + 'bbcutm', + 'bbcuts', + 'betarand', + 'bexprnd', + 'bformdec', + 'bformdec1', + 'bformenc', + 'bformenc1', + 'binit', + 'biquad', + 'biquada', + 'birnd', + 'bqrez', + 'buchla', + 'butbp', + 'butbr', + 'buthp', + 'butlp', + 'butterbp', + 'butterbr', + 'butterhp', + 'butterlp', + 'button', + 'buzz', + 'c2r', + 'cabasa', + 'cauchy', + 'cauchyi', + 'ceil', + 'cell', + 'cent', + 'centroid', + 'ceps', + #'cggoto', + 'chanctrl', + 'changed', + 'chani', + 'chano', + 'chebyshevpoly', + 'checkbox', + 'chn_S', + 'chn_a', + 'chn_k', + 'chnclear', + 'chnexport', + 'chnget', + 'chnmix', + 'chnparams', + 'chnset', + 'chuap', + #'cigoto', + #'cingoto', + #'ckgoto', + 'clear', + 'clfilt', + 'clip', + 'clockoff', + 'clockon', + 'cmplxprod', + #'cngoto', + 'comb', + 'combinv', + 'compilecsd', + 'compileorc', + 'compilestr', + 'compress', + 'connect', + 'control', + 'convle', + 'convolve', + 'copy2ftab', + 'copy2ttab', + 'copya2ftab', + 'copyf2array', + 'cos', + 'cosh', + 'cosinv', + 'cosseg', + 'cossegb', + 'cossegr', + 'cps2pch', + 'cpsmidi', + 'cpsmidib', + 'cpsmidinn', + 'cpsoct', + 'cpspch', + 'cpstmid', + 'cpstun', + 'cpstuni', + 'cpsxpch', + 'cpuprc', + 'cross2', + 'crossfm', + 'crossfmi', + 'crossfmpm', + 'crossfmpmi', + 'crosspm', + 'crosspmi', + 'crunch', + 'ctlchn', + 'ctrl14', + 'ctrl21', + 'ctrl7', + 'ctrlinit', + 'cuserrnd', + 'dam', + 'date', + 'dates', + 'db', + 'dbamp', + 'dbfsamp', + 'dcblock', + 'dcblock2', + 'dconv', + 'delay', + 'delay1', + 'delayk', + 'delayr', + 'delayw', + 'deltap', + 'deltap3', + 'deltapi', + 'deltapn', + 'deltapx', + 'deltapxw', + 'denorm', + 'diff', + 'diskgrain', + 'diskin', + 'diskin2', + 'dispfft', + 'display', + 'distort', + 'distort1', + 'divz', + 'doppler', + 'downsamp', + 'dripwater', + 'dumpk', + 'dumpk2', + 'dumpk3', + 'dumpk4', + 'duserrnd', + 'dust', + 'dust2', + #'endin', + #'endop', + 'envlpx', + 'envlpxr', + 'ephasor', + 'eqfil', + 'evalstr', + 'event', + 'event_i', + 'exciter', + 'exitnow', + 'exp', + 'expcurve', + 'expon', + 'exprand', + 'exprandi', + 'expseg', + 'expsega', + 'expsegb', + 'expsegba', + 'expsegr', + 'fareylen', + 'fareyleni', + 'faustaudio', + 'faustcompile', + 'faustctl', + 'faustgen', + 'fft', + 'fftinv', + 'ficlose', + 'filebit', + 'filelen', + 'filenchnls', + 'filepeak', + 'filesr', + 'filevalid', + 'fillarray', + 'filter2', + 'fin', + 'fini', + 'fink', + 'fiopen', + 'flanger', + 'flashtxt', + 'flooper', + 'flooper2', + 'floor', + 'fluidAllOut', + 'fluidCCi', + 'fluidCCk', + 'fluidControl', + 'fluidEngine', + 'fluidLoad', + 'fluidNote', + 'fluidOut', + 'fluidProgramSelect', + 'fluidSetInterpMethod', + 'fmb3', + 'fmbell', + 'fmmetal', + 'fmpercfl', + 'fmrhode', + 'fmvoice', + 'fmwurlie', + 'fof', + 'fof2', + 'fofilter', + 'fog', + 'fold', + 'follow', + 'follow2', + 'foscil', + 'foscili', + 'fout', + 'fouti', + 'foutir', + 'foutk', + 'fprintks', + 'fprints', + 'frac', + 'fractalnoise', + 'freeverb', + 'ftchnls', + 'ftconv', + 'ftcps', + 'ftfree', + 'ftgen', + 'ftgenonce', + 'ftgentmp', + 'ftlen', + 'ftload', + 'ftloadk', + 'ftlptim', + 'ftmorf', + 'ftresize', + 'ftresizei', + 'ftsave', + 'ftsavek', + 'ftsr', + 'gain', + 'gainslider', + 'gauss', + 'gaussi', + 'gausstrig', + 'gbuzz', + 'genarray', + 'genarray_i', + 'gendy', + 'gendyc', + 'gendyx', + 'getcfg', + 'getcol', + 'getrow', + 'gogobel', + #'goto', + 'grain', + 'grain2', + 'grain3', + 'granule', + 'guiro', + 'harmon', + 'harmon2', + 'harmon3', + 'harmon4', + 'hdf5read', + 'hdf5write', + 'hilbert', + 'hrtfearly', + 'hrtfer', + 'hrtfmove', + 'hrtfmove2', + 'hrtfreverb', + 'hrtfstat', + 'hsboscil', + 'hvs1', + 'hvs2', + 'hvs3', + 'i', + 'iceps', + #'igoto', + 'ihold', + 'imagecreate', + 'imagefree', + 'imagegetpixel', + 'imageload', + 'imagesave', + 'imagesetpixel', + 'imagesize', + 'in', + 'in32', + 'inch', + 'inh', + 'init', + 'initc14', + 'initc21', + 'initc7', + 'inleta', + 'inletf', + 'inletk', + 'inletkid', + 'inletv', + 'ino', + 'inq', + 'inrg', + 'ins', + 'insglobal', + 'insremot', + 'instr', + 'int', + 'integ', + 'interp', + 'invalue', + 'inx', + 'inz', + 'jitter', + 'jitter2', + 'jspline', + 'k', + #'kgoto', + 'ktableseg', + 'lenarray', + 'lentab', + 'lfo', + 'limit', + 'line', + 'linen', + 'linenr', + 'lineto', + 'linrand', + 'linseg', + 'linsegb', + 'linsegr', + 'locsend', + 'locsig', + 'log', + 'log10', + 'log2', + 'logbtwo', + 'logcurve', + #'loop_ge', + #'loop_gt', + #'loop_le', + #'loop_lt', + 'loopseg', + 'loopsegp', + 'looptseg', + 'loopxseg', + 'lorenz', + 'loscil', + 'loscil3', + 'loscilx', + 'lowpass2', + 'lowres', + 'lowresx', + 'lpf18', + 'lpform', + 'lpfreson', + 'lphasor', + 'lpinterp', + 'lposcil', + 'lposcil3', + 'lposcila', + 'lposcilsa', + 'lposcilsa2', + 'lpread', + 'lpreson', + 'lpshold', + 'lpsholdp', + 'lpslot', + #'lua_exec', + 'lua_ikopcall', + #'lua_opdef', + 'mac', + 'maca', + 'madsr', + 'mags', + 'mandel', + 'mandol', + 'maparray', + 'maparray_i', + 'marimba', + 'massign', + 'max', + 'max_k', + 'maxabs', + 'maxabsaccum', + 'maxaccum', + 'maxalloc', + 'maxarray', + 'maxtab', + 'mclock', + 'mdelay', + 'median', + 'mediank', + 'metro', + 'midglobal', + 'midic14', + 'midic21', + 'midic7', + 'midichannelaftertouch', + 'midichn', + 'midicontrolchange', + 'midictrl', + 'mididefault', + 'midifilestatus', + 'midiin', + 'midinoteoff', + 'midinoteoncps', + 'midinoteonkey', + 'midinoteonoct', + 'midinoteonpch', + 'midion', + 'midion2', + 'midiout', + 'midipgm', + 'midipitchbend', + 'midipolyaftertouch', + 'midiprogramchange', + 'miditempo', + 'midremot', + 'min', + 'minabs', + 'minabsaccum', + 'minaccum', + 'minarray', + 'mincer', + 'mintab', + 'mirror', + 'mode', + 'modmatrix', + 'monitor', + 'moog', + 'moogladder', + 'moogvcf', + 'moogvcf2', + 'moscil', + 'mp3bitrate', + 'mp3in', + 'mp3len', + 'mp3nchnls', + 'mp3sr', + 'mpulse', + 'mrtmsg', + 'multitap', + 'mute', + 'mxadsr', + 'nestedap', + 'nlalp', + 'nlfilt', + 'nlfilt2', + 'noise', + 'noteoff', + 'noteon', + 'noteondur', + 'noteondur2', + 'notnum', + 'nreverb', + 'nrpn', + 'nsamp', + 'nstance', + 'nstrnum', + 'ntrpol', + 'octave', + 'octcps', + 'octmidi', + 'octmidib', + 'octmidinn', + 'octpch', + #'opcode', + 'oscbnk', + 'oscil', + 'oscil1', + 'oscil1i', + 'oscil3', + 'oscili', + 'oscilikt', + 'osciliktp', + 'oscilikts', + 'osciln', + 'oscils', + 'oscilx', + 'out', + 'out32', + 'outc', + 'outch', + 'outh', + 'outiat', + 'outic', + 'outic14', + 'outipat', + 'outipb', + 'outipc', + 'outkat', + 'outkc', + 'outkc14', + 'outkpat', + 'outkpb', + 'outkpc', + 'outleta', + 'outletf', + 'outletk', + 'outletkid', + 'outletv', + 'outo', + 'outq', + 'outq1', + 'outq2', + 'outq3', + 'outq4', + 'outrg', + 'outs', + 'outs1', + 'outs2', + 'outvalue', + 'outx', + 'outz', + 'p', + 'pan', + 'pan2', + 'pareq', + 'partials', + 'partikkel', + 'partikkelget', + 'partikkelset', + 'partikkelsync', + 'passign', + 'pcauchy', + 'pchbend', + 'pchmidi', + 'pchmidib', + 'pchmidinn', + 'pchoct', + 'pconvolve', + 'pcount', + 'pdclip', + 'pdhalf', + 'pdhalfy', + 'peak', + 'pgmassign', + 'pgmchn', + 'phaser1', + 'phaser2', + 'phasor', + 'phasorbnk', + 'phs', + 'pindex', + 'pinker', + 'pinkish', + 'pitch', + 'pitchac', + 'pitchamdf', + 'planet', + 'platerev', + 'plltrack', + 'pluck', + 'poisson', + 'pol2rect', + 'polyaft', + 'polynomial', + 'pop', + 'pop_f', + 'port', + 'portk', + 'poscil', + 'poscil3', + 'pow', + 'powershape', + 'powoftwo', + 'prealloc', + 'prepiano', + 'print', + 'print_type', + 'printf', + 'printf_i', + 'printk', + 'printk2', + 'printks', + 'printks2', + 'prints', + 'product', + 'pset', + 'ptable', + 'ptable3', + 'ptablei', + 'ptableiw', + 'ptablew', + 'ptrack', + 'push', + 'push_f', + 'puts', + 'pvadd', + 'pvbufread', + 'pvcross', + 'pvinterp', + 'pvoc', + 'pvread', + 'pvs2array', + 'pvs2tab', + 'pvsadsyn', + 'pvsanal', + 'pvsarp', + 'pvsbandp', + 'pvsbandr', + 'pvsbin', + 'pvsblur', + 'pvsbuffer', + 'pvsbufread', + 'pvsbufread2', + 'pvscale', + 'pvscent', + 'pvsceps', + 'pvscross', + 'pvsdemix', + 'pvsdiskin', + 'pvsdisp', + 'pvsenvftw', + 'pvsfilter', + 'pvsfread', + 'pvsfreeze', + 'pvsfromarray', + 'pvsftr', + 'pvsftw', + 'pvsfwrite', + 'pvsgain', + 'pvsgendy', + 'pvshift', + 'pvsifd', + 'pvsin', + 'pvsinfo', + 'pvsinit', + 'pvslock', + 'pvsmaska', + 'pvsmix', + 'pvsmooth', + 'pvsmorph', + 'pvsosc', + 'pvsout', + 'pvspitch', + 'pvstanal', + 'pvstencil', + 'pvsvoc', + 'pvswarp', + 'pvsynth', + 'pwd', + 'pyassign', + 'pyassigni', + 'pyassignt', + 'pycall', + 'pycall1', + 'pycall1i', + 'pycall1t', + 'pycall2', + 'pycall2i', + 'pycall2t', + 'pycall3', + 'pycall3i', + 'pycall3t', + 'pycall4', + 'pycall4i', + 'pycall4t', + 'pycall5', + 'pycall5i', + 'pycall5t', + 'pycall6', + 'pycall6i', + 'pycall6t', + 'pycall7', + 'pycall7i', + 'pycall7t', + 'pycall8', + 'pycall8i', + 'pycall8t', + 'pycalli', + 'pycalln', + 'pycallni', + 'pycallt', + 'pyeval', + 'pyevali', + 'pyevalt', + 'pyexec', + 'pyexeci', + 'pyexect', + 'pyinit', + 'pylassign', + 'pylassigni', + 'pylassignt', + 'pylcall', + 'pylcall1', + 'pylcall1i', + 'pylcall1t', + 'pylcall2', + 'pylcall2i', + 'pylcall2t', + 'pylcall3', + 'pylcall3i', + 'pylcall3t', + 'pylcall4', + 'pylcall4i', + 'pylcall4t', + 'pylcall5', + 'pylcall5i', + 'pylcall5t', + 'pylcall6', + 'pylcall6i', + 'pylcall6t', + 'pylcall7', + 'pylcall7i', + 'pylcall7t', + 'pylcall8', + 'pylcall8i', + 'pylcall8t', + 'pylcalli', + 'pylcalln', + 'pylcallni', + 'pylcallt', + 'pyleval', + 'pylevali', + 'pylevalt', + 'pylexec', + 'pylexeci', + 'pylexect', + #'pylrun', + #'pylruni', + #'pylrunt', + #'pyrun', + #'pyruni', + #'pyrunt', + 'qinf', + 'qnan', + 'r2c', + 'rand', + 'randh', + 'randi', + 'random', + 'randomh', + 'randomi', + 'rbjeq', + 'readclock', + 'readf', + 'readfi', + 'readk', + 'readk2', + 'readk3', + 'readk4', + 'readks', + 'readscore', + 'readscratch', + 'rect2pol', + 'reinit', + 'release', + 'remoteport', + 'remove', + 'repluck', + 'reson', + 'resonk', + 'resonr', + 'resonx', + 'resonxk', + 'resony', + 'resonz', + 'resyn', + #'return', + 'reverb', + 'reverb2', + 'reverbsc', + 'rewindscore', + 'rezzy', + 'rfft', + 'rifft', + #'rigoto', + 'rireturn', + 'rms', + 'rnd', + 'rnd31', + 'round', + 'rspline', + 'rtclock', + 's16b14', + 's32b14', + 'samphold', + 'sandpaper', + 'scale', + 'scalearray', + 'scalet', + 'scanhammer', + 'scans', + 'scantable', + 'scanu', + 'schedkwhen', + 'schedkwhennamed', + 'schedule', + 'schedwhen', + #'scoreline', + #'scoreline_i', + 'seed', + 'sekere', + 'semitone', + 'sense', + 'sensekey', + 'seqtime', + 'seqtime2', + 'serialBegin', + 'serialEnd', + 'serialFlush', + 'serialPrint', + 'serialRead', + 'serialWrite', + 'serialWrite_i', + 'setcol', + 'setctrl', + 'setksmps', + 'setrow', + 'setscorepos', + 'sfilist', + 'sfinstr', + 'sfinstr3', + 'sfinstr3m', + 'sfinstrm', + 'sfload', + 'sflooper', + 'sfpassign', + 'sfplay', + 'sfplay3', + 'sfplay3m', + 'sfplaym', + 'sfplist', + 'sfpreset', + 'shaker', + 'shiftin', + 'shiftout', + 'signalflowgraph', + 'signum', + 'sin', + 'sinh', + 'sininv', + 'sinsyn', + 'sleighbells', + 'slicearray', + 'slider16', + 'slider16f', + 'slider16table', + 'slider16tablef', + 'slider32', + 'slider32f', + 'slider32table', + 'slider32tablef', + 'slider64', + 'slider64f', + 'slider64table', + 'slider64tablef', + 'slider8', + 'slider8f', + 'slider8table', + 'slider8tablef', + 'sliderKawai', + 'sndload', + 'sndloop', + 'sndwarp', + 'sndwarpst', + 'sockrecv', + 'sockrecvs', + 'socksend', + 'socksends', + 'soundin', + 'soundout', + 'soundouts', + 'space', + 'spat3d', + 'spat3di', + 'spat3dt', + 'spdist', + 'specaddm', + 'specdiff', + 'specdisp', + 'specfilt', + 'spechist', + 'specptrk', + 'specscal', + 'specsum', + 'spectrum', + 'splitrig', + 'sprintf', + 'sprintfk', + 'spsend', + 'sqrt', + 'stack', + 'statevar', + 'stix', + 'strcat', + 'strcatk', + 'strchar', + 'strchark', + 'strcmp', + 'strcmpk', + 'strcpy', + 'strcpyk', + 'strecv', + 'streson', + 'strfromurl', + 'strget', + 'strindex', + 'strindexk', + 'strlen', + 'strlenk', + 'strlower', + 'strlowerk', + 'strrindex', + 'strrindexk', + 'strset', + 'strsub', + 'strsubk', + 'strtod', + 'strtodk', + 'strtol', + 'strtolk', + 'strupper', + 'strupperk', + 'stsend', + 'subinstr', + 'subinstrinit', + 'sum', + 'sumarray', + 'sumtab', + 'svfilter', + 'syncgrain', + 'syncloop', + 'syncphasor', + 'system', + 'system_i', + 'tab', + 'tab2pvs', + 'tab_i', + 'tabgen', + 'table', + 'table3', + 'table3kt', + 'tablecopy', + 'tablefilter', + 'tablefilteri', + 'tablegpw', + 'tablei', + 'tableicopy', + 'tableigpw', + 'tableikt', + 'tableimix', + 'tableiw', + 'tablekt', + 'tablemix', + 'tableng', + 'tablera', + 'tableseg', + 'tableshuffle', + 'tableshufflei', + 'tablew', + 'tablewa', + 'tablewkt', + 'tablexkt', + 'tablexseg', + 'tabmap', + 'tabmap_i', + 'tabmorph', + 'tabmorpha', + 'tabmorphak', + 'tabmorphi', + 'tabplay', + 'tabrec', + 'tabslice', + 'tabsum', + 'tabw', + 'tabw_i', + 'tambourine', + 'tan', + 'tanh', + 'taninv', + 'taninv2', + 'tb0', + 'tb0_init', + 'tb1', + 'tb10', + 'tb10_init', + 'tb11', + 'tb11_init', + 'tb12', + 'tb12_init', + 'tb13', + 'tb13_init', + 'tb14', + 'tb14_init', + 'tb15', + 'tb15_init', + 'tb1_init', + 'tb2', + 'tb2_init', + 'tb3', + 'tb3_init', + 'tb4', + 'tb4_init', + 'tb5', + 'tb5_init', + 'tb6', + 'tb6_init', + 'tb7', + 'tb7_init', + 'tb8', + 'tb8_init', + 'tb9', + 'tb9_init', + 'tbvcf', + 'tempest', + 'tempo', + 'temposcal', + 'tempoval', + #'tigoto', + 'timedseq', + 'timeinstk', + 'timeinsts', + 'timek', + 'times', + #'timout', + 'tival', + 'tlineto', + 'tone', + 'tonek', + 'tonex', + 'tradsyn', + 'trandom', + 'transeg', + 'transegb', + 'transegr', + 'trcross', + 'trfilter', + 'trhighest', + 'trigger', + 'trigseq', + 'trirand', + 'trlowest', + 'trmix', + 'trscale', + 'trshift', + 'trsplit', + 'turnoff', + 'turnoff2', + 'turnon', + 'unirand', + 'unwrap', + 'upsamp', + 'urd', + 'vactrol', + 'vadd', + 'vadd_i', + 'vaddv', + 'vaddv_i', + 'vaget', + 'valpass', + 'vaset', + 'vbap', + 'vbap16', + 'vbap4', + 'vbap4move', + 'vbap8', + 'vbap8move', + 'vbapg', + 'vbapgmove', + 'vbaplsinit', + 'vbapmove', + 'vbapz', + 'vbapzmove', + 'vcella', + 'vco', + 'vco2', + 'vco2ft', + 'vco2ift', + 'vco2init', + 'vcomb', + 'vcopy', + 'vcopy_i', + 'vdel_k', + 'vdelay', + 'vdelay3', + 'vdelayk', + 'vdelayx', + 'vdelayxq', + 'vdelayxs', + 'vdelayxw', + 'vdelayxwq', + 'vdelayxws', + 'vdivv', + 'vdivv_i', + 'vecdelay', + 'veloc', + 'vexp', + 'vexp_i', + 'vexpseg', + 'vexpv', + 'vexpv_i', + 'vibes', + 'vibr', + 'vibrato', + 'vincr', + 'vlimit', + 'vlinseg', + 'vlowres', + 'vmap', + 'vmirror', + 'vmult', + 'vmult_i', + 'vmultv', + 'vmultv_i', + 'voice', + 'vosim', + 'vphaseseg', + 'vport', + 'vpow', + 'vpow_i', + 'vpowv', + 'vpowv_i', + 'vpvoc', + 'vrandh', + 'vrandi', + 'vsubv', + 'vsubv_i', + 'vtaba', + 'vtabi', + 'vtabk', + 'vtable1k', + 'vtablea', + 'vtablei', + 'vtablek', + 'vtablewa', + 'vtablewi', + 'vtablewk', + 'vtabwa', + 'vtabwi', + 'vtabwk', + 'vwrap', + 'waveset', + 'weibull', + 'wgbow', + 'wgbowedbar', + 'wgbrass', + 'wgclar', + 'wgflute', + 'wgpluck', + 'wgpluck2', + 'wguide1', + 'wguide2', + 'wiiconnect', + 'wiidata', + 'wiirange', + 'wiisend', + 'window', + 'wrap', + 'writescratch', + 'wterrain', + 'xadsr', + 'xin', + 'xout', + 'xscanmap', + 'xscans', + 'xscansmap', + 'xscanu', + 'xtratim', + 'xyin', + 'zacl', + 'zakinit', + 'zamod', + 'zar', + 'zarg', + 'zaw', + 'zawm', + 'zfilter2', + 'zir', + 'ziw', + 'ziwm', + 'zkcl', + 'zkmod', + 'zkr', + 'zkw', + 'zkwm' +))
pygments/lexers/csound.py+322 −0 added@@ -0,0 +1,322 @@ +# -*- coding: utf-8 -*- + +import re + +from pygments.lexer import RegexLexer, bygroups, default, include, using, words +from pygments.token import Comment, Keyword, Name, Number, Operator, Punctuation, String, Text, Whitespace + +# The CsoundDocumentLexer casuses a Pygments test to fail. +__all__ = ['CsoundScoreLexer', 'CsoundOrchestraLexer']#, 'CsoundDocumentLexer'] + +newline = (r'((?:;|//).*)*(\n)', bygroups(Comment.Single, Text)) + + +class CsoundLexer(RegexLexer): + # Subclasses must define a 'single-line string' state. + tokens = { + 'whitespace': [ + (r'[ \t]+', Text), + (r'\\\n', Text), + (r'/[*](.|\n)*?[*]/', Comment.Multiline) + ], + + 'macro call': [ + (r'(\$\w+\.?)(\()', bygroups(Comment.Preproc, Punctuation), 'function macro call'), + (r'\$\w+(\.|\b)', Comment.Preproc) + ], + 'function macro call': [ + (r"((?:\\['\)]|[^'\)])+)(')", bygroups(Comment.Preproc, Punctuation)), + (r"([^'\)]+)(\))", bygroups(Comment.Preproc, Punctuation), '#pop') + ], + + 'whitespace or macro call': [ + include('whitespace'), + include('macro call') + ], + + 'preprocessor directives': [ + (r'#(e(nd(if)?|lse)|ifn?def|undef)\b|##', Comment.Preproc), + (r'#include\b', Comment.Preproc, 'include'), + (r'#[ \t]*define\b', Comment.Preproc, 'macro name'), + (r'@+[ \t]*\d*', Comment.Preproc) + ], + + 'include': [ + include('whitespace'), + (r'"', String, 'single-line string') + ], + + 'macro name': [ + include('whitespace'), + (r'(\w+)(\()', bygroups(Comment.Preproc, Text), 'function macro argument list'), + (r'\w+', Comment.Preproc, 'object macro definition after name') + ], + 'object macro definition after name': [ + include('whitespace'), + (r'#', Punctuation, 'object macro replacement text') + ], + 'object macro replacement text': [ + (r'(\\#|[^#])+', Comment.Preproc), + (r'#', Punctuation, '#pop:3') + ], + 'function macro argument list': [ + (r"(\w+)(['#])", bygroups(Comment.Preproc, Punctuation)), + (r'(\w+)(\))', bygroups(Comment.Preproc, Punctuation), 'function macro definition after name') + ], + 'function macro definition after name': [ + (r'[ \t]+', Text), + (r'#', Punctuation, 'function macro replacement text') + ], + 'function macro replacement text': [ + (r'(\\#|[^#])+', Comment.Preproc), + (r'#', Punctuation, '#pop:4') + ] + } + + +class CsoundScoreLexer(CsoundLexer): + """ + For `Csound <http://csound.github.io>`_ scores. + """ + + name = 'Csound Score' + filenames = ['*.sco'] + + tokens = { + 'partial statement': [ + include('preprocessor directives'), + (r'\d+e[+-]?\d+|(\d+\.\d*|\d*\.\d+)(e[+-]?\d+)?', Number.Float), + (r'0[xX][a-fA-F0-9]+', Number.Hex), + (r'\d+', Number.Integer), + (r'"', String, 'single-line string'), + (r'[+\-*/%^!=<>|&#~.]', Operator), + (r'[]()[]', Punctuation), + (r'\w+', Comment.Preproc) + ], + + 'statement': [ + include('whitespace or macro call'), + newline + ('#pop',), + include('partial statement') + ], + + 'root': [ + newline, + include('whitespace or macro call'), + (r'[{}]', Punctuation, 'statement'), + (r'[abefimq-tv-z]|[nN][pP]?', Keyword, 'statement') + ], + + 'single-line string': [ + (r'"', String, '#pop'), + (r'[^\\"]+', String) + ] + } + + +from pygments.lexers._csound_builtins import OPCODES +from pygments.lexers.python import PythonLexer +from pygments.lexers.scripting import LuaLexer + +class CsoundOrchestraLexer(CsoundLexer): + """ + For `Csound <http://csound.github.io>`_ orchestras. + """ + + name = 'Csound Orchestra' + filenames = ['*.orc'] + + user_defined_opcodes = set() + + def opcode_name_callback(lexer, match): + opcode = match.group(0) + lexer.user_defined_opcodes.add(opcode) + yield match.start(), Name.Function, opcode + + def name_callback(lexer, match): + name = match.group(0) + if re.match('p\d+$', name) or name in OPCODES: + yield match.start(), Name.Builtin, name + elif name in lexer.user_defined_opcodes: + yield match.start(), Name.Function, name + else: + nameMatch = re.search(r'^(g?[aikSw])(\w+)', name) + if nameMatch: + yield nameMatch.start(1), Keyword.Type, nameMatch.group(1) + yield nameMatch.start(2), Name, nameMatch.group(2) + else: + yield match.start(), Name, name + + tokens = { + 'label': [ + (r'\b(\w+)(:)', bygroups(Name.Label, Punctuation)) + ], + + 'partial expression': [ + include('preprocessor directives'), + (r'\b(0dbfs|k(r|smps)|nchnls(_i)?|sr)\b', Name.Variable.Global), + (r'\d+e[+-]?\d+|(\d+\.\d*|\d*\.\d+)(e[+-]?\d+)?', Number.Float), + (r'0[xX][a-fA-F0-9]+', Number.Hex), + (r'\d+', Number.Integer), + (r'"', String, 'single-line string'), + (r'{{', String, 'multi-line string'), + (r'[+\-*/%^!=&|<>#~¬]', Operator), + (r'[](),?:[]', Punctuation), + (words(( + # Keywords + 'do', 'else', 'elseif', 'endif', 'enduntil', 'fi', 'if', 'ithen', 'kthen', 'od', 'then', 'until', 'while', + # Opcodes that act as control structures + 'return', 'timout' + ), prefix=r'\b', suffix=r'\b'), Keyword), + (words(('goto', 'igoto', 'kgoto', 'rigoto', 'tigoto'), prefix=r'\b', suffix=r'\b'), Keyword, 'goto label'), + (words(('cggoto', 'cigoto', 'cingoto', 'ckgoto', 'cngoto'), prefix=r'\b', suffix=r'\b'), Keyword, ('goto label', 'goto expression')), + (words(('loop_ge', 'loop_gt', 'loop_le', 'loop_lt'), prefix=r'\b', suffix=r'\b'), Keyword, ('goto label', 'goto expression', 'goto expression', 'goto expression')), + (r'\bscoreline(_i)?\b', Name.Builtin, 'scoreline opcode'), + (r'\bpyl?run[it]?\b', Name.Builtin, 'python opcode'), + (r'\blua_(exec|opdef)\b', Name.Builtin, 'lua opcode'), + (r'\b[a-zA-Z_]\w*\b', name_callback) + ], + + 'expression': [ + include('whitespace or macro call'), + newline + ('#pop',), + include('partial expression') + ], + + 'root': [ + newline, + include('whitespace or macro call'), + (r'\binstr\b', Keyword, ('instrument block', 'instrument name list')), + (r'\bopcode\b', Keyword, ('opcode block', 'opcode parameter list', 'opcode types', 'opcode types', 'opcode name')), + include('label'), + default('expression') + ], + + 'instrument name list': [ + include('whitespace or macro call'), + (r'\d+|\+?[a-zA-Z_]\w*', Name.Function), + (r',', Punctuation), + newline + ('#pop',) + ], + 'instrument block': [ + newline, + include('whitespace or macro call'), + (r'\bendin\b', Keyword, '#pop'), + include('label'), + default('expression') + ], + + 'opcode name': [ + include('whitespace or macro call'), + (r'[a-zA-Z_]\w*', opcode_name_callback, '#pop') + ], + 'opcode types': [ + include('whitespace or macro call'), + (r'0|[]afijkKoOpPStV[]+', Keyword.Type, '#pop'), + (r',', Punctuation) + ], + 'opcode parameter list': [ + include('whitespace or macro call'), + newline + ('#pop',) + ], + 'opcode block': [ + newline, + include('whitespace or macro call'), + (r'\bendop\b', Keyword, '#pop'), + include('label'), + default('expression') + ], + + 'goto label': [ + include('whitespace or macro call'), + (r'\w+', Name.Label, '#pop'), + default('#pop') + ], + 'goto expression': [ + include('whitespace or macro call'), + (r',', Punctuation, '#pop'), + include('partial expression') + ], + + 'single-line string': [ + include('macro call'), + (r'"', String, '#pop'), + # From https://github.com/csound/csound/blob/develop/Opcodes/fout.c#L1405 + (r'%\d*(\.\d+)?[cdhilouxX]', String.Interpol), + (r'%[!%nNrRtT]|[~^]|\\([\\aAbBnNrRtT"]|[0-7]{1,3})', String.Escape), + (r'[^\\"~$%\^\n]+', String), + (r'[\\"~$%\^\n]', String) + ], + 'multi-line string': [ + (r'}}', String, '#pop'), + (r'[^\}]+|\}(?!\})', String) + ], + + 'scoreline opcode': [ + include('whitespace or macro call'), + (r'{{', String, 'scoreline'), + default('#pop') + ], + 'scoreline': [ + (r'}}', String, '#pop'), + (r'([^\}]+)|\}(?!\})', using(CsoundScoreLexer)) + ], + + 'python opcode': [ + include('whitespace or macro call'), + (r'{{', String, 'python'), + default('#pop') + ], + 'python': [ + (r'}}', String, '#pop'), + (r'([^\}]+)|\}(?!\})', using(PythonLexer)) + ], + + 'lua opcode': [ + include('whitespace or macro call'), + (r'"', String, 'single-line string'), + (r'{{', String, 'lua'), + (r',', Punctuation), + default('#pop') + ], + 'lua': [ + (r'}}', String, '#pop'), + (r'([^\}]+)|\}(?!\})', using(LuaLexer)) + ] + } + + +# Below is a lexer for Csound documents, but it causes a Pygments test to fail. + +# import copy +# from pygments.lexers.html import HtmlLexer, XmlLexer +# +# class CsoundDocumentLexer(XmlLexer): +# """ +# For `Csound <http://csound.github.io>`_ documents. +# """ +# +# name = 'Csound Document' +# aliases = ['csound'] +# filenames = ['*.csd'] +# +# tokens = copy.deepcopy(XmlLexer.tokens) +# for i, item in enumerate(tokens['root']): +# if len(item) > 2 and item[2] == 'tag': +# (tokens['root']).insert(i, (r'(<)(\s*)(CsInstruments)(\s*)', bygroups(Name.Tag, Text, Name.Tag, Text), ('orchestra content', 'tag'))) +# (tokens['root']).insert(i, (r'(<)(\s*)(CsScore)(\s*)', bygroups(Name.Tag, Text, Name.Tag, Text), ('score content', 'tag'))) +# (tokens['root']).insert(i, (r'(<)(\s*)(html)(\s*)', bygroups(Name.Tag, Text, Name.Tag, Text), ('HTML', 'tag'))) +# break +# +# tokens['orchestra content'] = [ +# (r'(<)(\s*)(/)(\s*)(CsInstruments)(\s*)(>)', bygroups(Name.Tag, Text, Name.Tag, Text, Name.Tag, Text, Name.Tag), '#pop'), +# (r'.+?(?=<\s*/\s*CsInstruments\s*>)', using(CsoundOrchestraLexer)) +# ] +# tokens['score content'] = [ +# (r'(<)(\s*)(/)(\s*)(CsScore)(\s*)(>)', bygroups(Name.Tag, Text, Name.Tag, Text, Name.Tag, Text, Name.Tag), '#pop'), +# (r'.+?(?=<\s*/\s*CsScore\s*>)', using(CsoundScoreLexer)) +# ] +# tokens['HTML'] = [ +# (r'(<)(\s*)(/)(\s*)(html)(\s*)(>)', bygroups(Name.Tag, Text, Name.Tag, Text, Name.Tag, Text, Name.Tag), '#pop'), +# (r'.+?(?=<\s*/\s*html\s*>)', using(HtmlLexer)) +# ]
pygments/lexers/css.py+29 −4 modified@@ -13,12 +13,12 @@ import copy from pygments.lexer import ExtendedRegexLexer, RegexLexer, include, bygroups, \ - default, words + default, words, inherit from pygments.token import Text, Comment, Operator, Keyword, Name, String, \ Number, Punctuation from pygments.util import iteritems -__all__ = ['CssLexer', 'SassLexer', 'ScssLexer'] +__all__ = ['CssLexer', 'SassLexer', 'ScssLexer', 'LessCssLexer'] class CssLexer(RegexLexer): @@ -475,8 +475,9 @@ class ScssLexer(RegexLexer): (r'(@media)(\s+)', bygroups(Keyword, Text), 'value'), (r'@[\w-]+', Keyword, 'selector'), (r'(\$[\w-]*\w)([ \t]*:)', bygroups(Name.Variable, Operator), 'value'), - (r'(?=[^;{}][;}])', Name.Attribute, 'attr'), - (r'(?=[^;{}:]+:[^a-z])', Name.Attribute, 'attr'), + # TODO: broken, and prone to infinite loops. + #(r'(?=[^;{}][;}])', Name.Attribute, 'attr'), + #(r'(?=[^;{}:]+:[^a-z])', Name.Attribute, 'attr'), default('selector'), ], @@ -497,3 +498,27 @@ class ScssLexer(RegexLexer): tokens[group] = copy.copy(common) tokens['value'].extend([(r'\n', Text), (r'[;{}]', Punctuation, '#pop')]) tokens['selector'].extend([(r'\n', Text), (r'[;{}]', Punctuation, '#pop')]) + + +class LessCssLexer(CssLexer): + """ + For `LESS <http://lesscss.org/>`_ styleshets. + + .. versionadded:: 2.1 + """ + + name = 'LessCss' + aliases = ['less'] + filenames = ['*.less'] + mimetypes = ['text/x-less-css'] + + tokens = { + 'root': [ + (r'@\w+', Name.Variable), + inherit, + ], + 'content': [ + (r'{', Punctuation, '#push'), + inherit, + ], + }
pygments/lexers/dsls.py+74 −2 modified@@ -13,10 +13,11 @@ from pygments.lexer import RegexLexer, bygroups, words, include, default from pygments.token import Text, Comment, Operator, Keyword, Name, String, \ - Number, Punctuation, Literal + Number, Punctuation, Literal, Whitespace __all__ = ['ProtoBufLexer', 'BroLexer', 'PuppetLexer', 'RslLexer', - 'MscgenLexer', 'VGLLexer', 'AlloyLexer', 'PanLexer'] + 'MscgenLexer', 'VGLLexer', 'AlloyLexer', 'PanLexer', + 'CrmshLexer'] class ProtoBufLexer(RegexLexer): @@ -512,3 +513,74 @@ class PanLexer(RegexLexer): include('root'), ], } + + +class CrmshLexer(RegexLexer): + """ + Lexer for `crmsh <http://crmsh.github.io/>`_ configuration files + for Pacemaker clusters. + + .. versionadded:: 2.1 + """ + name = 'Crmsh' + aliases = ['crmsh', 'pcmk'] + filenames = ['*.crmsh', '*.pcmk'] + mimetypes = [] + + elem = words(( + 'node', 'primitive', 'group', 'clone', 'ms', 'location', + 'colocation', 'order', 'fencing_topology', 'rsc_ticket', + 'rsc_template', 'property', 'rsc_defaults', + 'op_defaults', 'acl_target', 'acl_group', 'user', 'role', + 'tag'), suffix=r'(?![\w#$-])') + sub = words(( + 'params', 'meta', 'operations', 'op', 'rule', + 'attributes', 'utilization'), suffix=r'(?![\w#$-])') + acl = words(('read', 'write', 'deny'), suffix=r'(?![\w#$-])') + bin_rel = words(('and', 'or'), suffix=r'(?![\w#$-])') + un_ops = words(('defined', 'not_defined'), suffix=r'(?![\w#$-])') + date_exp = words(('in_range', 'date', 'spec', 'in'), suffix=r'(?![\w#$-])') + acl_mod = (r'(?:tag|ref|reference|attribute|type|xpath)') + bin_ops = (r'(?:lt|gt|lte|gte|eq|ne)') + val_qual = (r'(?:string|version|number)') + rsc_role_action=(r'(?:Master|Started|Slave|Stopped|' + r'start|promote|demote|stop)') + + tokens = { + 'root': [ + (r'^#.*\n?', Comment), + # attr=value (nvpair) + (r'([\w#$-]+)(=)("(?:""|[^"])*"|\S+)', + bygroups(Name.Attribute, Punctuation, String)), + # need this construct, otherwise numeric node ids + # are matched as scores + # elem id: + (r'(node)(\s+)([\w#$-]+)(:)', + bygroups(Keyword, Whitespace, Name, Punctuation)), + # scores + (r'([+-]?([0-9]+|inf)):', Number), + # keywords (elements and other) + (elem, Keyword), + (sub, Keyword), + (acl, Keyword), + # binary operators + (r'(?:%s:)?(%s)(?![\w#$-])' % (val_qual,bin_ops), + Operator.Word), + # other operators + (bin_rel, Operator.Word), + (un_ops, Operator.Word), + (date_exp, Operator.Word), + # builtin attributes (e.g. #uname) + (r'#[a-z]+(?![\w#$-])', Name.Builtin), + # acl_mod:blah + (r'(%s)(:)("(?:""|[^"])*"|\S+)' % acl_mod, + bygroups(Keyword, Punctuation, Name)), + # rsc_id[:(role|action)] + # NB: this matches all other identifiers + (r'([\w#$-]+)(?:(:)(%s))?(?![\w#$-])' % rsc_role_action, + bygroups(Name, Punctuation, Operator.Word)), + # punctuation + (r'(\\(?=\n)|[[\](){}/:@])', Punctuation), + (r'\s+|\n', Whitespace), + ], + }
pygments/lexers/elm.py+119 −0 added@@ -0,0 +1,119 @@ +# -*- coding: utf-8 -*- +""" + pygments.lexers.elm + ~~~~~~~~~~~~~~~~~~~ + + Lexer for the Elm programming language. + +""" + +import re + +from pygments.lexer import RegexLexer, words, include +from pygments.token import Comment, Keyword, Name, Number, Operator, Punctuation, String, Text + +__all__ = ['ElmLexer'] + +class ElmLexer(RegexLexer): + """ + For `Elm <http://elm-lang.org/>`_ source code. + """ + + name = 'Elm' + aliases = ['elm'] + filenames = ['*.elm'] + mimetypes = ['text/x-elm'] + + validName = r'[a-z_][a-zA-Z_\']*' + + specialName = r'^main ' + + builtinOps = ( + '~', '||', '|>', '|', '`', '^', '\\', '\'', '>>', '>=', '>', '==', + '=', '<~', '<|', '<=', '<<', '<-', '<', '::', ':', '/=', '//', '/', + '..', '.', '->', '-', '++', '+', '*', '&&', '%', + ) + + reservedWords = words(( + 'alias', 'as', 'case', 'else', 'if', 'import', 'in', + 'let', 'module', 'of', 'port', 'then', 'type', 'where', + ), suffix=r'\b') + + tokens = { + 'root': [ + + # Comments + (r'{-', Comment.Multiline, 'comment'), + (r'--.*', Comment.Single), + + # Whitespace + (r'\s+', Text), + + # Strings + (r'"', String, 'doublequote'), + + # Modules + (r'^\s*module\s*', Keyword.Namespace, 'imports'), + + # Imports + (r'^\s*import\s*', Keyword.Namespace, 'imports'), + + # Shaders + (r'\[glsl\|.*', Name.Entity, 'shader'), + + # Keywords + (reservedWords, Keyword.Reserved), + + # Types + (r'[A-Z]\w*', Keyword.Type), + + # Main + (specialName, Keyword.Reserved), + + # Prefix Operators + (words((builtinOps), prefix=r'\(', suffix=r'\)'), Name.Function), + + # Infix Operators + (words((builtinOps)), Name.Function), + + # Numbers + include('numbers'), + + # Variable Names + (validName, Name.Variable), + + # Parens + (r'[,\(\)\[\]{}]', Punctuation), + + ], + + 'comment': [ + (r'-(?!})', Comment.Multiline), + (r'{-', Comment.Multiline, 'comment'), + (r'[^-}]', Comment.Multiline), + (r'-}', Comment.Multiline, '#pop'), + ], + + 'doublequote': [ + (r'\\u[0-9a-fA-F]\{4}', String.Escape), + (r'\\[nrfvb\\\"]', String.Escape), + (r'[^"]', String), + (r'"', String, '#pop'), + ], + + 'imports': [ + (r'\w+(\.\w+)*', Name.Class, '#pop'), + ], + + 'numbers': [ + (r'_?\d+\.(?=\d+)', Number.Float), + (r'_?\d+', Number.Integer), + ], + + 'shader': [ + (r'\|(?!\])', Name.Entity), + (r'\|\]', Name.Entity, '#pop'), + (r'.*\n', Name.Entity), + ], + } +
pygments/lexers/esoteric.py+48 −3 modified@@ -9,11 +9,11 @@ :license: BSD, see LICENSE for details. """ -from pygments.lexer import RegexLexer, include +from pygments.lexer import RegexLexer, include, words from pygments.token import Text, Comment, Operator, Keyword, Name, String, \ - Number, Punctuation, Error + Number, Punctuation, Error, Whitespace -__all__ = ['BrainfuckLexer', 'BefungeLexer', 'RedcodeLexer'] +__all__ = ['BrainfuckLexer', 'BefungeLexer', 'BoogieLexer', 'RedcodeLexer'] class BrainfuckLexer(RegexLexer): @@ -112,3 +112,48 @@ class RedcodeLexer(RegexLexer): (r'[-+]?\d+', Number.Integer), ], } + + +class BoogieLexer(RegexLexer): + """ + For `Boogie <https://boogie.codeplex.com/>`_ source code. + + .. versionadded:: 2.1 + """ + name = 'Boogie' + aliases = ['boogie'] + filenames = ['*.bpl'] + + tokens = { + 'root': [ + # Whitespace and Comments + (r'\n', Whitespace), + (r'\s+', Whitespace), + (r'//[/!](.*?)\n', Comment.Doc), + (r'//(.*?)\n', Comment.Single), + (r'/\*', Comment.Multiline, 'comment'), + + (words(( + 'axiom', 'break', 'call', 'ensures', 'else', 'exists', 'function', + 'forall', 'if', 'invariant', 'modifies', 'procedure', 'requires', + 'then', 'var', 'while'), + suffix=r'\b'), Keyword), + (words(('const',), suffix=r'\b'), Keyword.Reserved), + + (words(('bool', 'int', 'ref'), suffix=r'\b'), Keyword.Type), + include('numbers'), + (r"(>=|<=|:=|!=|==>|&&|\|\||[+/\-=>*<\[\]])", Operator), + (r"([{}():;,.])", Punctuation), + # Identifier + (r'[a-zA-Z_]\w*', Name), + ], + 'comment': [ + (r'[^*/]+', Comment.Multiline), + (r'/\*', Comment.Multiline, '#push'), + (r'\*/', Comment.Multiline, '#pop'), + (r'[*/]', Comment.Multiline), + ], + 'numbers': [ + (r'[0-9]+', Number.Integer), + ], + }
pygments/lexers/functional.py+1 −1 modified@@ -10,7 +10,7 @@ """ from pygments.lexers.lisp import SchemeLexer, CommonLispLexer, RacketLexer, \ - NewLispLexer + NewLispLexer, ShenLexer from pygments.lexers.haskell import HaskellLexer, LiterateHaskellLexer, \ KokaLexer from pygments.lexers.theorem import CoqLexer
pygments/lexers/graph.py+1 −0 modified@@ -61,6 +61,7 @@ class CypherLexer(RegexLexer): 'relations': [ (r'(-\[)(.*?)(\]->)', bygroups(Operator, using(this), Operator)), (r'(<-\[)(.*?)(\]-)', bygroups(Operator, using(this), Operator)), + (r'(-\[)(.*?)(\]-)', bygroups(Operator, using(this), Operator)), (r'-->|<--|\[|\]', Operator), (r'<|>|<>|=|<=|=>|\(|\)|\||:|,|;', Punctuation), (r'[.*{}]', Punctuation),
pygments/lexers/idl.py+9 −2 modified@@ -12,7 +12,7 @@ import re from pygments.lexer import RegexLexer, words -from pygments.token import Text, Comment, Operator, Keyword, Name, Number +from pygments.token import Text, Comment, Operator, Keyword, Name, Number, String __all__ = ['IDLLexer'] @@ -256,7 +256,14 @@ class IDLLexer(RegexLexer): (r'\+\+|--|->|\+|-|##|#|\*|/|<|>|&&|\^|~|\|\|\?|:', Operator), (r'\b(mod=|lt=|le=|eq=|ne=|ge=|gt=|not=|and=|or=|xor=)', Operator), (r'\b(mod|lt|le|eq|ne|ge|gt|not|and|or|xor)\b', Operator), - (r'\b[0-9](L|B|S|UL|ULL|LL)?\b', Number), + (r'"[^\"]*"', String.Double), + (r"'[^\']*'", String.Single), + (r'\b[\+\-]?([0-9]*\.[0-9]+|[0-9]+\.[0-9]*)(D|E)?([\+\-]?[0-9]+)?\b', Number.Float), + (r'\b\'[\+\-]?[0-9A-F]+\'X(U?(S?|L{1,2})|B)\b', Number.Hex), + (r'\b\'[\+\-]?[0-7]+\'O(U?(S?|L{1,2})|B)\b', Number.Oct), + (r'\b[\+\-]?[0-9]+U?L{1,2}\b', Number.Integer.Long), + (r'\b[\+\-]?[0-9]+U?S?\b', Number.Integer), + (r'\b[\+\-]?[0-9]+B\b', Number), (r'.', Text), ] }
pygments/lexers/javascript.py+242 −11 modified@@ -11,15 +11,16 @@ import re -from pygments.lexer import RegexLexer, include, bygroups, default, using, this +from pygments.lexer import RegexLexer, include, bygroups, default, using, \ + this, words, combined from pygments.token import Text, Comment, Operator, Keyword, Name, String, \ Number, Punctuation, Other from pygments.util import get_bool_opt, iteritems import pygments.unistring as uni __all__ = ['JavascriptLexer', 'KalLexer', 'LiveScriptLexer', 'DartLexer', 'TypeScriptLexer', 'LassoLexer', 'ObjectiveJLexer', - 'CoffeeScriptLexer', 'MaskLexer'] + 'CoffeeScriptLexer', 'MaskLexer', 'EarlGreyLexer'] JS_IDENT_START = ('(?:[$_' + uni.combine('Lu', 'Ll', 'Lt', 'Lm', 'Lo', 'Nl') + ']|\\\\u[a-fA-F0-9]{4})') @@ -36,9 +37,9 @@ class JavascriptLexer(RegexLexer): name = 'JavaScript' aliases = ['js', 'javascript'] - filenames = ['*.js', '*.jsm', ] + filenames = ['*.js', '*.jsm'] mimetypes = ['application/javascript', 'application/x-javascript', - 'text/x-javascript', 'text/javascript', ] + 'text/x-javascript', 'text/javascript'] flags = re.DOTALL | re.UNICODE | re.MULTILINE @@ -64,30 +65,48 @@ class JavascriptLexer(RegexLexer): (r'^(?=\s|/|<!--)', Text, 'slashstartsregex'), include('commentsandwhitespace'), (r'\+\+|--|~|&&|\?|:|\|\||\\(?=\n)|' - r'(<<|>>>?|==?|!=?|[-<>+*%&|^/])=?', Operator, 'slashstartsregex'), + r'(<<|>>>?|=>|==?|!=?|[-<>+*%&|^/])=?', Operator, 'slashstartsregex'), + (r'\.\.\.', Punctuation), (r'[{(\[;,]', Punctuation, 'slashstartsregex'), (r'[})\].]', Punctuation), (r'(for|in|while|do|break|return|continue|switch|case|default|if|else|' r'throw|try|catch|finally|new|delete|typeof|instanceof|void|yield|' - r'this)\b', Keyword, 'slashstartsregex'), + r'this|of)\b', Keyword, 'slashstartsregex'), (r'(var|let|with|function)\b', Keyword.Declaration, 'slashstartsregex'), (r'(abstract|boolean|byte|char|class|const|debugger|double|enum|export|' r'extends|final|float|goto|implements|import|int|interface|long|native|' r'package|private|protected|public|short|static|super|synchronized|throws|' r'transient|volatile)\b', Keyword.Reserved), (r'(true|false|null|NaN|Infinity|undefined)\b', Keyword.Constant), (r'(Array|Boolean|Date|Error|Function|Math|netscape|' - r'Number|Object|Packages|RegExp|String|sun|decodeURI|' + r'Number|Object|Packages|RegExp|String|Promise|Proxy|sun|decodeURI|' r'decodeURIComponent|encodeURI|encodeURIComponent|' - r'Error|eval|isFinite|isNaN|parseFloat|parseInt|document|this|' - r'window)\b', Name.Builtin), + r'Error|eval|isFinite|isNaN|isSafeInteger|parseFloat|parseInt|' + r'document|this|window)\b', Name.Builtin), (JS_IDENT, Name.Other), (r'[0-9][0-9]*\.[0-9]+([eE][0-9]+)?[fd]?', Number.Float), + (r'0b[01]+', Number.Bin), + (r'0o[0-7]+', Number.Oct), (r'0x[0-9a-fA-F]+', Number.Hex), (r'[0-9]+', Number.Integer), (r'"(\\\\|\\"|[^"])*"', String.Double), (r"'(\\\\|\\'|[^'])*'", String.Single), - ] + (r'`', String.Backtick, 'interp'), + ], + 'interp': [ + (r'`', String.Backtick, '#pop'), + (r'\\\\', String.Backtick), + (r'\\`', String.Backtick), + (r'\${', String.Interpol, 'interp-inside'), + (r'\$', String.Backtick), + (r'[^`\\$]+', String.Backtick), + ], + 'interp-inside': [ + # TODO: should this include single-line comments and allow nesting strings? + (r'}', String.Interpol, '#pop'), + include('root'), + ], + #(\\\\|\\`|[^`])*`', String.Backtick), } @@ -161,7 +180,8 @@ class KalLexer(RegexLexer): (r'(Array|Boolean|Date|Error|Function|Math|netscape|' r'Number|Object|Packages|RegExp|String|sun|decodeURI|' r'decodeURIComponent|encodeURI|encodeURIComponent|' - r'eval|isFinite|isNaN|parseFloat|parseInt|document|window|' + r'eval|isFinite|isNaN|isSafeInteger|parseFloat|parseInt|document|' + r'window|' r'print)\b', Name.Builtin), (r'[$a-zA-Z_][\w.$]*\s*(:|[+\-*/]?\=)?\b', Name.Variable), @@ -1197,3 +1217,214 @@ class MaskLexer(RegexLexer): include('string-base') ], } + +class EarlGreyLexer(RegexLexer): + """ + For `Earl-Grey`_ source code. + + .. _Earl-Grey: https://breuleux.github.io/earl-grey/ + + .. versionadded: 2.1 + """ + + name = 'Earl Grey' + aliases = ['earl-grey', 'earlgrey', 'eg'] + filenames = ['*.eg'] + mimetypes = ['text/x-earl-grey'] + + tokens = { + 'root': [ + (r'\n', Text), + include('control'), + (r'[^\S\n]+', Text), + (r';;.*\n', Comment), + (r'[\[\]\{\}\:\(\)\,\;]', Punctuation), + (r'\\\n', Text), + (r'\\', Text), + include('errors'), + (words(( + 'with', 'where', 'when', 'and', 'not', 'or', 'in', + 'as', 'of', 'is'), + prefix=r'(?<=\s|\[)', suffix=r'(?![\w\$\-])'), + Operator.Word), + (r'[\*@]?->', Name.Function), + (r'[+\-*/~^<>%&|?!@#.]*=', Operator.Word), + (r'\.{2,3}', Operator.Word), # Range Operator + (r'([+*/~^<>&|?!]+)|([#\-](?=\s))|@@+(?=\s)|=+', Operator), + (r'(?<![\w\$\-])(var|let)(?:[^\w\$])', Keyword.Declaration), + include('keywords'), + include('builtins'), + include('assignment'), + (r'''(?x) + (?:()([a-zA-Z$_](?:[a-zA-Z$0-9_\-]*[a-zA-Z$0-9_])?)| + (?<=[\s\{\[\(])(\.)([a-zA-Z$_](?:[a-zA-Z$0-9_\-]*[a-zA-Z$0-9_])?)) + (?=.*%)''', + bygroups(Punctuation, Name.Tag, Punctuation, Name.Class.Start), 'dbs'), + (r'[rR]?`', String.Backtick, 'bt'), + (r'[rR]?```', String.Backtick, 'tbt'), + (r'(?<=[\s\[\{\(,;])\.([a-zA-Z$_](?:[a-zA-Z$0-9_-]*[a-zA-Z$0-9_])?)(?=[\s\]\}\),;])', + String.Symbol), + include('nested'), + (r'(?:[rR]|[rR]\.[gmi]{1,3})?"', String, combined('stringescape', 'dqs')), + (r'(?:[rR]|[rR]\.[gmi]{1,3})?\'', String, combined('stringescape', 'sqs')), + (r'"""', String, combined('stringescape', 'tdqs')), + include('tuple'), + include('import_paths'), + include('name'), + include('numbers'), + ], + 'dbs': [ + (r'(\.)([a-zA-Z$_](?:[a-zA-Z$0-9_\-]*[a-zA-Z$0-9_])?)(?=[\[\.\s])', + bygroups(Punctuation, Name.Class.DBS)), + (r'(\[)([\^#][a-zA-Z$_](?:[a-zA-Z$0-9_\-]*[a-zA-Z$0-9_])?)(\])', + bygroups(Punctuation, Name.Entity.DBS, Punctuation)), + (r'\s+', Text), + (r'%', Operator.DBS, '#pop'), + ], + 'import_paths': [ + (r'(?<=[\s:;,])(\.{1,3}(?:[\w\-]*/)*)(\w(?:[\w\-]*\w)*)(?=[\s;,])', + bygroups(Text.Whitespace, Text)), + ], + 'assignment': [ + (r'(\.)?([a-zA-Z$_](?:[a-zA-Z$0-9_-]*[a-zA-Z$0-9_])?)(?=\s+[+\-*/~^<>%&|?!@#.]*\=\s)', + bygroups(Punctuation, Name.Variable)) + ], + 'errors': [ + (words(('Error', 'TypeError', 'ReferenceError'), + prefix=r'(?<![\w\$\-\.])', suffix=r'(?![\w\$\-\.])'), + Name.Exception), + (r'''(?x) + (?<![\w\$]) + E\.[\w\$](?:[\w\$\-]*[\w\$])? + (?:\.[\w\$](?:[\w\$\-]*[\w\$])?)* + (?=[\(\{\[\?\!\s])''', + Name.Exception), + ], + 'control': [ + (r'''(?x) + ([a-zA-Z$_](?:[a-zA-Z$0-9_-]*[a-zA-Z$0-9_])?) + (?!\n)\s+ + (?!and|as|each\*|each|in|is|mod|of|or|when|where|with) + (?=(?:[+\-*/~^<>%&|?!@#.])?[a-zA-Z$_](?:[a-zA-Z$0-9_-]*[a-zA-Z$0-9_])?)''', + Keyword.Control), + (r'([a-zA-Z$_](?:[a-zA-Z$0-9_-]*[a-zA-Z$0-9_])?)(?!\n)\s+(?=[\'"\d\{\[\(])', Keyword.Control), + (r'''(?x) + (?: + (?<=[%=])| + (?<=[=\-]>)| + (?<=with|each|with)| + (?<=each\*|where) + )(\s+) + ([a-zA-Z$_](?:[a-zA-Z$0-9_\-]*[a-zA-Z$0-9_])?)(:)''', + bygroups(Text, Keyword.Control, Punctuation)), + (r'''(?x) + (?<![+\-*/~^<>%&|?!@#.])(\s+) + ([a-zA-Z$_](?:[a-zA-Z$0-9_-]*[a-zA-Z$0-9_])?)(:)''', + bygroups(Text, Keyword.Control, Punctuation)), + ], + 'nested': [ + (r'''(?x) + (?<=[a-zA-Z$0-9_\]\}\)])(\.) + ([a-zA-Z$_](?:[a-zA-Z$0-9_-]*[a-zA-Z$0-9_])?) + (?=\s+with(?:\s|\n))''', + bygroups(Punctuation, Name.Function)), + (r'''(?x) + (?<!\s)(\.) + ([a-zA-Z$_](?:[a-zA-Z$0-9_-]*[a-zA-Z$0-9_])?) + (?=[\}\]\)\.,;:\s])''', + bygroups(Punctuation, Name.Field)), + (r'''(?x) + (?<=[a-zA-Z$0-9_\]\}\)])(\.) + ([a-zA-Z$_](?:[a-zA-Z$0-9_-]*[a-zA-Z$0-9_])?) + (?=[\[\{\(:])''', + bygroups(Punctuation, Name.Function)), + ], + 'keywords': [ + (words(( + 'each', 'each*', 'mod', 'await', 'break', 'chain', + 'continue', 'elif', 'expr-value', 'if', 'match', + 'return', 'yield', 'pass', 'else', 'require', 'var', + 'let', 'async', 'method', 'gen'), + prefix=r'(?<![\w\$\-\.])', suffix=r'(?![\w\$\-\.])'), + Keyword.Pseudo), + (words(('this', 'self', '@'), + prefix=r'(?<![\w\$\-\.])', suffix=r'(?![\w\$\-])'), + Keyword.Constant), + (words(( + 'Function', 'Object', 'Array', 'String', 'Number', + 'Boolean', 'ErrorFactory', 'ENode', 'Promise'), + prefix=r'(?<![\w\$\-\.])', suffix=r'(?![\w\$\-])'), + Keyword.Type), + ], + 'builtins': [ + (words(( + 'send', 'object', 'keys', 'items', 'enumerate', 'zip', + 'product', 'neighbours', 'predicate', 'equal', + 'nequal', 'contains', 'repr', 'clone', 'range', + 'getChecker', 'get-checker', 'getProperty', 'get-property', + 'getProjector', 'get-projector', 'consume', 'take', + 'promisify', 'spawn', 'constructor'), + prefix=r'(?<![\w\-#\.])', suffix=r'(?![\w\-\.])'), + Name.Builtin), + (words(( + 'true', 'false', 'null', 'undefined'), + prefix=r'(?<![\w\$\-\.])', suffix=r'(?![\w\$\-\.])'), + Name.Constant), + ], + 'name': [ + (r'@([a-zA-Z$_](?:[a-zA-Z$0-9_-]*[a-zA-Z$0-9_])?)', Name.Variable.Instance), + (r'([a-zA-Z$_](?:[a-zA-Z$0-9_-]*[a-zA-Z$0-9_])?)(\+\+|\-\-)?', bygroups(Name.Symbol, Operator.Word)) + ], + 'tuple': [ + (r'#[a-zA-Z_][a-zA-Z_\-0-9]*(?=[\s\{\(,;\n])', Name.Namespace) + ], + 'interpoling_string': [ + (r'\}', String.Interpol, '#pop'), + include('root') + ], + 'stringescape': [ + (r'\\([\\abfnrtv"\']|\n|N\{.*?\}|u[a-fA-F0-9]{4}|' + r'U[a-fA-F0-9]{8}|x[a-fA-F0-9]{2}|[0-7]{1,3})', String.Escape) + ], + 'strings': [ + (r'[^\\\'"]', String), + (r'[\'"\\]', String), + (r'\n', String) # All strings are multiline in EG + ], + 'dqs': [ + (r'"', String, '#pop'), + (r'\\\\|\\"|\\\n', String.Escape), + include('strings') + ], + 'sqs': [ + (r"'", String, '#pop'), + (r"\\\\|\\'|\\\n", String.Escape), + (r'\{', String.Interpol, 'interpoling_string'), + include('strings') + ], + 'tdqs': [ + (r'"""', String, '#pop'), + include('strings'), + ], + 'bt': [ + (r'`', String.Backtick, '#pop'), + (r'(?<!`)\n', String.Backtick), + (r'\^=?', String.Escape), + (r'.+', String.Backtick), + ], + 'tbt': [ + (r'```', String.Backtick, '#pop'), + (r'\n', String.Backtick), + (r'\^=?', String.Escape), + (r'[^\`]+', String.Backtick), + ], + 'numbers': [ + (r'\d+\.(?!\.)\d*([eE][+-]?[0-9]+)?', Number.Float), + (r'\d+[eE][+-]?[0-9]+', Number.Float), + (r'8r[0-7]+', Number.Oct), + (r'2r[01]+', Number.Bin), + (r'16r[a-fA-F0-9]+', Number.Hex), + (r'([3-79]|[1-2][0-9]|3[0-6])r[a-zA-Z\d]+(\.[a-zA-Z\d]+)?', Number.Radix), + (r'\d+', Number.Integer) + ], + }
pygments/lexers/j.py+141 −0 added@@ -0,0 +1,141 @@ +# -*- coding: utf-8 -*- +""" + pygments.lexers.j + ~~~~~~~~~~~~~~~~~ + + Lexer for the J programming language. + +""" + +import re + +from pygments.lexer import RegexLexer, words, include +from pygments.token import Comment, Error, Keyword, Name, Number, Operator, Punctuation, String, Text + +__all__ = ['JLexer'] + +class JLexer(RegexLexer): + """ + For `J <http://jsoftware.com/>`_ source code. + """ + + name = 'J' + aliases = ['j'] + filenames = ['*.ijs'] + mimetypes = ['text/x-j'] + + validName = r'\b[a-zA-Z]\w*' + + tokens = { + 'root': [ + # Shebang script + (r'#!.*$', Comment.Preproc), + + # Comments + (r'NB\..*', Comment.Single), + (r'\n+\s*Note', Comment.Multiline, 'comment'), + (r'\s*Note.*', Comment.Single), + + # Whitespace + (r'\s+', Text), + + # Strings + (r"'", String, 'singlequote'), + + # Definitions + (r'0\s+:\s*0|noun\s+define\s*$', Name.Entity, 'nounDefinition'), + (r'\b(([1-4]|13)\s+:\s*0)|((adverb|conjunction|dyad|monad|verb)\s+define)\b', Name.Function, 'explicitDefinition'), + + # Flow Control + (words(('for_', 'goto_', 'label_'), suffix=validName+'\.'), Name.Label), + (words(( + 'assert', 'break', 'case', 'catch', 'catchd', + 'catcht', 'continue', 'do', 'else', 'elseif', + 'end', 'fcase', 'for', 'if', 'return', + 'select', 'throw', 'try', 'while', 'whilst', + ), suffix='\.'), Name.Label), + + # Variable Names + (validName, Name.Variable), + + # Standard Library + (words(( + 'ARGV', 'CR', 'CRLF', 'DEL', 'Debug', + 'EAV', 'EMPTY', 'FF', 'JVERSION', 'LF', + 'LF2', 'Note', 'TAB', 'alpha17', 'alpha27', + 'apply', 'bind', 'boxopen', 'boxxopen', 'bx', + 'clear', 'cutLF', 'cutopen', 'datatype', 'def', + 'dfh', 'drop', 'each', 'echo', 'empty', + 'erase', 'every', 'evtloop', 'exit', 'expand', + 'fetch', 'file2url', 'fixdotdot', 'fliprgb', 'getargs', + 'getenv', 'hfd', 'inv', 'inverse', 'iospath', + 'isatty', 'isutf8', 'items', 'leaf', 'list', + 'nameclass', 'namelist', 'namelist', 'names', 'nc', + 'nl', 'on', 'pick', 'pick', 'rows', + 'script', 'scriptd', 'sign', 'sminfo', 'smoutput', + 'sort', 'split', 'stderr', 'stdin', 'stdout', + 'table', 'take', 'timespacex', 'timex', 'tmoutput', + 'toCRLF', 'toHOST', 'toJ', 'tolower', 'toupper', + 'type', 'ucp', 'ucpcount', 'usleep', 'utf8', + 'uucp', + )), Name.Function), + + # Copula + (r'=[.:]', Operator), + + # Builtins + (r'[-=+*#$%@!~`^&";:.,<>{}\[\]\\|/]', Operator), + + # Short Keywords + (r'[abCdDeEfHiIjLMoprtT]\.', Keyword.Reserved), + (r'[aDiLpqsStux]\:', Keyword.Reserved), + (r'(_[0-9])\:', Keyword.Constant), + + # Parens + (r'\(', Punctuation, 'parentheses'), + + # Numbers + include('numbers'), + ], + + 'comment': [ + (r'[^)]', Comment.Multiline), + (r'^\)', Comment.Multiline, '#pop'), + (r'[)]', Comment.Multiline), + ], + + 'explicitDefinition': [ + (r'\b[nmuvxy]\b', Name.Decorator), + include('root'), + (r'[^)]', Name), + (r'^\)', Name.Label, '#pop'), + (r'[)]', Name), + ], + + 'numbers': [ + (r'\b_{1,2}\b', Number), + (r'_?\d+(\.\d+)?(\s*[ejr]\s*)_?\d+(\.?=\d+)?', Number), + (r'_?\d+\.(?=\d+)', Number.Float), + (r'_?\d+x', Number.Integer.Long), + (r'_?\d+', Number.Integer), + ], + + 'nounDefinition': [ + (r'[^)]', String), + (r'^\)', Name.Label, '#pop'), + (r'[)]', String), + ], + + 'parentheses': [ + (r'\)', Punctuation, '#pop'), + #include('nounDefinition'), + include('explicitDefinition'), + include('root'), + ], + + 'singlequote': [ + (r"[^']", String), + (r"''", String), + (r"'", String, '#pop'), + ], + } \ No newline at end of file
pygments/lexers/julia.py+3 −2 modified@@ -14,7 +14,7 @@ from pygments.lexer import Lexer, RegexLexer, bygroups, combined, do_insertions from pygments.token import Text, Comment, Operator, Keyword, Name, String, \ Number, Punctuation, Generic -from pygments.util import shebang_matches +from pygments.util import shebang_matches, unirange __all__ = ['JuliaLexer', 'JuliaConsoleLexer'] @@ -91,7 +91,8 @@ class JuliaLexer(RegexLexer): # names (r'@[\w.]+', Name.Decorator), - (u'[a-zA-Z_\u00A1-\U0010FFFF][a-zA-Z_0-9\u00A1-\U0010FFFF]*!*', Name), + (u'(?:[a-zA-Z_\u00A1-\uffff]|%s)(?:[a-zA-Z_0-9\u00A1-\uffff]|%s)*!*' % + ((unirange(0x10000, 0x10ffff),)*2), Name), # numbers (r'(\d+(_\d+)+\.\d*|\d*\.\d+(_\d+)+)([eEf][+-]?[0-9]+)?', Number.Float),
pygments/lexers/lisp.py+244 −3 modified@@ -17,9 +17,9 @@ from pygments.lexers.python import PythonLexer -__all__ = ['SchemeLexer', 'CommonLispLexer', - 'HyLexer', 'RacketLexer', - 'NewLispLexer', 'EmacsLispLexer', ] +__all__ = ['SchemeLexer', 'CommonLispLexer', 'HyLexer', 'RacketLexer', + 'NewLispLexer', 'EmacsLispLexer', 'ShenLexer', 'CPSALexer'] + class SchemeLexer(RegexLexer): """ @@ -2121,3 +2121,244 @@ def get_tokens_unprocessed(self, text): (r'"', String, '#pop'), ], } + + +class ShenLexer(RegexLexer): + """ + Lexer for `Shen <http://shenlanguage.org/>`_ source code. + + .. versionadded:: 2.1 + """ + name = 'Shen' + aliases = ['shen'] + filenames = ['*.shen'] + mimetypes = ['text/x-shen', 'application/x-shen'] + + DECLARATIONS = re.findall(r'\S+', """ + datatype define defmacro defprolog defcc synonyms declare package + type function + """) + + SPECIAL_FORMS = re.findall(r'\S+', """ + lambda get let if cases cond put time freeze value load $ + protect or and not do output prolog? trap-error error + make-string /. set @p @s @v + """) + + BUILTINS = re.findall(r'\S+', """ + == = * + - / < > >= <= <-address <-vector abort absvector + absvector? address-> adjoin append arity assoc bind boolean? + bound? call cd close cn compile concat cons cons? cut destroy + difference element? empty? enable-type-theory error-to-string + eval eval-kl exception explode external fail fail-if file + findall fix fst fwhen gensym get-time hash hd hdstr hdv head + identical implementation in include include-all-but inferences + input input+ integer? intern intersection is kill language + length limit lineread loaded macro macroexpand map mapcan + maxinferences mode n->string nl nth null number? occurrences + occurs-check open os out port porters pos pr preclude + preclude-all-but print profile profile-results ps quit read + read+ read-byte read-file read-file-as-bytelist + read-file-as-string read-from-string release remove return + reverse run save set simple-error snd specialise spy step + stinput stoutput str string->n string->symbol string? subst + symbol? systemf tail tc tc? thaw tl tlstr tlv track tuple? + undefmacro unify unify! union unprofile unspecialise untrack + variable? vector vector-> vector? verified version warn when + write-byte write-to-file y-or-n? + """) + + BUILTINS_ANYWHERE = re.findall(r'\S+', """ + where skip >> _ ! <e> <!> + """) + + MAPPINGS = dict((s, Keyword) for s in DECLARATIONS) + MAPPINGS.update((s, Name.Builtin) for s in BUILTINS) + MAPPINGS.update((s, Keyword) for s in SPECIAL_FORMS) + + valid_symbol_chars = r'[\w!$%*+,<=>?/.\'@&#:_-]' + valid_name = '%s+' % valid_symbol_chars + symbol_name = r'[a-z!$%%*+,<=>?/.\'@&#_-]%s*' % valid_symbol_chars + variable = r'[A-Z]%s*' % valid_symbol_chars + + tokens = { + 'string': [ + (r'"', String, '#pop'), + (r'c#\d{1,3};', String.Escape), + (r'~[ARS%]', String.Interpol), + (r'(?s).', String), + ], + + 'root' : [ + (r'(?s)\\\*.*?\*\\', Comment.Multiline), # \* ... *\ + (r'\\\\.*', Comment.Single), # \\ ... + (r'\s+', Text), + (r'_{5,}', Punctuation), + (r'={5,}', Punctuation), + (r'(;|:=|\||--?>|<--?)', Punctuation), + (r'(:-|:|\{|\})', Literal), + (r'[+-]*\d*\.\d+(e[+-]?\d+)?', Number.Float), + (r'[+-]*\d+', Number.Integer), + (r'"', String, 'string'), + (variable, Name.Variable), + (r'(true|false|<>|\[\])', Keyword.Pseudo), + (symbol_name, Literal), + (r'(\[|\]|\(|\))', Punctuation), + ], + } + + def get_tokens_unprocessed(self, text): + tokens = RegexLexer.get_tokens_unprocessed(self, text) + tokens = self._process_symbols(tokens) + tokens = self._process_declarations(tokens) + return tokens + + def _relevant(self, token): + return token not in (Text, Comment.Single, Comment.Multiline) + + def _process_declarations(self, tokens): + opening_paren = False + for index, token, value in tokens: + yield index, token, value + if self._relevant(token): + if opening_paren and token == Keyword and value in self.DECLARATIONS: + declaration = value + for index, token, value \ + in self._process_declaration(declaration, tokens): + yield index, token, value + opening_paren = value == '(' and token == Punctuation + + def _process_symbols(self, tokens): + opening_paren = False + for index, token, value in tokens: + if opening_paren and token in (Literal, Name.Variable): + token = self.MAPPINGS.get(value, Name.Function) + elif token == Literal and value in self.BUILTINS_ANYWHERE: + token = Name.Builtin + opening_paren = value == '(' and token == Punctuation + yield index, token, value + + def _process_declaration(self, declaration, tokens): + for index, token, value in tokens: + if self._relevant(token): + break + yield index, token, value + + if declaration == 'datatype': + prev_was_colon = False + token = Keyword.Type if token == Literal else token + yield index, token, value + for index, token, value in tokens: + if prev_was_colon and token == Literal: + token = Keyword.Type + yield index, token, value + if self._relevant(token): + prev_was_colon = token == Literal and value == ':' + elif declaration == 'package': + token = Name.Namespace if token == Literal else token + yield index, token, value + elif declaration == 'define': + token = Name.Function if token == Literal else token + yield index, token , value + for index, token, value in tokens: + if self._relevant(token): + break + yield index, token, value + if value == '{' and token == Literal: + yield index, Punctuation, value + for index, token, value in self._process_signature(tokens): + yield index, token, value + else: + yield index, token, value + else: + token = Name.Function if token == Literal else token + yield index, token , value + + raise StopIteration + + def _process_signature(self, tokens): + for index, token, value in tokens: + if token == Literal and value == '}': + yield index, Punctuation, value + raise StopIteration + elif token in (Literal, Name.Function): + token = Name.Variable if value.istitle() else Keyword.Type + yield index, token, value + + +class CPSALexer(SchemeLexer): + """ + A CPSA lexer based on the CPSA language as of version 2.2.12 + + .. versionadded:: 2.1 + """ + name = 'CPSA' + aliases = ['cpsa'] + filenames = ['*.cpsa'] + mimetypes = [] + + # list of known keywords and builtins taken form vim 6.4 scheme.vim + # syntax file. + _keywords = ( + 'herald', 'vars', 'defmacro', 'include', 'defprotocol', 'defrole', + 'defskeleton', 'defstrand', 'deflistener', 'non-orig', 'uniq-orig', + 'pen-non-orig', 'precedes', 'trace', 'send', 'recv', 'name', 'text', + 'skey', 'akey', 'data', 'mesg', + ) + _builtins = ( + 'cat', 'enc', 'hash', 'privk', 'pubk', 'invk', 'ltk', 'gen', 'exp', + ) + + # valid names for identifiers + # well, names can only not consist fully of numbers + # but this should be good enough for now + valid_name = r'[a-zA-Z0-9!$%&*+,/:<=>?@^_~|-]+' + + tokens = { + 'root' : [ + # the comments - always starting with semicolon + # and going to the end of the line + (r';.*$', Comment.Single), + + # whitespaces - usually not relevant + (r'\s+', Text), + + # numbers + (r'-?\d+\.\d+', Number.Float), + (r'-?\d+', Number.Integer), + # support for uncommon kinds of numbers - + # have to figure out what the characters mean + #(r'(#e|#i|#b|#o|#d|#x)[\d.]+', Number), + + # strings, symbols and characters + (r'"(\\\\|\\"|[^"])*"', String), + (r"'" + valid_name, String.Symbol), + (r"#\\([()/'\"._!§$%& ?=+-]{1}|[a-zA-Z0-9]+)", String.Char), + + # constants + (r'(#t|#f)', Name.Constant), + + # special operators + (r"('|#|`|,@|,|\.)", Operator), + + # highlight the keywords + (words(_keywords, suffix=r'\b'), Keyword), + + # first variable in a quoted string like + # '(this is syntactic sugar) + (r"(?<='\()" + valid_name, Name.Variable), + (r"(?<=#\()" + valid_name, Name.Variable), + + # highlight the builtins + (words(_builtins, prefix=r'(?<=\()', suffix=r'\b'), Name.Builtin), + + # the remaining functions + (r'(?<=\()' + valid_name, Name.Function), + # find the remaining variables + (valid_name, Name.Variable), + + # the famous parentheses! + (r'(\(|\))', Punctuation), + (r'(\[|\])', Punctuation), + ], + }
pygments/lexers/make.py+1 −0 modified@@ -173,6 +173,7 @@ class CMakeLexer(RegexLexer): (r'\(', Punctuation, '#push'), (r'\)', Punctuation, '#pop'), (r'(\$\{)(.+?)(\})', bygroups(Operator, Name.Variable, Operator)), + (r'(\$ENV\{)(.+?)(\})', bygroups(Operator, Name.Variable, Operator)), (r'(\$<)(.+?)(>)', bygroups(Operator, Name.Variable, Operator)), (r'(?s)".*?"', String.Double), (r'\\\S+', String),
pygments/lexers/_mapping.py+32 −6 modified@@ -9,7 +9,7 @@ Do not alter the LEXERS dictionary by hand. - :copyright: Copyright 2006-2015 by the Pygments team, see AUTHORS. + :copyright: Copyright 2006-2014 by the Pygments team, see AUTHORS. :license: BSD, see LICENSE for details. """ @@ -45,18 +45,20 @@ 'BBCodeLexer': ('pygments.lexers.markup', 'BBCode', ('bbcode',), (), ('text/x-bbcode',)), 'BaseMakefileLexer': ('pygments.lexers.make', 'Base Makefile', ('basemake',), (), ()), 'BashLexer': ('pygments.lexers.shell', 'Bash', ('bash', 'sh', 'ksh', 'shell'), ('*.sh', '*.ksh', '*.bash', '*.ebuild', '*.eclass', '.bashrc', 'bashrc', '.bash_*', 'bash_*', 'PKGBUILD'), ('application/x-sh', 'application/x-shellscript')), - 'BashSessionLexer': ('pygments.lexers.shell', 'Bash Session', ('console',), ('*.sh-session',), ('application/x-shell-session',)), + 'BashSessionLexer': ('pygments.lexers.shell', 'Bash Session', ('console', 'shell-session'), ('*.sh-session', '*.shell-session'), ('application/x-shell-session', 'application/x-sh-session')), 'BatchLexer': ('pygments.lexers.shell', 'Batchfile', ('bat', 'batch', 'dosbatch', 'winbatch'), ('*.bat', '*.cmd'), ('application/x-dos-batch',)), 'BefungeLexer': ('pygments.lexers.esoteric', 'Befunge', ('befunge',), ('*.befunge',), ('application/x-befunge',)), 'BlitzBasicLexer': ('pygments.lexers.basic', 'BlitzBasic', ('blitzbasic', 'b3d', 'bplus'), ('*.bb', '*.decls'), ('text/x-bb',)), 'BlitzMaxLexer': ('pygments.lexers.basic', 'BlitzMax', ('blitzmax', 'bmax'), ('*.bmx',), ('text/x-bmx',)), 'BooLexer': ('pygments.lexers.dotnet', 'Boo', ('boo',), ('*.boo',), ('text/x-boo',)), + 'BoogieLexer': ('pygments.lexers.esoteric', 'Boogie', ('boogie',), ('*.bpl',), ()), 'BrainfuckLexer': ('pygments.lexers.esoteric', 'Brainfuck', ('brainfuck', 'bf'), ('*.bf', '*.b'), ('application/x-brainfuck',)), 'BroLexer': ('pygments.lexers.dsls', 'Bro', ('bro',), ('*.bro',), ()), 'BugsLexer': ('pygments.lexers.modeling', 'BUGS', ('bugs', 'winbugs', 'openbugs'), ('*.bug',), ()), 'CLexer': ('pygments.lexers.c_cpp', 'C', ('c',), ('*.c', '*.h', '*.idc'), ('text/x-chdr', 'text/x-csrc')), 'CMakeLexer': ('pygments.lexers.make', 'CMake', ('cmake',), ('*.cmake', 'CMakeLists.txt'), ('text/x-cmake',)), 'CObjdumpLexer': ('pygments.lexers.asm', 'c-objdump', ('c-objdump',), ('*.c-objdump',), ('text/x-c-objdump',)), + 'CPSALexer': ('pygments.lexers.lisp', 'CPSA', ('cpsa',), ('*.cpsa',), ()), 'CSharpAspxLexer': ('pygments.lexers.dotnet', 'aspx-cs', ('aspx-cs',), ('*.aspx', '*.asax', '*.ascx', '*.ashx', '*.asmx', '*.axd'), ()), 'CSharpLexer': ('pygments.lexers.dotnet', 'C#', ('csharp', 'c#'), ('*.cs',), ('text/x-csharp',)), 'Ca65Lexer': ('pygments.lexers.asm', 'ca65 assembler', ('ca65',), ('*.s',), ()), @@ -81,11 +83,15 @@ 'ColdfusionHtmlLexer': ('pygments.lexers.templates', 'Coldfusion HTML', ('cfm',), ('*.cfm', '*.cfml'), ('application/x-coldfusion',)), 'ColdfusionLexer': ('pygments.lexers.templates', 'cfstatement', ('cfs',), (), ()), 'CommonLispLexer': ('pygments.lexers.lisp', 'Common Lisp', ('common-lisp', 'cl', 'lisp'), ('*.cl', '*.lisp'), ('text/x-common-lisp',)), + 'ComponentPascalLexer': ('pygments.lexers.oberon', 'Component Pascal', ('componentpascal', 'cp'), ('*.cp', '*.cps'), ('text/x-component-pascal',)), 'CoqLexer': ('pygments.lexers.theorem', 'Coq', ('coq',), ('*.v',), ('text/x-coq',)), 'CppLexer': ('pygments.lexers.c_cpp', 'C++', ('cpp', 'c++'), ('*.cpp', '*.hpp', '*.c++', '*.h++', '*.cc', '*.hh', '*.cxx', '*.hxx', '*.C', '*.H', '*.cp', '*.CPP'), ('text/x-c++hdr', 'text/x-c++src')), 'CppObjdumpLexer': ('pygments.lexers.asm', 'cpp-objdump', ('cpp-objdump', 'c++-objdumb', 'cxx-objdump'), ('*.cpp-objdump', '*.c++-objdump', '*.cxx-objdump'), ('text/x-cpp-objdump',)), + 'CrmshLexer': ('pygments.lexers.dsls', 'Crmsh', ('crmsh', 'pcmk'), ('*.crmsh', '*.pcmk'), ()), 'CrocLexer': ('pygments.lexers.d', 'Croc', ('croc',), ('*.croc',), ('text/x-crocsrc',)), 'CryptolLexer': ('pygments.lexers.haskell', 'Cryptol', ('cryptol', 'cry'), ('*.cry',), ('text/x-cryptol',)), + 'CsoundOrchestraLexer': ('pygments.lexers.csound', 'Csound Orchestra', (), ('*.orc',), ()), + 'CsoundScoreLexer': ('pygments.lexers.csound', 'Csound Score', (), ('*.sco',), ()), 'CssDjangoLexer': ('pygments.lexers.templates', 'CSS+Django/Jinja', ('css+django', 'css+jinja'), (), ('text/css+django', 'text/css+jinja')), 'CssErbLexer': ('pygments.lexers.templates', 'CSS+Ruby', ('css+erb', 'css+ruby'), (), ('text/css+ruby',)), 'CssGenshiLexer': ('pygments.lexers.templates', 'CSS+Genshi Text', ('css+genshitext', 'css+genshi'), (), ('text/css+genshi',)), @@ -112,10 +118,13 @@ 'DylanLidLexer': ('pygments.lexers.dylan', 'DylanLID', ('dylan-lid', 'lid'), ('*.lid', '*.hdp'), ('text/x-dylan-lid',)), 'ECLLexer': ('pygments.lexers.ecl', 'ECL', ('ecl',), ('*.ecl',), ('application/x-ecl',)), 'ECLexer': ('pygments.lexers.c_like', 'eC', ('ec',), ('*.ec', '*.eh'), ('text/x-echdr', 'text/x-ecsrc')), + 'EarlGreyLexer': ('pygments.lexers.javascript', 'Earl Grey', ('earl-grey', 'earlgrey', 'eg'), ('*.eg',), ('text/x-earl-grey',)), + 'EasytrieveLexer': ('pygments.lexers.scripting', 'Easytrieve', ('easytrieve',), ('*.ezt', '*.mac'), ('text/x-easytrieve',)), 'EbnfLexer': ('pygments.lexers.parsers', 'EBNF', ('ebnf',), ('*.ebnf',), ('text/x-ebnf',)), 'EiffelLexer': ('pygments.lexers.eiffel', 'Eiffel', ('eiffel',), ('*.e',), ('text/x-eiffel',)), 'ElixirConsoleLexer': ('pygments.lexers.erlang', 'Elixir iex session', ('iex',), (), ('text/x-elixir-shellsession',)), 'ElixirLexer': ('pygments.lexers.erlang', 'Elixir', ('elixir', 'ex', 'exs'), ('*.ex', '*.exs'), ('text/x-elixir',)), + 'ElmLexer': ('pygments.lexers.elm', 'Elm', ('elm',), ('*.elm',), ('text/x-elm',)), 'EmacsLispLexer': ('pygments.lexers.lisp', 'EmacsLisp', ('emacs', 'elisp'), ('*.el',), ('text/x-elisp', 'application/x-elisp')), 'ErbLexer': ('pygments.lexers.templates', 'ERB', ('erb',), (), ('application/x-ruby-templating',)), 'ErlangLexer': ('pygments.lexers.erlang', 'Erlang', ('erlang',), ('*.erl', '*.hrl', '*.es', '*.escript'), ('text/x-erlang',)), @@ -128,6 +137,7 @@ 'FancyLexer': ('pygments.lexers.ruby', 'Fancy', ('fancy', 'fy'), ('*.fy', '*.fancypack'), ('text/x-fancysrc',)), 'FantomLexer': ('pygments.lexers.fantom', 'Fantom', ('fan',), ('*.fan',), ('application/x-fantom',)), 'FelixLexer': ('pygments.lexers.felix', 'Felix', ('felix', 'flx'), ('*.flx', '*.flxh'), ('text/x-felix',)), + 'FishShellLexer': ('pygments.lexers.shell', 'Fish', ('fish', 'fishshell'), ('*.fish', '*.load'), ('application/x-fish',)), 'FortranFixedLexer': ('pygments.lexers.fortran', 'FortranFixed', ('fortranfixed',), ('*.f', '*.F'), ()), 'FortranLexer': ('pygments.lexers.fortran', 'Fortran', ('fortran',), ('*.f03', '*.f90', '*.F03', '*.F90'), ('text/x-fortran',)), 'FoxProLexer': ('pygments.lexers.foxpro', 'FoxPro', ('foxpro', 'vfp', 'clipper', 'xbase'), ('*.PRG', '*.prg'), ()), @@ -166,11 +176,12 @@ 'Inform6Lexer': ('pygments.lexers.int_fiction', 'Inform 6', ('inform6', 'i6'), ('*.inf',), ()), 'Inform6TemplateLexer': ('pygments.lexers.int_fiction', 'Inform 6 template', ('i6t',), ('*.i6t',), ()), 'Inform7Lexer': ('pygments.lexers.int_fiction', 'Inform 7', ('inform7', 'i7'), ('*.ni', '*.i7x'), ()), - 'IniLexer': ('pygments.lexers.configs', 'INI', ('ini', 'cfg', 'dosini'), ('*.ini', '*.cfg'), ('text/x-ini',)), + 'IniLexer': ('pygments.lexers.configs', 'INI', ('ini', 'cfg', 'dosini'), ('*.ini', '*.cfg', '*.inf'), ('text/x-ini', 'text/inf')), 'IoLexer': ('pygments.lexers.iolang', 'Io', ('io',), ('*.io',), ('text/x-iosrc',)), 'IokeLexer': ('pygments.lexers.jvm', 'Ioke', ('ioke', 'ik'), ('*.ik',), ('text/x-iokesrc',)), 'IrcLogsLexer': ('pygments.lexers.textfmts', 'IRC logs', ('irc',), ('*.weechatlog',), ('text/x-irclog',)), 'IsabelleLexer': ('pygments.lexers.theorem', 'Isabelle', ('isabelle',), ('*.thy',), ('text/x-isabelle',)), + 'JLexer': ('pygments.lexers.j', 'J', ('j',), ('*.ijs',), ('text/x-j',)), 'JadeLexer': ('pygments.lexers.html', 'Jade', ('jade',), ('*.jade',), ('text/x-jade',)), 'JagsLexer': ('pygments.lexers.modeling', 'JAGS', ('jags',), ('*.jag', '*.bug'), ()), 'JasminLexer': ('pygments.lexers.jvm', 'Jasmin', ('jasmin', 'jasminxt'), ('*.j',), ()), @@ -181,6 +192,7 @@ 'JavascriptLexer': ('pygments.lexers.javascript', 'JavaScript', ('js', 'javascript'), ('*.js', '*.jsm'), ('application/javascript', 'application/x-javascript', 'text/x-javascript', 'text/javascript')), 'JavascriptPhpLexer': ('pygments.lexers.templates', 'JavaScript+PHP', ('js+php', 'javascript+php'), (), ('application/x-javascript+php', 'text/x-javascript+php', 'text/javascript+php')), 'JavascriptSmartyLexer': ('pygments.lexers.templates', 'JavaScript+Smarty', ('js+smarty', 'javascript+smarty'), (), ('application/x-javascript+smarty', 'text/x-javascript+smarty', 'text/javascript+smarty')), + 'JclLexer': ('pygments.lexers.scripting', 'JCL', ('jcl',), ('*.jcl',), ('text/x-jcl',)), 'JsonLdLexer': ('pygments.lexers.data', 'JSON-LD', ('jsonld', 'json-ld'), ('*.jsonld',), ('application/ld+json',)), 'JsonLexer': ('pygments.lexers.data', 'JSON', ('json',), ('*.json',), ('application/json',)), 'JspLexer': ('pygments.lexers.templates', 'Java Server Page', ('jsp',), ('*.jsp',), ('application/x-jsp',)), @@ -197,6 +209,7 @@ 'LassoLexer': ('pygments.lexers.javascript', 'Lasso', ('lasso', 'lassoscript'), ('*.lasso', '*.lasso[89]'), ('text/x-lasso',)), 'LassoXmlLexer': ('pygments.lexers.templates', 'XML+Lasso', ('xml+lasso',), (), ('application/xml+lasso',)), 'LeanLexer': ('pygments.lexers.theorem', 'Lean', ('lean',), ('*.lean',), ('text/x-lean',)), + 'LessCssLexer': ('pygments.lexers.css', 'LessCss', ('less',), ('*.less',), ('text/x-less-css',)), 'LighttpdConfLexer': ('pygments.lexers.configs', 'Lighttpd configuration file', ('lighty', 'lighttpd'), (), ('text/x-lighttpd-conf',)), 'LimboLexer': ('pygments.lexers.inferno', 'Limbo', ('limbo',), ('*.b',), ('text/limbo',)), 'LiquidLexer': ('pygments.lexers.templates', 'liquid', ('liquid',), ('*.liquid',), ()), @@ -210,6 +223,7 @@ 'LogtalkLexer': ('pygments.lexers.prolog', 'Logtalk', ('logtalk',), ('*.lgt', '*.logtalk'), ('text/x-logtalk',)), 'LuaLexer': ('pygments.lexers.scripting', 'Lua', ('lua',), ('*.lua', '*.wlua'), ('text/x-lua', 'application/x-lua')), 'MOOCodeLexer': ('pygments.lexers.scripting', 'MOOCode', ('moocode', 'moo'), ('*.moo',), ('text/x-moocode',)), + 'MSDOSSessionLexer': ('pygments.lexers.shell', 'MSDOS Session', ('doscon',), (), ()), 'MakefileLexer': ('pygments.lexers.make', 'Makefile', ('make', 'makefile', 'mf', 'bsdmake'), ('*.mak', '*.mk', 'Makefile', 'makefile', 'Makefile.*', 'GNUmakefile'), ('text/x-makefile',)), 'MakoCssLexer': ('pygments.lexers.templates', 'CSS+Mako', ('css+mako',), (), ('text/css+mako',)), 'MakoHtmlLexer': ('pygments.lexers.templates', 'HTML+Mako', ('html+mako',), (), ('text/html+mako',)), @@ -266,6 +280,7 @@ 'OpaLexer': ('pygments.lexers.ml', 'Opa', ('opa',), ('*.opa',), ('text/x-opa',)), 'OpenEdgeLexer': ('pygments.lexers.business', 'OpenEdge ABL', ('openedge', 'abl', 'progress'), ('*.p', '*.cls'), ('text/x-openedge', 'application/x-openedge')), 'PanLexer': ('pygments.lexers.dsls', 'Pan', ('pan',), ('*.pan',), ()), + 'ParaSailLexer': ('pygments.lexers.parasail', 'ParaSail', ('parasail',), ('*.psi', '*.psl'), ('text/x-parasail',)), 'PawnLexer': ('pygments.lexers.pawn', 'Pawn', ('pawn',), ('*.p', '*.pwn', '*.inc'), ('text/x-pawn',)), 'Perl6Lexer': ('pygments.lexers.perl', 'Perl6', ('perl6', 'pl6'), ('*.pl', '*.pm', '*.nqp', '*.p6', '*.6pl', '*.p6l', '*.pl6', '*.6pm', '*.p6m', '*.pm6', '*.t'), ('text/x-perl6', 'application/x-perl6')), 'PerlLexer': ('pygments.lexers.perl', 'Perl', ('perl', 'pl'), ('*.pl', '*.pm', '*.t'), ('text/x-perl', 'application/x-perl')), @@ -278,6 +293,8 @@ 'PostgresLexer': ('pygments.lexers.sql', 'PostgreSQL SQL dialect', ('postgresql', 'postgres'), (), ('text/x-postgresql',)), 'PovrayLexer': ('pygments.lexers.graphics', 'POVRay', ('pov',), ('*.pov', '*.inc'), ('text/x-povray',)), 'PowerShellLexer': ('pygments.lexers.shell', 'PowerShell', ('powershell', 'posh', 'ps1', 'psm1'), ('*.ps1', '*.psm1'), ('text/x-powershell',)), + 'PowerShellSessionLexer': ('pygments.lexers.shell', 'PowerShell Session', ('ps1con',), (), ()), + 'PraatLexer': ('pygments.lexers.praat', 'Praat', ('praat',), ('*.praat', '*.proc', '*.psc'), ()), 'PrologLexer': ('pygments.lexers.prolog', 'Prolog', ('prolog',), ('*.ecl', '*.prolog', '*.pro', '*.pl'), ('text/x-prolog',)), 'PropertiesLexer': ('pygments.lexers.configs', 'Properties', ('properties', 'jproperties'), ('*.properties',), ('text/x-java-properties',)), 'ProtoBufLexer': ('pygments.lexers.dsls', 'Protocol Buffer', ('protobuf', 'proto'), ('*.proto',), ()), @@ -289,7 +306,7 @@ 'PythonLexer': ('pygments.lexers.python', 'Python', ('python', 'py', 'sage'), ('*.py', '*.pyw', '*.sc', 'SConstruct', 'SConscript', '*.tac', '*.sage'), ('text/x-python', 'application/x-python')), 'PythonTracebackLexer': ('pygments.lexers.python', 'Python Traceback', ('pytb',), ('*.pytb',), ('text/x-python-traceback',)), 'QBasicLexer': ('pygments.lexers.basic', 'QBasic', ('qbasic', 'basic'), ('*.BAS', '*.bas'), ('text/basic',)), - 'QmlLexer': ('pygments.lexers.webmisc', 'QML', ('qml',), ('*.qml',), ('application/x-qml',)), + 'QmlLexer': ('pygments.lexers.webmisc', 'QML', ('qml', 'qbs'), ('*.qml', '*.qbs'), ('application/x-qml', 'application/x-qt.qbs+qml')), 'RConsoleLexer': ('pygments.lexers.r', 'RConsole', ('rconsole', 'rout'), ('*.Rout',), ()), 'RPMSpecLexer': ('pygments.lexers.installers', 'RPMSpec', ('spec',), ('*.spec',), ('text/x-rpm-spec',)), 'RacketLexer': ('pygments.lexers.lisp', 'Racket', ('racket', 'rkt'), ('*.rkt', '*.rktd', '*.rktl'), ('text/x-racket', 'application/x-racket')), @@ -310,12 +327,15 @@ 'ResourceLexer': ('pygments.lexers.resource', 'ResourceBundle', ('resource', 'resourcebundle'), ('*.txt',), ()), 'RexxLexer': ('pygments.lexers.scripting', 'Rexx', ('rexx', 'arexx'), ('*.rexx', '*.rex', '*.rx', '*.arexx'), ('text/x-rexx',)), 'RhtmlLexer': ('pygments.lexers.templates', 'RHTML', ('rhtml', 'html+erb', 'html+ruby'), ('*.rhtml',), ('text/html+ruby',)), + 'RoboconfGraphLexer': ('pygments.lexers.roboconf', 'Roboconf Graph', ('roboconf-graph',), ('*.graph',), ()), + 'RoboconfInstancesLexer': ('pygments.lexers.roboconf', 'Roboconf Instances', ('roboconf-instances',), ('*.instances',), ()), 'RobotFrameworkLexer': ('pygments.lexers.robotframework', 'RobotFramework', ('robotframework',), ('*.txt', '*.robot'), ('text/x-robotframework',)), 'RqlLexer': ('pygments.lexers.sql', 'RQL', ('rql',), ('*.rql',), ('text/x-rql',)), 'RslLexer': ('pygments.lexers.dsls', 'RSL', ('rsl',), ('*.rsl',), ('text/rsl',)), 'RstLexer': ('pygments.lexers.markup', 'reStructuredText', ('rst', 'rest', 'restructuredtext'), ('*.rst', '*.rest'), ('text/x-rst', 'text/prs.fallenstein.rst')), + 'RtsLexer': ('pygments.lexers.trafficscript', 'TrafficScript', ('rts', 'trafficscript'), ('*.rts',), ()), 'RubyConsoleLexer': ('pygments.lexers.ruby', 'Ruby irb session', ('rbcon', 'irb'), (), ('text/x-ruby-shellsession',)), - 'RubyLexer': ('pygments.lexers.ruby', 'Ruby', ('rb', 'ruby', 'duby'), ('*.rb', '*.rbw', 'Rakefile', '*.rake', '*.gemspec', '*.rbx', '*.duby'), ('text/x-ruby', 'application/x-ruby')), + 'RubyLexer': ('pygments.lexers.ruby', 'Ruby', ('rb', 'ruby', 'duby'), ('*.rb', '*.rbw', 'Rakefile', '*.rake', '*.gemspec', '*.rbx', '*.duby', 'Gemfile'), ('text/x-ruby', 'application/x-ruby')), 'RustLexer': ('pygments.lexers.rust', 'Rust', ('rust',), ('*.rs',), ('text/rust',)), 'SLexer': ('pygments.lexers.r', 'S', ('splus', 's', 'r'), ('*.S', '*.R', '.Rhistory', '.Rprofile', '.Renviron'), ('text/S-plus', 'text/S', 'text/x-r-source', 'text/x-r', 'text/x-R', 'text/x-r-history', 'text/x-r-profile')), 'SMLLexer': ('pygments.lexers.ml', 'Standard ML', ('sml',), ('*.sml', '*.sig', '*.fun'), ('text/x-standardml', 'application/x-standardml')), @@ -325,7 +345,7 @@ 'SchemeLexer': ('pygments.lexers.lisp', 'Scheme', ('scheme', 'scm'), ('*.scm', '*.ss'), ('text/x-scheme', 'application/x-scheme')), 'ScilabLexer': ('pygments.lexers.matlab', 'Scilab', ('scilab',), ('*.sci', '*.sce', '*.tst'), ('text/scilab',)), 'ScssLexer': ('pygments.lexers.css', 'SCSS', ('scss',), ('*.scss',), ('text/x-scss',)), - 'ShellSessionLexer': ('pygments.lexers.shell', 'Shell Session', ('shell-session',), ('*.shell-session',), ('application/x-sh-session',)), + 'ShenLexer': ('pygments.lexers.lisp', 'Shen', ('shen',), ('*.shen',), ('text/x-shen', 'application/x-shen')), 'SlimLexer': ('pygments.lexers.webmisc', 'Slim', ('slim',), ('*.slim',), ('text/x-slim',)), 'SmaliLexer': ('pygments.lexers.dalvik', 'Smali', ('smali',), ('*.smali',), ('text/smali',)), 'SmalltalkLexer': ('pygments.lexers.smalltalk', 'Smalltalk', ('smalltalk', 'squeak', 'st'), ('*.st',), ('text/x-smalltalk',)), @@ -339,17 +359,22 @@ 'SquidConfLexer': ('pygments.lexers.configs', 'SquidConf', ('squidconf', 'squid.conf', 'squid'), ('squid.conf',), ('text/x-squidconf',)), 'SspLexer': ('pygments.lexers.templates', 'Scalate Server Page', ('ssp',), ('*.ssp',), ('application/x-ssp',)), 'StanLexer': ('pygments.lexers.modeling', 'Stan', ('stan',), ('*.stan',), ()), + 'SuperColliderLexer': ('pygments.lexers.supercollider', 'SuperCollider', ('sc', 'supercollider'), ('*.sc', '*.scd'), ('application/supercollider', 'text/supercollider')), 'SwiftLexer': ('pygments.lexers.objective', 'Swift', ('swift',), ('*.swift',), ('text/x-swift',)), 'SwigLexer': ('pygments.lexers.c_like', 'SWIG', ('swig',), ('*.swg', '*.i'), ('text/swig',)), 'SystemVerilogLexer': ('pygments.lexers.hdl', 'systemverilog', ('systemverilog', 'sv'), ('*.sv', '*.svh'), ('text/x-systemverilog',)), + 'TAPLexer': ('pygments.lexers.tap', 'TAP', ('tap',), ('*.tap',), ()), 'Tads3Lexer': ('pygments.lexers.int_fiction', 'TADS 3', ('tads3',), ('*.t',), ()), 'TclLexer': ('pygments.lexers.tcl', 'Tcl', ('tcl',), ('*.tcl', '*.rvt'), ('text/x-tcl', 'text/x-script.tcl', 'application/x-tcl')), 'TcshLexer': ('pygments.lexers.shell', 'Tcsh', ('tcsh', 'csh'), ('*.tcsh', '*.csh'), ('application/x-csh',)), + 'TcshSessionLexer': ('pygments.lexers.shell', 'Tcsh Session', ('tcshcon',), (), ()), 'TeaTemplateLexer': ('pygments.lexers.templates', 'Tea', ('tea',), ('*.tea',), ('text/x-tea',)), + 'TerraformLexer': ('pygments.lexers.configs', 'Terraform', ('terraform', 'tf'), ('*.tf',), ('application/x-tf', 'application/x-terraform')), 'TexLexer': ('pygments.lexers.markup', 'TeX', ('tex', 'latex'), ('*.tex', '*.aux', '*.toc'), ('text/x-tex', 'text/x-latex')), 'TextLexer': ('pygments.lexers.special', 'Text only', ('text',), ('*.txt',), ('text/plain',)), 'TodotxtLexer': ('pygments.lexers.textfmts', 'Todotxt', ('todotxt',), ('todo.txt', '*.todotxt'), ('text/x-todo',)), 'TreetopLexer': ('pygments.lexers.parsers', 'Treetop', ('treetop',), ('*.treetop', '*.tt'), ()), + 'TurtleLexer': ('pygments.lexers.rdf', 'Turtle', ('turtle',), ('*.ttl',), ('text/turtle', 'application/x-turtle')), 'TwigHtmlLexer': ('pygments.lexers.templates', 'HTML+Twig', ('html+twig',), ('*.twig',), ('text/html+twig',)), 'TwigLexer': ('pygments.lexers.templates', 'Twig', ('twig',), (), ('application/x-twig',)), 'TypeScriptLexer': ('pygments.lexers.javascript', 'TypeScript', ('ts',), ('*.ts',), ('text/x-typescript',)), @@ -365,6 +390,7 @@ 'VerilogLexer': ('pygments.lexers.hdl', 'verilog', ('verilog', 'v'), ('*.v',), ('text/x-verilog',)), 'VhdlLexer': ('pygments.lexers.hdl', 'vhdl', ('vhdl',), ('*.vhdl', '*.vhd'), ('text/x-vhdl',)), 'VimLexer': ('pygments.lexers.textedit', 'VimL', ('vim',), ('*.vim', '.vimrc', '.exrc', '.gvimrc', '_vimrc', '_exrc', '_gvimrc', 'vimrc', 'gvimrc'), ('text/x-vim',)), + 'X10Lexer': ('pygments.lexers.x10', 'X10', ('x10', 'xten'), ('*.x10',), ('text/x-x10',)), 'XQueryLexer': ('pygments.lexers.webmisc', 'XQuery', ('xquery', 'xqy', 'xq', 'xql', 'xqm'), ('*.xqy', '*.xquery', '*.xq', '*.xql', '*.xqm'), ('text/xquery', 'application/xquery')), 'XmlDjangoLexer': ('pygments.lexers.templates', 'XML+Django/Jinja', ('xml+django', 'xml+jinja'), (), ('application/xml+django', 'application/xml+jinja')), 'XmlErbLexer': ('pygments.lexers.templates', 'XML+Ruby', ('xml+erb', 'xml+ruby'), (), ('application/xml+ruby',)),
pygments/lexers/modeling.py+4 −2 modified@@ -284,8 +284,8 @@ class StanLexer(RegexLexer): """Pygments Lexer for Stan models. The Stan modeling language is specified in the *Stan Modeling Language - User's Guide and Reference Manual, v2.7.0*, - `pdf <https://github.com/stan-dev/stan/releases/download/v2.7.0/stan-reference-2.7.0.pdf>`__. + User's Guide and Reference Manual, v2.8.0*, + `pdf <https://github.com/stan-dev/stan/releases/download/v2.8.8/stan-reference-2.8.0.pdf>`__. .. versionadded:: 1.6 """ @@ -332,6 +332,8 @@ class StanLexer(RegexLexer): # Special names ending in __, like lp__ (r'[A-Za-z]\w*__\b', Name.Builtin.Pseudo), (r'(%s)\b' % r'|'.join(_stan_builtins.RESERVED), Keyword.Reserved), + # user-defined functions + (r'[A-Za-z]\w*(?=\s*\()]', Name.Function), # Regular variable names (r'[A-Za-z]\w*\b', Name), # Real Literals
pygments/lexers/oberon.py+105 −0 added@@ -0,0 +1,105 @@ +# -*- coding: utf-8 -*- +""" + pygments.lexers.oberon + ~~~~~~~~~~~~~~~~~~~~~~ + + Lexers for Oberon family languages. + + :copyright: Copyright 2006-2015 by the Pygments team, see AUTHORS. + :license: BSD, see LICENSE for details. +""" + +import re + +from pygments.lexer import RegexLexer, include, words +from pygments.token import Text, Comment, Operator, Keyword, Name, String, \ + Number, Punctuation + +__all__ = ['ComponentPascalLexer'] + + +class ComponentPascalLexer(RegexLexer): + """ + For `Component Pascal <http://www.oberon.ch/pdf/CP-Lang.pdf>`_ source code. + + .. versionadded:: 2.1 + """ + name = 'Component Pascal' + aliases = ['componentpascal', 'cp'] + filenames = ['*.cp', '*.cps'] + mimetypes = ['text/x-component-pascal'] + + flags = re.MULTILINE | re.DOTALL + + tokens = { + 'root': [ + include('whitespace'), + include('comments'), + include('punctuation'), + include('numliterals'), + include('strings'), + include('operators'), + include('builtins'), + include('identifiers'), + ], + 'whitespace': [ + (r'\n+', Text), # blank lines + (r'\s+', Text), # whitespace + ], + 'comments': [ + (r'\(\*([^\$].*?)\*\)', Comment.Multiline), + # TODO: nested comments (* (* ... *) ... (* ... *) *) not supported! + ], + 'punctuation': [ + (r'[\(\)\[\]\{\},.:;\|]', Punctuation), + ], + 'numliterals': [ + (r'[0-9A-F]+X\b', Number.Hex), # char code + (r'[0-9A-F]+[HL]\b', Number.Hex), # hexadecimal number + (r'[0-9]+\.[0-9]+E[+-][0-9]+', Number.Float), # real number + (r'[0-9]+\.[0-9]+', Number.Float), # real number + (r'[0-9]+', Number.Integer), # decimal whole number + ], + 'strings': [ + (r"'[^\n']*'", String), # single quoted string + (r'"[^\n"]*"', String), # double quoted string + ], + 'operators': [ + # Arithmetic Operators + (r'[+-]', Operator), + (r'[*/]', Operator), + # Relational Operators + (r'[=#<>]', Operator), + # Dereferencing Operator + (r'\^', Operator), + # Logical AND Operator + (r'&', Operator), + # Logical NOT Operator + (r'~', Operator), + # Assignment Symbol + (r':=', Operator), + # Range Constructor + (r'\.\.', Operator), + (r'\$', Operator), + ], + 'identifiers': [ + (r'([a-zA-Z_\$][\w\$]*)', Name), + ], + 'builtins': [ + (words(( + 'ANYPTR', 'ANYREC', 'BOOLEAN', 'BYTE', 'CHAR', 'INTEGER', 'LONGINT', + 'REAL', 'SET', 'SHORTCHAR', 'SHORTINT', 'SHORTREAL' + ), suffix=r'\b'), Keyword.Type), + (words(( + 'ABS', 'ABSTRACT', 'ARRAY', 'ASH', 'ASSERT', 'BEGIN', 'BITS', 'BY', + 'CAP', 'CASE', 'CHR', 'CLOSE', 'CONST', 'DEC', 'DIV', 'DO', 'ELSE', + 'ELSIF', 'EMPTY', 'END', 'ENTIER', 'EXCL', 'EXIT', 'EXTENSIBLE', 'FOR', + 'HALT', 'IF', 'IMPORT', 'IN', 'INC', 'INCL', 'IS', 'LEN', 'LIMITED', + 'LONG', 'LOOP', 'MAX', 'MIN', 'MOD', 'MODULE', 'NEW', 'ODD', 'OF', + 'OR', 'ORD', 'OUT', 'POINTER', 'PROCEDURE', 'RECORD', 'REPEAT', 'RETURN', + 'SHORT','SHORTCHAR', 'SHORTINT', 'SIZE', 'THEN', 'TYPE', 'TO', 'UNTIL', + 'VAR', 'WHILE', 'WITH' + ), suffix=r'\b'), Keyword.Reserved), + (r'(TRUE|FALSE|NIL|INF)\b', Keyword.Constant), + ] + }
pygments/lexers/parasail.py+81 −0 added@@ -0,0 +1,81 @@ +# -*- coding: utf-8 -*- +""" + pygments.lexers.parasail + ~~~~~~~~~~~~~~~~~~~~~~~~ + + Lexer for ParaSail. + + :copyright: Copyright 2006-2015 by the Pygments team, see AUTHORS. + :license: BSD, see LICENSE for details. +""" + +import re + +from pygments.lexer import Lexer, RegexLexer, include, bygroups, using, \ + this, combined, inherit, do_insertions, default +from pygments.util import get_bool_opt, get_list_opt +from pygments.token import Text, Comment, Operator, Keyword, Name, String, \ + Number, Punctuation, Literal + +__all__ = ['ParaSailLexer'] + + +class ParaSailLexer(RegexLexer): + """ + For `ParaSail <http://www.parasail-lang.org>`_ source code. + + .. versionadded:: 2.1 + """ + + name = 'ParaSail' + aliases = ['parasail'] + filenames = ['*.psi', '*.psl'] + mimetypes = ['text/x-parasail'] + + flags = re.MULTILINE + + tokens = { + 'root': [ + (r'[^\S\n]+', Text), + (r'//.*?\n', Comment.Single), + (r'\b(and|or|xor)=', Operator.Word), + (r'\b(and(\s+then)?|or(\s+else)?|xor|rem|mod|' + r'(is|not)\s+null)\b', + Operator.Word), + # Keywords + (r'\b(abs|abstract|all|block|class|concurrent|const|continue|' + r'each|end|exit|extends|exports|forward|func|global|implements|' + r'import|in|interface|is|lambda|locked|new|not|null|of|op|' + r'optional|private|queued|ref|return|reverse|separate|some|' + r'type|until|var|with|' + # Control flow + r'if|then|else|elsif|case|for|while|loop)\b', + Keyword.Reserved), + (r'(abstract\s+)?(interface|class|op|func|type)', + Keyword.Declaration), + # Literals + (r'"[^"]*"', String), + (r'\\[\'ntrf"0]', String.Escape), + (r'#[a-zA-Z]\w*', Literal), #Enumeration + include('numbers'), + (r"'[^']'", String.Char), + (r'[a-zA-Z]\w*', Name), + # Operators and Punctuation + (r'(<==|==>|<=>|\*\*=|<\|=|<<=|>>=|==|!=|=\?|<=|>=|' + r'\*\*|<<|>>|=>|:=|\+=|-=|\*=|\||\|=|/=|\+|-|\*|/|' + r'\.\.|<\.\.|\.\.<|<\.\.<)', + Operator), + (r'(<|>|\[|\]|\(|\)|\||:|;|,|.|\{|\}|->)', + Punctuation), + (r'\n+', Text), + ], + 'numbers' : [ + (r'\d[0-9_]*#[0-9a-fA-F][0-9a-fA-F_]*#', Number.Hex), # any base + (r'0[xX][0-9a-fA-F][0-9a-fA-F_]*', Number.Hex), # C-like hex + (r'0[bB][01][01_]*', Number.Bin), # C-like bin + (r'\d[0-9_]*\.\d[0-9_]*[eE][+-]\d[0-9_]*', # float exp + Number.Float), + (r'\d[0-9_]*\.\d[0-9_]*', Number.Float), # float + (r'\d[0-9_]*', Number.Integer), # integer + ], + }
pygments/lexers/praat.py+286 −0 added@@ -0,0 +1,286 @@ +# -*- coding: utf-8 -*- +""" + pygments.lexers.praat + ~~~~~~~~~~~~~~~~~~~~~ + + Lexer for Praat + + :copyright: Copyright 2006-2015 by the Pygments team, see AUTHORS. + :license: BSD, see LICENSE for details. +""" + +import re + +from pygments.lexer import RegexLexer, words, bygroups, default, include +from pygments.token import * + +__all__ = ['PraatLexer'] + + +class PraatLexer(RegexLexer): + """ + For `Praat <http://www.praat.org>`_ scripts. + + .. versionadded:: 2.1 + """ + + name = 'Praat' + aliases = ['praat'] + filenames = ['*.praat', '*.proc', '*.psc'] + + keywords = [ + 'if', 'then', 'else', 'elsif', 'elif', 'endif', 'fi', 'for', 'from', 'to', 'endfor', 'endproc', + 'while', 'endwhile', 'repeat', 'until', 'select', 'plus', 'minus', 'demo', 'assert', 'stopwatch', + 'nocheck', 'nowarn', 'noprogress', 'editor', 'endeditor', 'clearinfo' + ] + + functions_string = [ + 'backslashTrigraphsToUnicode', 'chooseDirectory', 'chooseReadFile', + 'chooseWriteFile', 'date', 'demoKey', 'do', 'environment', 'extractLine', 'extractWord', + 'fixed', 'info', 'left', 'mid', 'percent', 'readFile', 'replace', 'replace_regex', 'right', + 'selected', 'string', 'unicodeToBackslashTrigraphs', + ] + + functions_numeric = [ + 'abs', 'appendFile', 'appendFileLine', 'appendInfo', 'appendInfoLine', 'arccos', 'arccosh', + 'arcsin', 'arcsinh', 'arctan', 'arctan2', 'arctanh', 'barkToHertz', 'beginPause', + 'beginSendPraat', 'besselI', 'besselK', 'beta', 'beta2', 'binomialP', 'binomialQ', 'boolean', + 'ceiling', 'chiSquareP', 'chiSquareQ', 'choice', 'comment', 'cos', 'cosh', 'createDirectory', + 'deleteFile', 'demoClicked', 'demoClickedIn', 'demoCommandKeyPressed', + 'demoExtraControlKeyPressed', 'demoInput', 'demoKeyPressed', + 'demoOptionKeyPressed', 'demoShiftKeyPressed', 'demoShow', 'demoWaitForInput', + 'demoWindowTitle', 'demoX', 'demoY', 'differenceLimensToPhon', 'do', 'editor', 'endPause', + 'endSendPraat', 'endsWith', 'erb', 'erbToHertz', 'erf', 'erfc', 'exitScript', 'exp', + 'extractNumber', 'fileReadable', 'fisherP', 'fisherQ', 'floor', 'gaussP', 'gaussQ', + 'hertzToBark', 'hertzToErb', 'hertzToMel', 'hertzToSemitones', 'imax', 'imin', + 'incompleteBeta', 'incompleteGammaP', 'index', 'index_regex', 'invBinomialP', + 'invBinomialQ', 'invChiSquareQ', 'invFisherQ', 'invGaussQ', 'invSigmoid', 'invStudentQ', + 'length', 'ln', 'lnBeta', 'lnGamma', 'log10', 'log2', 'max', 'melToHertz', 'min', 'minusObject', + 'natural', 'number', 'numberOfColumns', 'numberOfRows', 'numberOfSelected', + 'objectsAreIdentical', 'option', 'optionMenu', 'pauseScript', + 'phonToDifferenceLimens', 'plusObject', 'positive', 'randomBinomial', 'randomGauss', + 'randomInteger', 'randomPoisson', 'randomUniform', 'real', 'readFile', 'removeObject', + 'rindex', 'rindex_regex', 'round', 'runScript', 'runSystem', 'runSystem_nocheck', + 'selectObject', 'selected', 'semitonesToHertz', 'sentencetext', 'sigmoid', 'sin', 'sinc', + 'sincpi', 'sinh', 'soundPressureToPhon', 'sqrt', 'startsWith', 'studentP', 'studentQ', 'tan', + 'tanh', 'variableExists', 'word', 'writeFile', 'writeFileLine', 'writeInfo', + 'writeInfoLine', + ] + + functions_array = [ + 'linear', 'randomGauss', 'randomInteger', 'randomUniform', 'zero', + ] + + objects = [ + 'Activation', 'AffineTransform', 'AmplitudeTier', 'Art', 'Artword', 'Autosegment', + 'BarkFilter', 'BarkSpectrogram', 'CCA', 'Categories', 'Cepstrogram', 'Cepstrum', + 'Cepstrumc', 'ChebyshevSeries', 'ClassificationTable', 'Cochleagram', 'Collection', + 'ComplexSpectrogram', 'Configuration', 'Confusion', 'ContingencyTable', 'Corpus', + 'Correlation', 'Covariance', 'CrossCorrelationTable', 'CrossCorrelationTables', 'DTW', + 'DataModeler', 'Diagonalizer', 'Discriminant', 'Dissimilarity', 'Distance', + 'Distributions', 'DurationTier', 'EEG', 'ERP', 'ERPTier', 'EditCostsTable', + 'EditDistanceTable', 'Eigen', 'Excitation', 'Excitations', 'ExperimentMFC', 'FFNet', + 'FeatureWeights', 'FileInMemory', 'FilesInMemory', 'Formant', 'FormantFilter', + 'FormantGrid', 'FormantModeler', 'FormantPoint', 'FormantTier', 'GaussianMixture', 'HMM', + 'HMM_Observation', 'HMM_ObservationSequence', 'HMM_State', 'HMM_StateSequence', + 'Harmonicity', 'ISpline', 'Index', 'Intensity', 'IntensityTier', 'IntervalTier', 'KNN', + 'KlattGrid', 'KlattTable', 'LFCC', 'LPC', 'Label', 'LegendreSeries', 'LinearRegression', + 'LogisticRegression', 'LongSound', 'Ltas', 'MFCC', 'MSpline', 'ManPages', 'Manipulation', + 'Matrix', 'MelFilter', 'MelSpectrogram', 'MixingMatrix', 'Movie', 'Network', 'OTGrammar', + 'OTHistory', 'OTMulti', 'PCA', 'PairDistribution', 'ParamCurve', 'Pattern', 'Permutation', + 'Photo', 'Pitch', 'PitchModeler', 'PitchTier', 'PointProcess', 'Polygon', 'Polynomial', + 'PowerCepstrogram', 'PowerCepstrum', 'Procrustes', 'RealPoint', 'RealTier', 'ResultsMFC', + 'Roots', 'SPINET', 'SSCP', 'SVD', 'Salience', 'ScalarProduct', 'Similarity', 'SimpleString', + 'SortedSetOfString', 'Sound', 'Speaker', 'Spectrogram', 'Spectrum', 'SpectrumTier', + 'SpeechSynthesizer', 'SpellingChecker', 'Strings', 'StringsIndex', 'Table', + 'TableOfReal', 'TextGrid', 'TextInterval', 'TextPoint', 'TextTier', 'Tier', 'Transition', + 'VocalTract', 'VocalTractTier', 'Weight', 'WordList', + ] + + variables_numeric = [ + 'macintosh', 'windows', 'unix', 'praatVersion', 'pi', 'e', 'undefined', + ] + + variables_string = [ + 'praatVersion', 'tab', 'shellDirectory', 'homeDirectory', + 'preferencesDirectory', 'newline', 'temporaryDirectory', + 'defaultDirectory', + ] + + tokens = { + 'root': [ + (r'(\s+)(#.*?$)', bygroups(Text, Comment.Single)), + (r'^#.*?$', Comment.Single), + (r';[^\n]*', Comment.Single), + (r'\s+', Text), + + (r'\bprocedure\b', Keyword, 'procedure_definition'), + (r'\bcall\b', Keyword, 'procedure_call'), + (r'@', Name.Function, 'procedure_call'), + + include('function_call'), + + (words(keywords, suffix=r'\b'), Keyword), + + (r'(\bform\b)(\s+)([^\n]+)', + bygroups(Keyword, Text, String), 'old_form'), + + (r'(print(?:line|tab)?|echo|exit|asserterror|pause|send(?:praat|socket)|include|execute|system(?:_nocheck)?)(\s+)', + bygroups(Keyword, Text), 'string_unquoted'), + + (r'(goto|label)(\s+)(\w+)', bygroups(Keyword, Text, Name.Label)), + + include('variable_name'), + include('number'), + + (r'"', String, 'string'), + + (words((objects), suffix=r'(?=\s+\S+\n)'), Name.Class, 'string_unquoted'), + + (r'\b[A-Z]', Keyword, 'command'), + (r'(\.{3}|[)(,])', Punctuation), + ], + 'command': [ + (r'( ?[\w()-]+ ?)', Keyword), + (r"'(?=.*')", String.Interpol, 'string_interpolated'), + (r'\.{3}', Keyword, ('#pop', 'old_arguments')), + (r':', Keyword, ('#pop', 'comma_list')), + (r'[\s\n]', Text, '#pop'), + ], + 'procedure_call': [ + (r'\s+', Text), + (r'([\w.]+)(:|\s*\()', + bygroups(Name.Function, Text), '#pop'), + (r'([\w.]+)', Name.Function, ('#pop', 'old_arguments')), + ], + 'procedure_definition': [ + (r'\s', Text), + (r'([\w.]+)(\s*?[(:])', + bygroups(Name.Function, Text), '#pop'), + (r'([\w.]+)([^\n]*)', + bygroups(Name.Function, Text), '#pop'), + ], + 'function_call': [ + (words(functions_string , suffix=r'\$(?=\s*[:(])'), Name.Function, 'function'), + (words(functions_array , suffix=r'#(?=\s*[:(])'), Name.Function, 'function'), + (words(functions_numeric, suffix=r'(?=\s*[:(])'), Name.Function, 'function'), + ], + 'function': [ + (r'\s+', Text), + (r':', Punctuation, ('#pop', 'comma_list')), + (r'\s*\(', Punctuation, ('#pop', 'comma_list')), + ], + 'comma_list': [ + (r'(\s*\n\s*)(\.{3})', bygroups(Text, Punctuation)), + + (r'(\s*[])\n])', Text, '#pop'), + + (r'\s+', Text), + (r'"', String, 'string'), + (r'\b(if|then|else|fi|endif)\b', Keyword), + + include('function_call'), + include('variable_name'), + include('operator'), + include('number'), + + (r',', Punctuation), + ], + 'old_arguments': [ + (r'\n', Text, '#pop'), + + include('variable_name'), + include('operator'), + include('number'), + + (r'"', String, 'string'), + (r'[^\n]', Text), + ], + 'number': [ + (r'\b\d+(\.\d*)?([eE][-+]?\d+)?%?', Number), + ], + 'object_attributes': [ + (r'\.?(n(col|row)|[xy]min|[xy]max|[nd][xy])\b', Name.Builtin, '#pop'), + (r'(\.?(?:col|row)\$)(\[)', + bygroups(Name.Builtin, Text), 'variable_name'), + (r'(\$?)(\[)', + bygroups(Name.Builtin, Text), ('#pop', 'comma_list')), + ], + 'variable_name': [ + include('operator'), + include('number'), + + (words(variables_string, suffix=r'\$'), Name.Variable.Global), + (words(variables_numeric, suffix=r'\b'), Name.Variable.Global), + + (r'\bObject_\w+', Name.Builtin, 'object_attributes'), + (words(objects, prefix=r'\b', suffix=r'_\w+'), Name.Builtin, 'object_attributes'), + + (r"\b(Object_)(')", + bygroups(Name.Builtin, String.Interpol), + ('object_attributes', 'string_interpolated')), + (words(objects, prefix=r'\b', suffix=r"(_)(')"), + bygroups(Name.Builtin, Name.Builtin, String.Interpol), + ('object_attributes', 'string_interpolated')), + + (r'\.?_?[a-z][a-zA-Z0-9_.]*(\$|#)?', Text), + (r'[\[\]]', Punctuation, 'comma_list'), + (r"'(?=.*')", String.Interpol, 'string_interpolated'), + ], + 'operator': [ + (r'([+\/*<>=!-]=?|[&*|][&*|]?|\^|<>)', Operator), + (r'\b(and|or|not|div|mod)\b', Operator.Word), + ], + 'string_interpolated': [ + (r'\.?[_a-z][a-zA-Z0-9_.]*[\$#]?(?:\[[a-zA-Z0-9,]+\])?(:[0-9]+)?', String.Interpol), + (r"'", String.Interpol, '#pop'), + ], + 'string_unquoted': [ + (r'(\n\s*)(\.{3})', bygroups(Text, Punctuation)), + + (r'\n', Text, '#pop'), + (r'\s', Text), + (r"'(?=.*')", String.Interpol, 'string_interpolated'), + (r"'", String), + (r"[^'\n]+", String), + ], + 'string': [ + (r'(\n\s*)(\.{3})', bygroups(Text, Punctuation)), + + (r'"', String, '#pop'), + (r"'(?=.*')", String.Interpol, 'string_interpolated'), + (r"'", String), + (r'[^\'"\n]+', String), + ], + 'old_form': [ + (r'\s+', Text), + + (r'(optionmenu|choice)([ \t]+\S+:[ \t]+)', + bygroups(Keyword, Text), 'number'), + + (r'(option|button)([ \t]+)', + bygroups(Keyword, Text), 'number'), + + (r'(option|button)([ \t]+)', + bygroups(Keyword, Text), 'string_unquoted'), + + (r'(sentence|text)([ \t]+\S+)', + bygroups(Keyword, Text), 'string_unquoted'), + + (r'(word)([ \t]+\S+[ \t]*)(\S+)?([ \t]+.*)?', + bygroups(Keyword, Text, String, Text)), + + (r'(boolean)(\s+\S+\s*)(0|1|"?(?:yes|no)"?)', + bygroups(Keyword, Text, Name.Variable)), + + # Ideally processing of the number would happend in the 'number' + # but that doesn't seem to work + (r'(real|natural|positive|integer)([ \t]+\S+[ \t]*)([+-]?)(\d+(?:\.\d*)?(?:[eE][-+]?\d+)?%?)', + bygroups(Keyword, Text, Operator, Number)), + + (r'(comment)(\s+)', + bygroups(Keyword, Text), 'string_unquoted'), + + (r'\bendform\b', Keyword, '#pop'), + ] + }
pygments/lexers/prolog.py+4 −4 modified@@ -155,11 +155,11 @@ class LogtalkLexer(RegexLexer): # Term creation and decomposition (r'(functor|arg|copy_term|numbervars|term_variables)(?=[(])', Keyword), # Evaluable functors - (r'(rem|m(ax|in|od)|abs|sign)(?=[(])', Keyword), + (r'(div|rem|m(ax|in|od)|abs|sign)(?=[(])', Keyword), (r'float(_(integer|fractional)_part)?(?=[(])', Keyword), - (r'(floor|truncate|round|ceiling)(?=[(])', Keyword), + (r'(floor|t(an|runcate)|round|ceiling)(?=[(])', Keyword), # Other arithmetic functors - (r'(cos|a(cos|sin|tan)|exp|log|s(in|qrt))(?=[(])', Keyword), + (r'(cos|a(cos|sin|tan|tan2)|exp|log|s(in|qrt)|xor)(?=[(])', Keyword), # Term testing (r'(var|atom(ic)?|integer|float|c(allable|ompound)|n(onvar|umber)|' r'ground|acyclic_term)(?=[(])', Keyword), @@ -212,7 +212,7 @@ class LogtalkLexer(RegexLexer): (r'(==|\\==|@=<|@<|@>=|@>)', Operator), # Evaluable functors (r'(//|[-+*/])', Operator), - (r'\b(e|pi|mod|rem)\b', Operator), + (r'\b(e|pi|div|mod|rem)\b', Operator), # Other arithemtic functors (r'\b\*\*\b', Operator), # DCG rules
pygments/lexers/rdf.py+179 −23 modified@@ -12,10 +12,10 @@ import re from pygments.lexer import RegexLexer, bygroups, default -from pygments.token import Keyword, Punctuation, String, Number, Operator, \ +from pygments.token import Keyword, Punctuation, String, Number, Operator, Generic, \ Whitespace, Name, Literal, Comment, Text -__all__ = ['SparqlLexer'] +__all__ = ['SparqlLexer', 'TurtleLexer'] class SparqlLexer(RegexLexer): @@ -29,39 +29,190 @@ class SparqlLexer(RegexLexer): filenames = ['*.rq', '*.sparql'] mimetypes = ['application/sparql-query'] - flags = re.IGNORECASE + # terminal productions :: + + PN_CHARS_BASE = (u'(?:[a-zA-Z' + u'\u00c0-\u00d6' + u'\u00d8-\u00f6' + u'\u00f8-\u02ff' + u'\u0370-\u037d' + u'\u037f-\u1fff' + u'\u200c-\u200d' + u'\u2070-\u218f' + u'\u2c00-\u2fef' + u'\u3001-\ud7ff' + u'\uf900-\ufdcf' + u'\ufdf0-\ufffd]|' + u'[^\u0000-\uffff]|' + u'[\ud800-\udbff][\udc00-\udfff])') + + PN_CHARS_U = '(?:' + PN_CHARS_BASE + '|_)' + + PN_CHARS = ('(?:' + PN_CHARS_U + r'|[\-0-9' + + u'\u00b7' + + u'\u0300-\u036f' + + u'\u203f-\u2040])') + + HEX = '[0-9A-Fa-f]' + + PN_LOCAL_ESC_CHARS = r'[ _~.\-!$&""()*+,;=/?#@%]' + + IRIREF = r'<(?:[^<>"{}|^`\\\x00-\x20])*>' + + BLANK_NODE_LABEL = '_:(?:' + PN_CHARS_U + '|[0-9])(?:(?:' + PN_CHARS + '|\.)*' + PN_CHARS + ')?' + + PN_PREFIX = PN_CHARS_BASE + '(?:(?:' + PN_CHARS + '|\.)*' + PN_CHARS + ')?' + + VARNAME = '(?:' + PN_CHARS_U + '|[0-9])(?:' + PN_CHARS_U + u'|[0-9\u00b7\u0300-\u036f\u203f-\u2040])*' + + PERCENT = '%' + HEX + HEX + + PN_LOCAL_ESC = r'\\' + PN_LOCAL_ESC_CHARS + + PLX = '(?:' + PERCENT + ')|(?:' + PN_LOCAL_ESC + ')' + + PN_LOCAL = ('(?:(?:' + PN_CHARS_U + '|[:0-9])|' + PLX + ')' + + '(?:(?:(?:' + PN_CHARS + '|[.:])|' + PLX + ')*(?:(?:' + PN_CHARS + '|:)|' + PLX + '))?') + + EXPONENT = r'[eE][+-]?\d+' + + # Lexer token definitions :: tokens = { 'root': [ - (r'\s+', Whitespace), - (r'(select|construct|describe|ask|where|filter|group\s+by|minus|' - r'distinct|reduced|from named|from|order\s+by|limit|' + (r'\s+', Text), + # keywords :: + (r'((?i)select|construct|describe|ask|where|filter|group\s+by|minus|' + r'distinct|reduced|from\s+named|from|order\s+by|desc|asc|limit|' r'offset|bindings|load|clear|drop|create|add|move|copy|' r'insert\s+data|delete\s+data|delete\s+where|delete|insert|' - r'using named|using|graph|default|named|all|optional|service|' - r'silent|bind|union|not in|in|as|a)', Keyword), - (r'(prefix|base)(\s+)([a-z][\w-]*)(\s*)(\:)', - bygroups(Keyword, Whitespace, Name.Namespace, Whitespace, - Punctuation)), - (r'\?[a-z_]\w*', Name.Variable), - (r'<[^>]+>', Name.Label), - (r'([a-z][\w-]*)(\:)([a-z][\w-]*)', + r'using\s+named|using|graph|default|named|all|optional|service|' + r'silent|bind|union|not\s+in|in|as|having|to|prefix|base)\b', Keyword), + (r'(a)\b', Keyword), + # IRIs :: + ('(' + IRIREF + ')', Name.Label), + # blank nodes :: + ('(' + BLANK_NODE_LABEL + ')', Name.Label), + # # variables :: + ('[?$]' + VARNAME, Name.Variable), + # prefixed names :: + (r'(' + PN_PREFIX + ')?(\:)(' + PN_LOCAL + ')?', bygroups(Name.Namespace, Punctuation, Name.Tag)), - (r'(str|lang|langmatches|datatype|bound|iri|uri|bnode|rand|abs|' + # function names :: + (r'((?i)str|lang|langmatches|datatype|bound|iri|uri|bnode|rand|abs|' r'ceil|floor|round|concat|strlen|ucase|lcase|encode_for_uri|' r'contains|strstarts|strends|strbefore|strafter|year|month|day|' r'hours|minutes|seconds|timezone|tz|now|md5|sha1|sha256|sha384|' r'sha512|coalesce|if|strlang|strdt|sameterm|isiri|isuri|isblank|' - r'isliteral|isnumeric|regex|substr|replace|exists|not exists|' + r'isliteral|isnumeric|regex|substr|replace|exists|not\s+exists|' r'count|sum|min|max|avg|sample|group_concat|separator)\b', Name.Function), - (r'(true|false)', Literal), + # boolean literals :: + (r'(true|false)', Keyword.Constant), + # double literals :: + (r'[+\-]?(\d+\.\d*' + EXPONENT + '|\.?\d+' + EXPONENT + ')', Number.Float), + # decimal literals :: + (r'[+\-]?(\d+\.\d*|\.\d+)', Number.Float), + # integer literals :: + (r'[+\-]?\d+', Number.Integer), + # operators :: + (r'(\|\||&&|=|\*|\-|\+|/|!=|<=|>=|!|<|>)', Operator), + # punctuation characters :: + (r'[(){}.;,:^\[\]]', Punctuation), + # line comments :: + (r'#[^\n]*', Comment), + # strings :: + (r'"""', String, 'triple-double-quoted-string'), + (r'"', String, 'single-double-quoted-string'), + (r"'''", String, 'triple-single-quoted-string'), + (r"'", String, 'single-single-quoted-string'), + ], + 'triple-double-quoted-string': [ + (r'"""', String, 'end-of-string'), + (r'[^\\]+', String), + (r'\\', String, 'string-escape'), + ], + 'single-double-quoted-string': [ + (r'"', String, 'end-of-string'), + (r'[^"\\\n]+', String), + (r'\\', String, 'string-escape'), + ], + 'triple-single-quoted-string': [ + (r"'''", String, 'end-of-string'), + (r'[^\\]+', String), + (r'\\', String.Escape, 'string-escape'), + ], + 'single-single-quoted-string': [ + (r"'", String, 'end-of-string'), + (r"[^'\\\n]+", String), + (r'\\', String, 'string-escape'), + ], + 'string-escape': [ + (r'u' + HEX + '{4}', String.Escape, '#pop'), + (r'U' + HEX + '{8}', String.Escape, '#pop'), + (r'.', String.Escape, '#pop'), + ], + 'end-of-string': [ + (r'(@)([a-zA-Z]+(?:-[a-zA-Z0-9]+)*)', + bygroups(Operator, Name.Function), '#pop:2'), + (r'\^\^', Operator, '#pop:2'), + default('#pop:2'), + ], + } + + +class TurtleLexer(RegexLexer): + """ + Lexer for `Turtle <http://www.w3.org/TR/turtle/>`_ data language. + + .. versionadded:: 2.1 + """ + name = 'Turtle' + aliases = ['turtle'] + filenames = ['*.ttl'] + mimetypes = ['text/turtle', 'application/x-turtle'] + + flags = re.IGNORECASE + + patterns = { + 'PNAME_NS': r'((?:[a-zA-Z][\w-]*)?\:)', # Simplified character range + 'IRIREF': r'(<[^<>"{}|^`\\\x00-\x20]*>)' + } + + # PNAME_NS PN_LOCAL (with simplified character range) + patterns['PrefixedName'] = r'%(PNAME_NS)s([a-z][\w-]*)' % patterns + + tokens = { + 'root': [ + (r'\s+', Whitespace), + + # Base / prefix + (r'(@base|BASE)(\s+)%(IRIREF)s(\s*)(\.?)' % patterns, + bygroups(Keyword, Whitespace, Name.Variable, Whitespace, + Punctuation)), + (r'(@prefix|PREFIX)(\s+)%(PNAME_NS)s(\s+)%(IRIREF)s(\s*)(\.?)' % patterns, + bygroups(Keyword, Whitespace, Name.Namespace, Whitespace, + Name.Variable, Whitespace, Punctuation)), + + # The shorthand predicate 'a' + (r'(?<=\s)a(?=\s)', Keyword.Type), + + # IRIREF + (r'%(IRIREF)s' % patterns, Name.Variable), + + # PrefixedName + (r'%(PrefixedName)s' % patterns, + bygroups(Name.Namespace, Name.Tag)), + + # Comment + (r'#[^\n]+', Comment), + + (r'\b(true|false)\b', Literal), (r'[+\-]?\d*\.\d+', Number.Float), (r'[+\-]?\d*(:?\.\d+)?E[+\-]?\d+', Number.Float), (r'[+\-]?\d+', Number.Integer), - (r'(\|\||&&|=|\*|\-|\+|/)', Operator), - (r'[(){}.;,:^]', Punctuation), - (r'#[^\n]+', Comment), + (r'[\[\](){}.;,:^]', Punctuation), + (r'"""', String, 'triple-double-quoted-string'), (r'"', String, 'single-double-quoted-string'), (r"'''", String, 'triple-single-quoted-string'), @@ -91,9 +242,14 @@ class SparqlLexer(RegexLexer): (r'.', String, '#pop'), ], 'end-of-string': [ - (r'(@)([a-z]+(:?-[a-z0-9]+)*)', - bygroups(Operator, Name.Function), '#pop:2'), - (r'\^\^', Operator, '#pop:2'), + + (r'(@)([a-zA-Z]+(:?-[a-zA-Z0-9]+)*)', + bygroups(Operator, Generic.Emph), '#pop:2'), + + (r'(\^\^)%(IRIREF)s' % patterns, bygroups(Operator, Generic.Emph), '#pop:2'), + (r'(\^\^)%(PrefixedName)s' % patterns, bygroups(Operator, Generic.Emph, Generic.Emph), '#pop:2'), + default('#pop:2'), + ], }
pygments/lexers/roboconf.py+82 −0 added@@ -0,0 +1,82 @@ +# -*- coding: utf-8 -*- +""" + pygments.lexers.roboconf + ~~~~~~~~~~~~~~~~~~~~~~~~ + + Lexers for Roboconf DSL. + + :copyright: Copyright 2006-2015 by the Pygments team, see AUTHORS. + :license: BSD, see LICENSE for details. +""" + +from pygments.lexer import RegexLexer, words, bygroups, re, include +from pygments.token import Text, Operator, Keyword, Name, Comment + +__all__ = ['RoboconfGraphLexer', 'RoboconfInstancesLexer'] + + +class RoboconfGraphLexer(RegexLexer): + """ + Lexer for `Roboconf <http://roboconf.net/en/roboconf.html>`_ graph files. + + .. versadded:: 2.1 + """ + name = 'Roboconf Graph' + aliases = ['roboconf-graph'] + filenames = ['*.graph'] + + flags = re.IGNORECASE | re.MULTILINE + tokens = { + 'root': [ + # Skip white spaces + (r'\s+', Text), + + # There is one operator + (r'=',Operator), + + # Keywords + (words(('facet', 'import'), suffix=r'\s*\b', prefix=r'\b'), Keyword), + (words(( + 'installer', 'extends', 'exports', 'imports', 'facets', + 'children'), suffix=r'\s*:?', prefix=r'\b'), Name), + + # Comments + (r'#.*\n', Comment), + + # Default + (r'[^#]', Text), + (r'.*\n', Text) + ] + } + + +class RoboconfInstancesLexer(RegexLexer): + """ + Lexer for `Roboconf <http://roboconf.net/en/roboconf.html>`_ instances files. + + .. versadded:: 2.1 + """ + name = 'Roboconf Instances' + aliases = ['roboconf-instances'] + filenames = ['*.instances'] + + flags = re.IGNORECASE | re.MULTILINE + tokens = { + 'root': [ + + # Skip white spaces + (r'\s+', Text), + + # Keywords + (words(('instance of', 'import'), suffix=r'\s*\b', prefix=r'\b'), Keyword), + (words(('name', 'count'), suffix=r's*:?', prefix=r'\b'), Name), + (r'\s*[\w.-]+\s*:', Name), + + # Comments + (r'#.*\n', Comment), + + # Default + (r'[^#]', Text), + (r'.*\n', Text) + ] + }
pygments/lexers/ruby.py+1 −1 modified@@ -36,7 +36,7 @@ class RubyLexer(ExtendedRegexLexer): name = 'Ruby' aliases = ['rb', 'ruby', 'duby'] filenames = ['*.rb', '*.rbw', 'Rakefile', '*.rake', '*.gemspec', - '*.rbx', '*.duby'] + '*.rbx', '*.duby', 'Gemfile'] mimetypes = ['text/x-ruby', 'application/x-ruby'] flags = re.DOTALL | re.MULTILINE
pygments/lexers/scripting.py+275 −2 modified@@ -14,11 +14,12 @@ from pygments.lexer import RegexLexer, include, bygroups, default, combined, \ words from pygments.token import Text, Comment, Operator, Keyword, Name, String, \ - Number, Punctuation, Error, Whitespace + Number, Punctuation, Error, Whitespace, Other from pygments.util import get_bool_opt, get_list_opt, iteritems __all__ = ['LuaLexer', 'MoonScriptLexer', 'ChaiscriptLexer', 'LSLLexer', - 'AppleScriptLexer', 'RexxLexer', 'MOOCodeLexer', 'HybrisLexer'] + 'AppleScriptLexer', 'RexxLexer', 'MOOCodeLexer', 'HybrisLexer', + 'EasytrieveLexer', 'JclLexer'] class LuaLexer(RegexLexer): @@ -921,3 +922,275 @@ class HybrisLexer(RegexLexer): (r'[\w.]+\*?', Name.Namespace, '#pop') ], } + + +class EasytrieveLexer(RegexLexer): + """ + Easytrieve Plus is a programming language for extracting, filtering and + converting sequential data. Furthermore it can layout data for reports. + It is mainly used on mainframe platforms and can access several of the + mainframe's native file formats. It is somewhat comparable to awk. + + .. versionadded:: 2.1 + """ + name = 'Easytrieve' + aliases = ['easytrieve'] + filenames = ['*.ezt', '*.mac'] + mimetypes = ['text/x-easytrieve'] + flags = 0 + + # Note: We cannot use r'\b' at the start and end of keywords because + # Easytrieve Plus delimiter characters are: + # + # * space ( ) + # * apostrophe (') + # * period (.) + # * comma (,) + # * paranthesis ( and ) + # * colon (:) + # + # Additionally words end once a '*' appears, indicatins a comment. + _DELIMITERS = r' \'.,():\n' + _DELIMITERS_OR_COMENT = _DELIMITERS + '*' + _DELIMITER_PATTERN = '[' + _DELIMITERS + ']' + _DELIMITER_PATTERN_CAPTURE = '(' + _DELIMITER_PATTERN + ')' + _NON_DELIMITER_OR_COMMENT_PATTERN = '[^' + _DELIMITERS_OR_COMENT + ']' + _OPERATORS_PATTERN = u'[.+\\-/=\\[\\](){}<>;,&%¬]' + _KEYWORDS = [ + 'AFTER-BREAK', 'AFTER-LINE', 'AFTER-SCREEN', 'AIM', 'AND', 'ATTR', + 'BEFORE', 'BEFORE-BREAK', 'BEFORE-LINE', 'BEFORE-SCREEN', 'BUSHU', + 'BY', 'CALL', 'CASE', 'CHECKPOINT', 'CHKP', 'CHKP-STATUS', 'CLEAR', + 'CLOSE', 'COL', 'COLOR', 'COMMIT', 'CONTROL', 'COPY', 'CURSOR', 'D', + 'DECLARE', 'DEFAULT', 'DEFINE', 'DELETE', 'DENWA', 'DISPLAY', 'DLI', + 'DO', 'DUPLICATE', 'E', 'ELSE', 'ELSE-IF', 'END', 'END-CASE', + 'END-DO', 'END-IF', 'END-PROC', 'ENDPAGE', 'ENDTABLE', 'ENTER', 'EOF', + 'EQ', 'ERROR', 'EXIT', 'EXTERNAL', 'EZLIB', 'F1', 'F10', 'F11', 'F12', + 'F13', 'F14', 'F15', 'F16', 'F17', 'F18', 'F19', 'F2', 'F20', 'F21', + 'F22', 'F23', 'F24', 'F25', 'F26', 'F27', 'F28', 'F29', 'F3', 'F30', + 'F31', 'F32', 'F33', 'F34', 'F35', 'F36', 'F4', 'F5', 'F6', 'F7', + 'F8', 'F9', 'FETCH', 'FILE-STATUS', 'FILL', 'FINAL', 'FIRST', + 'FIRST-DUP', 'FOR', 'GE', 'GET', 'GO', 'GOTO', 'GQ', 'GR', 'GT', + 'HEADING', 'HEX', 'HIGH-VALUES', 'IDD', 'IDMS', 'IF', 'IN', 'INSERT', + 'JUSTIFY', 'KANJI-DATE', 'KANJI-DATE-LONG', 'KANJI-TIME', 'KEY', + 'KEY-PRESSED', 'KOKUGO', 'KUN', 'LAST-DUP', 'LE', 'LEVEL', 'LIKE', + 'LINE', 'LINE-COUNT', 'LINE-NUMBER', 'LINK', 'LIST', 'LOW-VALUES', + 'LQ', 'LS', 'LT', 'MACRO', 'MASK', 'MATCHED', 'MEND', 'MESSAGE', + 'MOVE', 'MSTART', 'NE', 'NEWPAGE', 'NOMASK', 'NOPRINT', 'NOT', + 'NOTE', 'NOVERIFY', 'NQ', 'NULL', 'OF', 'OR', 'OTHERWISE', 'PA1', + 'PA2', 'PA3', 'PAGE-COUNT', 'PAGE-NUMBER', 'PARM-REGISTER', + 'PATH-ID', 'PATTERN', 'PERFORM', 'POINT', 'POS', 'PRIMARY', 'PRINT', + 'PROCEDURE', 'PROGRAM', 'PUT', 'READ', 'RECORD', 'RECORD-COUNT', + 'RECORD-LENGTH', 'REFRESH', 'RELEASE', 'RENUM', 'REPEAT', 'REPORT', + 'REPORT-INPUT', 'RESHOW', 'RESTART', 'RETRIEVE', 'RETURN-CODE', + 'ROLLBACK', 'ROW', 'S', 'SCREEN', 'SEARCH', 'SECONDARY', 'SELECT', + 'SEQUENCE', 'SIZE', 'SKIP', 'SOKAKU', 'SORT', 'SQL', 'STOP', 'SUM', + 'SYSDATE', 'SYSDATE-LONG', 'SYSIN', 'SYSIPT', 'SYSLST', 'SYSPRINT', + 'SYSSNAP', 'SYSTIME', 'TALLY', 'TERM-COLUMNS', 'TERM-NAME', + 'TERM-ROWS', 'TERMINATION', 'TITLE', 'TO', 'TRANSFER', 'TRC', + 'UNIQUE', 'UNTIL', 'UPDATE', 'UPPERCASE', 'USER', 'USERID', 'VALUE', + 'VERIFY', 'W', 'WHEN', 'WHILE', 'WORK', 'WRITE', 'X', 'XDM', 'XRST' + ] + + tokens = { + 'root': [ + (r'\*.*\n', Comment.Single), + (r'\n+', Whitespace), + # Macro argument + (r'&' + _NON_DELIMITER_OR_COMMENT_PATTERN + r'+\.', Name.Variable, 'after_macro_argument'), + # Macro call + (r'%' + _NON_DELIMITER_OR_COMMENT_PATTERN + r'+', Name.Variable), + (r'(FILE|MACRO|REPORT)(\s+)', + bygroups(Keyword.Declaration, Whitespace), 'after_declaration'), + (r'(JOB|PARM)' + r'(' + _DELIMITER_PATTERN + r')', + bygroups(Keyword.Declaration, Operator)), + (words(_KEYWORDS, suffix=_DELIMITER_PATTERN_CAPTURE), + bygroups(Keyword.Reserved, Operator)), + (_OPERATORS_PATTERN, Operator), + # Procedure declaration + (r'(' + _NON_DELIMITER_OR_COMMENT_PATTERN + r'+)(\s*)(\.?)(\s*)(PROC)(\s*\n)', + bygroups(Name.Function, Whitespace, Operator, Whitespace, Keyword.Declaration, Whitespace)), + (r'[0-9]+\.[0-9]*', Number.Float), + (r'[0-9]+', Number.Integer), + (r"'(''|[^'])*'", String), + (r'\s+', Whitespace), + (_NON_DELIMITER_OR_COMMENT_PATTERN + r'+', Name) # Everything else just belongs to a name + ], + 'after_declaration': [ + (_NON_DELIMITER_OR_COMMENT_PATTERN + r'+', Name.Function), + ('', Whitespace, '#pop') + ], + 'after_macro_argument': [ + (r'\*.*\n', Comment.Single, '#pop'), + (r'\s+', Whitespace, '#pop'), + (_OPERATORS_PATTERN, Operator, '#pop'), + (r"'(''|[^'])*'", String, '#pop'), + (_NON_DELIMITER_OR_COMMENT_PATTERN + r'+', Name) # Everything else just belongs to a name + ], + } + _COMMENT_LINE_REGEX = re.compile(r'^\s*\*') + _MACRO_HEADER_REGEX = re.compile(r'^\s*MACRO') + + def analyse_text(text): + """ + Perform a structural analysis for basic Easytrieve constructs. + """ + result = 0.0 + lines = text.split('\n') + hasEndProc = False + hasHeaderComment = False + hasFile = False + hasJob = False + hasProc = False + hasParm = False + hasReport = False + + def isCommentLine(line): + return EasytrieveLexer._COMMENT_LINE_REGEX.match(lines[0]) is not None + + def isEmptyLine(line): + return not bool(line.strip()) + + # Remove possible empty lines and header comments. + while lines and (isEmptyLine(lines[0]) or isCommentLine(lines[0])): + if not isEmptyLine(lines[0]): + hasHeaderComment = True + del lines[0] + + if EasytrieveLexer._MACRO_HEADER_REGEX.match(lines[0]): + # Looks like an Easytrieve macro. + result = 0.4 + if hasHeaderComment: + result += 0.4 + else: + # Scan the source for lines starting with indicators. + for line in lines: + words = line.split() + if (len(words) >= 2): + firstWord = words[0] + if not hasReport: + if not hasJob: + if not hasFile: + if not hasParm: + if firstWord == 'PARM': + hasParm = True + if firstWord == 'FILE': + hasFile = True + if firstWord == 'JOB': + hasJob = True + elif firstWord == 'PROC': + hasProc = True + elif firstWord == 'END-PROC': + hasEndProc = True + elif firstWord == 'REPORT': + hasReport = True + + # Weight the findings. + if hasJob and (hasProc == hasEndProc): + if hasHeaderComment: + result += 0.1 + if hasParm: + if hasProc: + # Found PARM, JOB and PROC/END-PROC: + # pretty sure this is Easytrieve. + result += 0.8 + else: + # Found PARAM and JOB: probably this is Easytrieve + result += 0.5 + else: + # Found JOB and possibly other keywords: might be Easytrieve + result += 0.11 + if hasParm: + # Note: PARAM is not a proper English word, so this is + # regarded a much better indicator for Easytrieve than + # the other words. + result += 0.2 + if hasFile: + result += 0.01 + if hasReport: + result += 0.01 + assert 0.0 <= result <= 1.0 + return result + + +class JclLexer(RegexLexer): + """ + `Job Control Language (JCL) <http://publibz.boulder.ibm.com/cgi-bin/bookmgr_OS390/BOOKS/IEA2B570/CCONTENTS>`_ + is a scripting language used on mainframe platforms to instruct the system + on how to run a batch job or start a subsystem. It is somewhat + comparable to MS DOS batch and Unix shell scripts. + + .. versionadded:: 2.1 + """ + name = 'JCL' + aliases = ['jcl'] + filenames = ['*.jcl'] + mimetypes = ['text/x-jcl'] + flags = re.IGNORECASE + + tokens = { + 'root': [ + (r'//\*.*\n', Comment.Single), + (r'//', Keyword.Pseudo, 'statement'), + (r'/\*', Keyword.Pseudo, 'jes2_statement'), + # TODO: JES3 statement + (r'.*\n', Other) # Input text or inline code in any language. + ], + 'statement': [ + (r'\s*\n', Whitespace, '#pop'), + (r'([a-z][a-z_0-9]*)(\s+)(exec|job)(\s*)', + bygroups(Name.Label, Whitespace, Keyword.Reserved, Whitespace), + 'option'), + (r'[a-z][a-z_0-9]*', Name.Variable, 'statement_command'), + (r'\s+', Whitespace, 'statement_command'), + ], + 'statement_command': [ + (r'\s+(command|cntl|dd|endctl|endif|else|include|jcllib|' + r'output|pend|proc|set|then|xmit)\s+', Keyword.Reserved, 'option'), + include('option') + ], + 'jes2_statement': [ + (r'\s*\n', Whitespace, '#pop'), + (r'\$', Keyword, 'option'), + (r'\b(jobparam|message|netacct|notify|output|priority|route|' + r'setup|signoff|xeq|xmit)\b', Keyword, 'option'), + ], + 'option': [ + #(r'\n', Text, 'root'), + (r'\*', Name.Builtin), + (r'[\[\](){}<>;,]', Punctuation), + (r'[-+*/=&%]', Operator), + (r'[a-z_][a-z_0-9]*', Name), + (r'[0-9]+\.[0-9]*', Number.Float), + (r'\.[0-9]+', Number.Float), + (r'[0-9]+', Number.Integer), + (r"'", String, 'option_string'), + (r'[ \t]+', Whitespace, 'option_comment'), + (r'\.', Punctuation), + ], + 'option_string': [ + (r"(\n)(//)", bygroups(Text, Keyword.Pseudo)), + (r"''", String), + (r"[^']", String), + (r"'", String, '#pop'), + ], + 'option_comment': [ + #(r'\n', Text, 'root'), + (r'.+', Comment.Single), + ] + } + + _JOB_HEADER_PATTERN = re.compile(r'^//[a-z#$@][a-z0-9#$@]{0,7}\s+job(\s+.*)?$', re.IGNORECASE) + + def analyse_text(text): + """ + Recognize JCL job by header. + """ + result = 0.0 + lines = text.split('\n') + if len(lines) > 0: + if JclLexer._JOB_HEADER_PATTERN.match(lines[0]): + result = 1.0 + assert 0.0 <= result <= 1.0 + return result + +
pygments/lexers/shell.py+439 −95 modified@@ -11,14 +11,16 @@ import re -from pygments.lexer import Lexer, RegexLexer, do_insertions, bygroups, include +from pygments.lexer import Lexer, RegexLexer, do_insertions, bygroups, \ + include, default, this, using, words from pygments.token import Punctuation, \ Text, Comment, Operator, Keyword, Name, String, Number, Generic from pygments.util import shebang_matches __all__ = ['BashLexer', 'BashSessionLexer', 'TcshLexer', 'BatchLexer', - 'PowerShellLexer', 'ShellSessionLexer'] + 'MSDOSSessionLexer', 'PowerShellLexer', + 'PowerShellSessionLexer', 'TcshSessionLexer', 'FishShellLexer'] line_re = re.compile('.*?\n') @@ -47,7 +49,9 @@ class BashLexer(RegexLexer): (r'\$\(\(', Keyword, 'math'), (r'\$\(', Keyword, 'paren'), (r'\$\{#?', String.Interpol, 'curly'), - (r'\$(\w+|.)', Name.Variable), + (r'\$[a-fA-F_][a-fA-F0-9_]*', Name.Variable), # user variable + (r'\$(?:\d+|[#$?!_*@-])', Name.Variable), # builtin + (r'\$', Text), ], 'basic': [ (r'\b(if|fi|else|while|do|done|for|then|return|function|case|' @@ -120,29 +124,22 @@ def analyse_text(text): return 0.2 -class BashSessionLexer(Lexer): +class ShellSessionBaseLexer(Lexer): """ - Lexer for simplistic shell sessions. + Base lexer for simplistic shell sessions. - .. versionadded:: 1.1 + .. versionadded:: 2.1 """ - - name = 'Bash Session' - aliases = ['console'] - filenames = ['*.sh-session'] - mimetypes = ['application/x-shell-session'] - def get_tokens_unprocessed(self, text): - bashlexer = BashLexer(**self.options) + innerlexer = self._innerLexerCls(**self.options) pos = 0 curcode = '' insertions = [] for match in line_re.finditer(text): line = match.group() - m = re.match(r'^((?:\(\S+\))?(?:|sh\S*?|\w+\S+[@:]\S+(?:\s+\S+)' - r'?|\[\S+[@:][^\n]+\].+)[$#%])(.*\n?)' , line) + m = re.match(self._ps1rgx, line) if m: # To support output lexers (say diff output), the output # needs to be broken by prompts whenever the output lexer @@ -153,68 +150,41 @@ def get_tokens_unprocessed(self, text): insertions.append((len(curcode), [(0, Generic.Prompt, m.group(1))])) curcode += m.group(2) - elif line.startswith('>'): + elif line.startswith(self._ps2): insertions.append((len(curcode), - [(0, Generic.Prompt, line[:1])])) - curcode += line[1:] + [(0, Generic.Prompt, line[:len(self._ps2)])])) + curcode += line[len(self._ps2):] else: if insertions: - toks = bashlexer.get_tokens_unprocessed(curcode) + toks = innerlexer.get_tokens_unprocessed(curcode) for i, t, v in do_insertions(insertions, toks): yield pos+i, t, v yield match.start(), Generic.Output, line insertions = [] curcode = '' if insertions: for i, t, v in do_insertions(insertions, - bashlexer.get_tokens_unprocessed(curcode)): + innerlexer.get_tokens_unprocessed(curcode)): yield pos+i, t, v -class ShellSessionLexer(Lexer): +class BashSessionLexer(ShellSessionBaseLexer): """ - Lexer for shell sessions that works with different command prompts + Lexer for simplistic shell sessions. - .. versionadded:: 1.6 + .. versionadded:: 1.1 """ - name = 'Shell Session' - aliases = ['shell-session'] - filenames = ['*.shell-session'] - mimetypes = ['application/x-sh-session'] - - def get_tokens_unprocessed(self, text): - bashlexer = BashLexer(**self.options) - - pos = 0 - curcode = '' - insertions = [] - - for match in line_re.finditer(text): - line = match.group() - m = re.match(r'^((?:\[?\S+@[^$#%]+\]?\s*)[$#%])(.*\n?)', line) - if m: - # To support output lexers (say diff output), the output - # needs to be broken by prompts whenever the output lexer - # changes. - if not insertions: - pos = match.start() + name = 'Bash Session' + aliases = ['console', 'shell-session'] + filenames = ['*.sh-session', '*.shell-session'] + mimetypes = ['application/x-shell-session', 'application/x-sh-session'] - insertions.append((len(curcode), - [(0, Generic.Prompt, m.group(1))])) - curcode += m.group(2) - else: - if insertions: - toks = bashlexer.get_tokens_unprocessed(curcode) - for i, t, v in do_insertions(insertions, toks): - yield pos+i, t, v - yield match.start(), Generic.Output, line - insertions = [] - curcode = '' - if insertions: - for i, t, v in do_insertions(insertions, - bashlexer.get_tokens_unprocessed(curcode)): - yield pos+i, t, v + _innerLexerCls = BashLexer + _ps1rgx = \ + r'^((?:(?:\[.*?\])|(?:\(\S+\))?(?:| |sh\S*?|\w+\S+[@:]\S+(?:\s+\S+)' \ + r'?|\[\S+[@:][^\n]+\].+))\s*[$#%])(.*\n?)' + _ps2 = '>' class BatchLexer(RegexLexer): @@ -230,49 +200,317 @@ class BatchLexer(RegexLexer): flags = re.MULTILINE | re.IGNORECASE + _nl = r'\n\x1a' + _punct = r'&<>|' + _ws = r'\t\v\f\r ,;=\xa0' + _space = r'(?:(?:(?:\^[%s])?[%s])+)' % (_nl, _ws) + _keyword_terminator = (r'(?=(?:\^[%s]?)?[%s+./:[\\\]]|[%s%s(])' % + (_nl, _ws, _nl, _punct)) + _token_terminator = r'(?=\^?[%s]|[%s%s])' % (_ws, _punct, _nl) + _start_label = r'((?:(?<=^[^:])|^[^:]?)[%s]*)(:)' % _ws + _label = r'(?:(?:[^%s%s%s+:^]|\^[%s]?[\w\W])*)' % (_nl, _punct, _ws, _nl) + _label_compound = (r'(?:(?:[^%s%s%s+:^)]|\^[%s]?[^)])*)' % + (_nl, _punct, _ws, _nl)) + _number = r'(?:-?(?:0[0-7]+|0x[\da-f]+|\d+)%s)' % _token_terminator + _opword = r'(?:equ|geq|gtr|leq|lss|neq)' + _string = r'(?:"[^%s"]*"?)' % _nl + _variable = (r'(?:(?:%%(?:\*|(?:~[a-z]*(?:\$[^:]+:)?)?\d|' + r'[^%%:%s]+(?::(?:~(?:-?\d+)?(?:,(?:-?\d+)?)?|(?:[^%%%s^]|' + r'\^[^%%%s])[^=%s]*=(?:[^%%%s^]|\^[^%%%s])*)?)?%%))|' + r'(?:\^?![^!:%s]+(?::(?:~(?:-?\d+)?(?:,(?:-?\d+)?)?|(?:' + r'[^!%s^]|\^[^!%s])[^=%s]*=(?:[^!%s^]|\^[^!%s])*)?)?\^?!))' % + (_nl, _nl, _nl, _nl, _nl, _nl, _nl, _nl, _nl, _nl, _nl, _nl)) + _core_token = r'(?:(?:(?:\^[%s]?)?[^%s%s%s])+)' % (_nl, _nl, _punct, _ws) + _core_token_compound = r'(?:(?:(?:\^[%s]?)?[^%s%s%s)])+)' % (_nl, _nl, + _punct, _ws) + _token = r'(?:[%s]+|%s)' % (_punct, _core_token) + _token_compound = r'(?:[%s]+|%s)' % (_punct, _core_token_compound) + _stoken = (r'(?:[%s]+|(?:%s|%s|%s)+)' % + (_punct, _string, _variable, _core_token)) + + def _make_begin_state(compound, _core_token=_core_token, + _core_token_compound=_core_token_compound, + _keyword_terminator=_keyword_terminator, + _nl=_nl, _punct=_punct, _string=_string, + _space=_space, _start_label=_start_label, + _stoken=_stoken, _token_terminator=_token_terminator, + _variable=_variable, _ws=_ws): + rest = '(?:%s|%s|[^"%%%s%s%s])*' % (_string, _variable, _nl, _punct, + ')' if compound else '') + rest_of_line = r'(?:(?:[^%s^]|\^[%s]?[\w\W])*)' % (_nl, _nl) + rest_of_line_compound = r'(?:(?:[^%s^)]|\^[%s]?[^)])*)' % (_nl, _nl) + set_space = r'((?:(?:\^[%s]?)?[^\S\n])*)' % _nl + suffix = '' + if compound: + _keyword_terminator = r'(?:(?=\))|%s)' % _keyword_terminator + _token_terminator = r'(?:(?=\))|%s)' % _token_terminator + suffix = '/compound' + return [ + ((r'\)', Punctuation, '#pop') if compound else + (r'\)((?=\()|%s)%s' % (_token_terminator, rest_of_line), + Comment.Single)), + (r'(?=%s)' % _start_label, Text, 'follow%s' % suffix), + (_space, using(this, state='text')), + include('redirect%s' % suffix), + (r'[%s]+' % _nl, Text), + (r'\(', Punctuation, 'root/compound'), + (r'@+', Punctuation), + (r'((?:for|if|rem)(?:(?=(?:\^[%s]?)?/)|(?:(?!\^)|' + r'(?<=m))(?:(?=\()|%s)))(%s?%s?(?:\^[%s]?)?/(?:\^[%s]?)?\?)' % + (_nl, _token_terminator, _space, + _core_token_compound if compound else _core_token, _nl, _nl), + bygroups(Keyword, using(this, state='text')), + 'follow%s' % suffix), + (r'(goto%s)(%s(?:\^[%s]?)?/(?:\^[%s]?)?\?%s)' % + (_keyword_terminator, rest, _nl, _nl, rest), + bygroups(Keyword, using(this, state='text')), + 'follow%s' % suffix), + (words(('assoc', 'break', 'cd', 'chdir', 'cls', 'color', 'copy', + 'date', 'del', 'dir', 'dpath', 'echo', 'endlocal', 'erase', + 'exit', 'ftype', 'keys', 'md', 'mkdir', 'mklink', 'move', + 'path', 'pause', 'popd', 'prompt', 'pushd', 'rd', 'ren', + 'rename', 'rmdir', 'setlocal', 'shift', 'start', 'time', + 'title', 'type', 'ver', 'verify', 'vol'), + suffix=_keyword_terminator), Keyword, 'follow%s' % suffix), + (r'(call)(%s?)(:)' % _space, + bygroups(Keyword, using(this, state='text'), Punctuation), + 'call%s' % suffix), + (r'call%s' % _keyword_terminator, Keyword), + (r'(for%s(?!\^))(%s)(/f%s)' % + (_token_terminator, _space, _token_terminator), + bygroups(Keyword, using(this, state='text'), Keyword), + ('for/f', 'for')), + (r'(for%s(?!\^))(%s)(/l%s)' % + (_token_terminator, _space, _token_terminator), + bygroups(Keyword, using(this, state='text'), Keyword), + ('for/l', 'for')), + (r'for%s(?!\^)' % _token_terminator, Keyword, ('for2', 'for')), + (r'(goto%s)(%s?)(:?)' % (_keyword_terminator, _space), + bygroups(Keyword, using(this, state='text'), Punctuation), + 'label%s' % suffix), + (r'(if(?:(?=\()|%s)(?!\^))(%s?)((?:/i%s)?)(%s?)((?:not%s)?)(%s?)' % + (_token_terminator, _space, _token_terminator, _space, + _token_terminator, _space), + bygroups(Keyword, using(this, state='text'), Keyword, + using(this, state='text'), Keyword, + using(this, state='text')), ('(?', 'if')), + (r'rem(((?=\()|%s)%s?%s?.*|%s%s)' % + (_token_terminator, _space, _stoken, _keyword_terminator, + rest_of_line_compound if compound else rest_of_line), + Comment.Single, 'follow%s' % suffix), + (r'(set%s)%s(/a)' % (_keyword_terminator, set_space), + bygroups(Keyword, using(this, state='text'), Keyword), + 'arithmetic%s' % suffix), + (r'(set%s)%s((?:/p)?)%s((?:(?:(?:\^[%s]?)?[^"%s%s^=%s]|' + r'\^[%s]?[^"=])+)?)((?:(?:\^[%s]?)?=)?)' % + (_keyword_terminator, set_space, set_space, _nl, _nl, _punct, + ')' if compound else '', _nl, _nl), + bygroups(Keyword, using(this, state='text'), Keyword, + using(this, state='text'), using(this, state='variable'), + Punctuation), + 'follow%s' % suffix), + default('follow%s' % suffix) + ] + + def _make_follow_state(compound, _label=_label, + _label_compound=_label_compound, _nl=_nl, + _space=_space, _start_label=_start_label, + _token=_token, _token_compound=_token_compound, + _ws=_ws): + suffix = '/compound' if compound else '' + state = [] + if compound: + state.append((r'(?=\))', Text, '#pop')) + state += [ + (r'%s([%s]*)(%s)(.*)' % + (_start_label, _ws, _label_compound if compound else _label), + bygroups(Text, Punctuation, Text, Name.Label, Comment.Single)), + include('redirect%s' % suffix), + (r'(?=[%s])' % _nl, Text, '#pop'), + (r'\|\|?|&&?', Punctuation, '#pop'), + include('text') + ] + return state + + def _make_arithmetic_state(compound, _nl=_nl, _punct=_punct, + _string=_string, _variable=_variable, _ws=_ws): + op = r'=+\-*/!~' + state = [] + if compound: + state.append((r'(?=\))', Text, '#pop')) + state += [ + (r'0[0-7]+', Number.Oct), + (r'0x[\da-f]+', Number.Hex), + (r'\d+', Number.Integer), + (r'[(),]+', Punctuation), + (r'([%s]|%%|\^\^)+' % op, Operator), + (r'(%s|%s|(\^[%s]?)?[^()%s%%^"%s%s%s]|\^[%s%s]?%s)+' % + (_string, _variable, _nl, op, _nl, _punct, _ws, _nl, _ws, + r'[^)]' if compound else r'[\w\W]'), + using(this, state='variable')), + (r'(?=[\x00|&])', Text, '#pop'), + include('follow') + ] + return state + + def _make_call_state(compound, _label=_label, + _label_compound=_label_compound): + state = [] + if compound: + state.append((r'(?=\))', Text, '#pop')) + state.append((r'(:?)(%s)' % (_label_compound if compound else _label), + bygroups(Punctuation, Name.Label), '#pop')) + return state + + def _make_label_state(compound, _label=_label, + _label_compound=_label_compound, _nl=_nl, + _punct=_punct, _string=_string, _variable=_variable): + state = [] + if compound: + state.append((r'(?=\))', Text, '#pop')) + state.append((r'(%s?)((?:%s|%s|\^[%s]?%s|[^"%%^%s%s%s])*)' % + (_label_compound if compound else _label, _string, + _variable, _nl, r'[^)]' if compound else r'[\w\W]', _nl, + _punct, r')' if compound else ''), + bygroups(Name.Label, Comment.Single), '#pop')) + return state + + def _make_redirect_state(compound, + _core_token_compound=_core_token_compound, + _nl=_nl, _punct=_punct, _stoken=_stoken, + _string=_string, _space=_space, + _variable=_variable, _ws=_ws): + stoken_compound = (r'(?:[%s]+|(?:%s|%s|%s)+)' % + (_punct, _string, _variable, _core_token_compound)) + return [ + (r'((?:(?<=[%s%s])\d)?)(>>?&|<&)([%s%s]*)(\d)' % + (_nl, _ws, _nl, _ws), + bygroups(Number.Integer, Punctuation, Text, Number.Integer)), + (r'((?:(?<=[%s%s])(?<!\^[%s])\d)?)(>>?|<)(%s?%s)' % + (_nl, _ws, _nl, _space, stoken_compound if compound else _stoken), + bygroups(Number.Integer, Punctuation, using(this, state='text'))) + ] + tokens = { - 'root': [ - # Lines can start with @ to prevent echo - (r'^\s*@', Punctuation), - (r'^(\s*)(rem\s.*)$', bygroups(Text, Comment)), - (r'".*?"', String.Double), - (r"'.*?'", String.Single), - # If made more specific, make sure you still allow expansions - # like %~$VAR:zlt - (r'%%?[~$:\w]+%?', Name.Variable), - (r'::.*', Comment), # Technically :: only works at BOL - (r'\b(set)(\s+)(\w+)', bygroups(Keyword, Text, Name.Variable)), - (r'\b(call)(\s+)(:\w+)', bygroups(Keyword, Text, Name.Label)), - (r'\b(goto)(\s+)(\w+)', bygroups(Keyword, Text, Name.Label)), - (r'\b(set|call|echo|on|off|endlocal|for|do|goto|if|pause|' - r'setlocal|shift|errorlevel|exist|defined|cmdextversion|' - r'errorlevel|else|cd|md|del|deltree|cls|choice)\b', Keyword), - (r'\b(equ|neq|lss|leq|gtr|geq)\b', Operator), - include('basic'), - (r'.', Text), + 'root': _make_begin_state(False), + 'follow': _make_follow_state(False), + 'arithmetic': _make_arithmetic_state(False), + 'call': _make_call_state(False), + 'label': _make_label_state(False), + 'redirect': _make_redirect_state(False), + 'root/compound': _make_begin_state(True), + 'follow/compound': _make_follow_state(True), + 'arithmetic/compound': _make_arithmetic_state(True), + 'call/compound': _make_call_state(True), + 'label/compound': _make_label_state(True), + 'redirect/compound': _make_redirect_state(True), + 'variable-or-escape': [ + (_variable, Name.Variable), + (r'%%%%|\^[%s]?(\^!|[\w\W])' % _nl, String.Escape) ], - 'echo': [ - # Escapes only valid within echo args? - (r'\^\^|\^<|\^>|\^\|', String.Escape), - (r'\n', Text, '#pop'), - include('basic'), - (r'[^\'"^]+', Text), + 'string': [ + (r'"', String.Double, '#pop'), + (_variable, Name.Variable), + (r'\^!|%%', String.Escape), + (r'[^"%%^%s]+|[%%^]' % _nl, String.Double), + default('#pop') ], - 'basic': [ - (r'".*?"', String.Double), - (r"'.*?'", String.Single), - (r'`.*?`', String.Backtick), - (r'-?\d+', Number), - (r',', Punctuation), - (r'=', Operator), - (r'/\S+', Name), - (r':\w+', Name.Label), - (r'\w:\w+', Text), - (r'([<>|])(\s*)(\w+)', bygroups(Punctuation, Text, Name)), + 'sqstring': [ + include('variable-or-escape'), + (r'[^%]+|%', String.Single) ], + 'bqstring': [ + include('variable-or-escape'), + (r'[^%]+|%', String.Backtick) + ], + 'text': [ + (r'"', String.Double, 'string'), + include('variable-or-escape'), + (r'[^"%%^%s%s%s\d)]+|.' % (_nl, _punct, _ws), Text) + ], + 'variable': [ + (r'"', String.Double, 'string'), + include('variable-or-escape'), + (r'[^"%%^%s]+|.' % _nl, Name.Variable) + ], + 'for': [ + (r'(%s)(in)(%s)(\()' % (_space, _space), + bygroups(using(this, state='text'), Keyword, + using(this, state='text'), Punctuation), '#pop'), + include('follow') + ], + 'for2': [ + (r'\)', Punctuation), + (r'(%s)(do%s)' % (_space, _token_terminator), + bygroups(using(this, state='text'), Keyword), '#pop'), + (r'[%s]+' % _nl, Text), + include('follow') + ], + 'for/f': [ + (r'(")((?:%s|[^"])*?")([%s%s]*)(\))' % (_variable, _nl, _ws), + bygroups(String.Double, using(this, state='string'), Text, + Punctuation)), + (r'"', String.Double, ('#pop', 'for2', 'string')), + (r"('(?:%s|[\w\W])*?')([%s%s]*)(\))" % (_variable, _nl, _ws), + bygroups(using(this, state='sqstring'), Text, Punctuation)), + (r'(`(?:%s|[\w\W])*?`)([%s%s]*)(\))' % (_variable, _nl, _ws), + bygroups(using(this, state='bqstring'), Text, Punctuation)), + include('for2') + ], + 'for/l': [ + (r'-?\d+', Number.Integer), + include('for2') + ], + 'if': [ + (r'((?:cmdextversion|errorlevel)%s)(%s)(\d+)' % + (_token_terminator, _space), + bygroups(Keyword, using(this, state='text'), + Number.Integer), '#pop'), + (r'(defined%s)(%s)(%s)' % (_token_terminator, _space, _stoken), + bygroups(Keyword, using(this, state='text'), + using(this, state='variable')), '#pop'), + (r'(exist%s)(%s%s)' % (_token_terminator, _space, _stoken), + bygroups(Keyword, using(this, state='text')), '#pop'), + (r'(%s%s?)(==)(%s?%s)' % (_stoken, _space, _space, _stoken), + bygroups(using(this, state='text'), Operator, + using(this, state='text')), '#pop'), + (r'(%s%s)(%s)(%s%s)' % (_number, _space, _opword, _space, _number), + bygroups(using(this, state='arithmetic'), Operator.Word, + using(this, state='arithmetic')), '#pop'), + (r'(%s%s)(%s)(%s%s)' % (_stoken, _space, _opword, _space, _stoken), + bygroups(using(this, state='text'), Operator.Word, + using(this, state='text')), '#pop') + ], + '(?': [ + (_space, using(this, state='text')), + (r'\(', Punctuation, ('#pop', 'else?', 'root/compound')), + default('#pop') + ], + 'else?': [ + (_space, using(this, state='text')), + (r'else%s' % _token_terminator, Keyword, '#pop'), + default('#pop') + ] } +class MSDOSSessionLexer(ShellSessionBaseLexer): + """ + Lexer for simplistic MSDOS sessions. + + .. versionadded:: 2.1 + """ + + name = 'MSDOS Session' + aliases = ['doscon'] + filenames = [] + mimetypes = [] + + _innerLexerCls = BatchLexer + _ps1rgx = r'^([^>]+>)(.*\n?)' + _ps2 = 'More? ' + + class TcshLexer(RegexLexer): """ Lexer for tcsh scripts. @@ -340,6 +578,22 @@ class TcshLexer(RegexLexer): ], } +class TcshSessionLexer(ShellSessionBaseLexer): + """ + Lexer for Tcsh sessions. + + .. versionadded:: 2.1 + """ + + name = 'Tcsh Session' + aliases = ['tcshcon'] + filenames = [] + mimetypes = [] + + _innerLexerCls = TcshLexer + _ps1rgx = r'^([^>]+>)(.*\n?)' + _ps2 = '? ' + class PowerShellLexer(RegexLexer): """ @@ -436,3 +690,93 @@ class PowerShellLexer(RegexLexer): (r".", String.Heredoc), ] } + + +class FishShellLexer(RegexLexer): + """ + Lexer for Fish shell scripts. + + .. versionadded:: 2.1 + """ + + name = 'Fish' + aliases = ['fish', 'fishshell'] + filenames = ['*.fish', '*.load'] + mimetypes = ['application/x-fish'] + + tokens = { + 'root': [ + include('basic'), + include('data'), + include('interp'), + ], + 'interp': [ + (r'\$\(\(', Keyword, 'math'), + (r'\(', Keyword, 'paren'), + (r'\$#?(\w+|.)', Name.Variable), + ], + 'basic': [ + (r'\b(begin|end|if|else|while|break|for|in|return|function|block|' + r'case|continue|switch|not|and|or|set|echo|exit|pwd|true|false|' + r'cd|count|test)(\s*)\b', + bygroups(Keyword, Text)), + (r'\b(alias|bg|bind|breakpoint|builtin|command|commandline|' + r'complete|contains|dirh|dirs|emit|eval|exec|fg|fish|fish_config|' + r'fish_indent|fish_pager|fish_prompt|fish_right_prompt|' + r'fish_update_completions|fishd|funced|funcsave|functions|help|' + r'history|isatty|jobs|math|mimedb|nextd|open|popd|prevd|psub|' + r'pushd|random|read|set_color|source|status|trap|type|ulimit|' + r'umask|vared|fc|getopts|hash|kill|printf|time|wait)\s*\b(?!\.)', + Name.Builtin), + (r'#.*\n', Comment), + (r'\\[\w\W]', String.Escape), + (r'(\b\w+)(\s*)(=)', bygroups(Name.Variable, Text, Operator)), + (r'[\[\]()=]', Operator), + (r'<<-?\s*(\'?)\\?(\w+)[\w\W]+?\2', String), + ], + 'data': [ + (r'(?s)\$?"(\\\\|\\[0-7]+|\\.|[^"\\$])*"', String.Double), + (r'"', String.Double, 'string'), + (r"(?s)\$'(\\\\|\\[0-7]+|\\.|[^'\\])*'", String.Single), + (r"(?s)'.*?'", String.Single), + (r';', Punctuation), + (r'&|\||\^|<|>', Operator), + (r'\s+', Text), + (r'\d+(?= |\Z)', Number), + (r'[^=\s\[\]{}()$"\'`\\<&|;]+', Text), + ], + 'string': [ + (r'"', String.Double, '#pop'), + (r'(?s)(\\\\|\\[0-7]+|\\.|[^"\\$])+', String.Double), + include('interp'), + ], + 'paren': [ + (r'\)', Keyword, '#pop'), + include('root'), + ], + 'math': [ + (r'\)\)', Keyword, '#pop'), + (r'[-+*/%^|&]|\*\*|\|\|', Operator), + (r'\d+#\d+', Number), + (r'\d+#(?! )', Number), + (r'\d+', Number), + include('root'), + ], + } + + +class PowerShellSessionLexer(ShellSessionBaseLexer): + """ + Lexer for simplistic Windows PowerShell sessions. + + .. versionadded:: 2.1 + """ + + name = 'PowerShell Session' + aliases = ['ps1con'] + filenames = [] + mimetypes = [] + + _innerLexerCls = PowerShellLexer + _ps1rgx = r'^(PS [^>]+> )(.*\n?)' + _ps2 = '>> '
pygments/lexers/sql.py+2 −2 modified@@ -489,8 +489,8 @@ class MySqlLexer(RegexLexer): r'day_hour|day_microsecond|day_minute|day_second|dec|decimal|' r'declare|default|delayed|delete|desc|describe|deterministic|' r'distinct|distinctrow|div|double|drop|dual|each|else|elseif|' - r'enclosed|escaped|exists|exit|explain|fetch|float|float4|float8' - r'|for|force|foreign|from|fulltext|grant|group|having|' + r'enclosed|escaped|exists|exit|explain|fetch|flush|float|float4|' + r'float8|for|force|foreign|from|fulltext|grant|group|having|' r'high_priority|hour_microsecond|hour_minute|hour_second|if|' r'ignore|in|index|infile|inner|inout|insensitive|insert|int|' r'int1|int2|int3|int4|int8|integer|interval|into|is|iterate|'
pygments/lexers/_stan_builtins.py+8 −3 modified@@ -4,7 +4,7 @@ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ This file contains the names of functions for Stan used by - ``pygments.lexers.math.StanLexer. This is for Stan language version 2.7.0 + ``pygments.lexers.math.StanLexer. This is for Stan language version 2.8.0. :copyright: Copyright 2006-2015 by the Pygments team, see AUTHORS. :license: BSD, see LICENSE for details. @@ -39,8 +39,7 @@ 'simplex', 'unit_vector', 'vector', - 'void' -) + 'void') FUNCTIONS = ( 'Phi', @@ -105,6 +104,11 @@ 'cos', 'cosh', 'crossprod', + 'csr_extract_u', + 'csr_extract_v', + 'csr_extract_w', + 'csr_matrix_times_vector', + 'csr_to_dense_matrix', 'cumulative_sum', 'determinant', 'diag_matrix', @@ -187,6 +191,7 @@ 'inv_gamma_log', 'inv_gamma_rng', 'inv_logit', + 'inv_phi', 'inv_sqrt', 'inv_square', 'inv_wishart_log',
pygments/lexers/supercollider.py+89 −0 added@@ -0,0 +1,89 @@ +# -*- coding: utf-8 -*- +""" + pygments.lexers.supercollider + ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + + Lexer for SuperCollider + + :copyright: Copyright 2006-2015 by the Pygments team, see AUTHORS. + :license: BSD, see LICENSE for details. +""" + +import re + +from pygments.lexer import RegexLexer, include, words +from pygments.token import Text, Comment, Operator, Keyword, Name, String, \ + Number, Punctuation, Other + +__all__ = ['SuperColliderLexer'] + +class SuperColliderLexer(RegexLexer): + """ + For `SuperCollider <http://supercollider.github.io/>`_ source code. + + .. versionadded:: 2.1 + """ + + name = 'SuperCollider' + aliases = ['sc', 'supercollider'] + filenames = ['*.sc', '*.scd'] + mimetypes = ['application/supercollider', 'text/supercollider', ] + + flags = re.DOTALL | re.MULTILINE + tokens = { + 'commentsandwhitespace': [ + (r'\s+', Text), + (r'<!--', Comment), + (r'//.*?\n', Comment.Single), + (r'/\*.*?\*/', Comment.Multiline) + ], + 'slashstartsregex': [ + include('commentsandwhitespace'), + (r'/(\\.|[^[/\\\n]|\[(\\.|[^\]\\\n])*])+/' + r'([gim]+\b|\B)', String.Regex, '#pop'), + (r'(?=/)', Text, ('#pop', 'badregex')), + (r'', Text, '#pop') + ], + 'badregex': [ + (r'\n', Text, '#pop') + ], + 'root': [ + (r'^(?=\s|/|<!--)', Text, 'slashstartsregex'), + include('commentsandwhitespace'), + (r'\+\+|--|~|&&|\?|:|\|\||\\(?=\n)|' + r'(<<|>>>?|==?|!=?|[-<>+*%&|^/])=?', Operator, 'slashstartsregex'), + (r'[{(\[;,]', Punctuation, 'slashstartsregex'), + (r'[})\].]', Punctuation), + (words(( + 'for', 'in', 'while', 'do', 'break', 'return', 'continue', + 'switch', 'case', 'default', 'if', 'else', 'throw', 'try', + 'catch', 'finally', 'new', 'delete', 'typeof', 'instanceof', + 'void'), suffix=r'\b'), + Keyword, 'slashstartsregex'), + (words(('var', 'let', 'with', 'function', 'arg'), suffix=r'\b'), + Keyword.Declaration, 'slashstartsregex'), + (words(( + '(abstract', 'boolean', 'byte', 'char', 'class', 'const', + 'debugger', 'double', 'enum', 'export', 'extends', 'final', + 'float', 'goto', 'implements', 'import', 'int', 'interface', + 'long', 'native', 'package', 'private', 'protected', 'public', + 'short', 'static', 'super', 'synchronized', 'throws', + 'transient', 'volatile'), suffix=r'\b'), + Keyword.Reserved), + (words(('true', 'false', 'nil', 'inf'), suffix=r'\b'), Keyword.Constant), + (words(( + 'Array', 'Boolean', 'Date', 'Error', 'Function', 'Number', + 'Object', 'Packages', 'RegExp', 'String', 'Error', + 'isFinite', 'isNaN', 'parseFloat', 'parseInt', 'super', + 'thisFunctionDef', 'thisFunction', 'thisMethod', 'thisProcess', + 'thisThread', 'this'), suffix=r'\b'), + Name.Builtin), + (r'[$a-zA-Z_][a-zA-Z0-9_]*', Name.Other), + (r'\\?[$a-zA-Z_][a-zA-Z0-9_]*', String.Symbol), + (r'[0-9][0-9]*\.[0-9]+([eE][0-9]+)?[fd]?', Number.Float), + (r'0x[0-9a-fA-F]+', Number.Hex), + (r'[0-9]+', Number.Integer), + (r'"(\\\\|\\"|[^"])*"', String.Double), + (r"'(\\\\|\\'|[^'])*'", String.Single), + ] + }
pygments/lexers/tap.py+91 −0 added@@ -0,0 +1,91 @@ +# -*- coding: utf-8 -*- +""" + pygments.lexers.tap + ~~~~~~~~~~~~~~~~~~~ + + Lexer for the Test Anything Protocol (TAP). + + :copyright: Copyright 2006-2015 by the Pygments team, see AUTHORS. + :license: BSD, see LICENSE for details. +""" + +from pygments.lexer import bygroups, RegexLexer +from pygments.token import Comment, Generic, Keyword, Name, Number, Text + +__all__ = ['TAPLexer'] + + +class TAPLexer(RegexLexer): + """ + For Test Anything Protocol (TAP) output. + + .. versionadded:: 2.1 + """ + name = 'TAP' + aliases = ['tap'] + filenames = ['*.tap'] + + tokens = { + 'root': [ + # A TAP version may be specified. + (r'^TAP version \d+\n', Name.Namespace), + + # Specify a plan with a plan line. + (r'^1..\d+', Keyword.Declaration, 'plan'), + + # A test failure + (r'^(not ok)([^\S\n]*)(\d*)', + bygroups(Generic.Error, Text, Number.Integer), 'test'), + + # A test success + (r'^(ok)([^\S\n]*)(\d*)', + bygroups(Keyword.Reserved, Text, Number.Integer), 'test'), + + # Diagnostics start with a hash. + (r'^#.*\n', Comment), + + # TAP's version of an abort statement. + (r'^Bail out!.*\n', Generic.Error), + + # TAP ignores any unrecognized lines. + (r'^.*\n', Text), + ], + 'plan': [ + # Consume whitespace (but not newline). + (r'[^\S\n]+', Text), + + # A plan may have a directive with it. + (r'#', Comment, 'directive'), + + # Or it could just end. + (r'\n', Comment, '#pop'), + + # Anything else is wrong. + (r'.*\n', Generic.Error, '#pop'), + ], + 'test': [ + # Consume whitespace (but not newline). + (r'[^\S\n]+', Text), + + # A test may have a directive with it. + (r'#', Comment, 'directive'), + + (r'\S+', Text), + + (r'\n', Text, '#pop'), + ], + 'directive': [ + # Consume whitespace (but not newline). + (r'[^\S\n]+', Comment), + + # Extract todo items. + (r'(?i)\bTODO\b', Comment.Preproc), + + # Extract skip items. + (r'(?i)\bSKIP\S*', Comment.Preproc), + + (r'\S+', Comment), + + (r'\n', Comment, '#pop:2'), + ], + }
pygments/lexers/templates.py+6 −4 modified@@ -568,10 +568,12 @@ class MasonLexer(RegexLexer): } def analyse_text(text): - rv = 0.0 - if re.search('<&', text) is not None: - rv = 1.0 - return rv + result = 0.0 + if re.search(r'</%(class|doc|init)%>', text) is not None: + result = 1.0 + elif re.search(r'<&.+&>', text, re.DOTALL) is not None: + result = 0.11 + return result class MakoLexer(RegexLexer):
pygments/lexers/trafficscript.py+53 −0 added@@ -0,0 +1,53 @@ +# -*- coding: utf-8 -*- +""" + + pygments.lexers.trafficscript + ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + + Lexer for RiverBed's TrafficScript (RTS) language. + + :copyright: Copyright 2006-2015 by the Pygments team, see AUTHORS. + :license: BSD, see LICENSE for details. +""" +import re + +from pygments.lexer import RegexLexer +from pygments.token import String, Number, Name, Keyword, Operator, Text, Comment + +__all__ = ['RtsLexer'] + +class RtsLexer(RegexLexer): + """ + For `Riverbed Stingray Traffic Manager <http://www.riverbed.com/stingray>`_ + + .. versionadded:: 2.1 + """ + name = 'TrafficScript' + aliases = ['rts','trafficscript'] + filenames = ['*.rts'] + + tokens = { + 'root' : [ + (r"'(\\\\|\\[^\\]|[^'\\])*'", String), + (r'"', String, 'escapable-string'), + (r'(0x[0-9a-fA-F]+|\d+)', Number), + (r'\d+\.\d+', Number.Float), + (r'\$[a-zA-Z](\w|_)*', Name.Variable), + (r'(if|else|for(each)?|in|while|do|break|sub|return|import)', Keyword), + (r'[a-zA-Z][\w.]*', Name.Function), + (r'[-+*/%=,;(){}<>^.!~|&\[\]\?\:]', Operator), + (r'(>=|<=|==|!=|' + r'&&|\|\||' + r'\+=|.=|-=|\*=|/=|%=|<<=|>>=|&=|\|=|\^=|' + r'>>|<<|' + r'\+\+|--|=>)', Operator), + (r'[ \t\r]+', Text), + (r'#[^\n]*', Comment), + ], + 'escapable-string' : [ + (r'\\[tsn]', String.Escape), + (r'[^"]', String), + (r'"', String, '#pop'), + ], + + }
pygments/lexers/webmisc.py+57 −22 modified@@ -333,13 +333,14 @@ def pushstate_operator_callback(lexer, match, ctx): (r'(\{)', pushstate_root_callback), (r'then|else|external|at|div|except', Keyword, 'root'), (r'order by', Keyword, 'root'), + (r'group by', Keyword, 'root'), (r'is|mod|order\s+by|stable\s+order\s+by', Keyword, 'root'), (r'and|or', Operator.Word, 'root'), (r'(eq|ge|gt|le|lt|ne|idiv|intersect|in)(?=\b)', Operator.Word, 'root'), (r'return|satisfies|to|union|where|preserve\s+strip', Keyword, 'root'), - (r'(>=|>>|>|<=|<<|<|-|\*|!=|\+|\|\||\||:=|=)', + (r'(>=|>>|>|<=|<<|<|-|\*|!=|\+|\|\||\||:=|=|!)', operator_root_callback), (r'(::|;|\[|//|/|,)', punctuation_root_callback), @@ -349,6 +350,8 @@ def pushstate_operator_callback(lexer, match, ctx): bygroups(Keyword, Text, Keyword), 'itemtype'), (r'(treat)(\s+)(as)\b', bygroups(Keyword, Text, Keyword), 'itemtype'), + (r'(case)(\s+)(' + stringdouble + ')', bygroups(Keyword, Text, String.Double), 'itemtype'), + (r'(case)(\s+)(' + stringsingle + ')', bygroups(Keyword, Text, String.Single), 'itemtype'), (r'(case|as)\b', Keyword, 'itemtype'), (r'(\))(\s*)(as)', bygroups(Punctuation, Text, Keyword), 'itemtype'), @@ -361,6 +364,13 @@ def pushstate_operator_callback(lexer, match, ctx): (r'ascending|descending|default', Keyword, '#push'), (r'external', Keyword), (r'collation', Keyword, 'uritooperator'), + + # eXist specific XQUF + (r'(into|following|preceding|with)', Keyword, 'root'), + + # support for current context on rhs of Simple Map Operator + (r'\.', Operator), + # finally catch all string literals and stay in operator state (stringdouble, String.Double), (stringsingle, String.Single), @@ -394,9 +404,21 @@ def pushstate_operator_callback(lexer, match, ctx): (r'preserve|no-preserve', Keyword), (r',', Punctuation), ], + 'annotationname':[ + (r'\(:', Comment, 'comment'), + (qname, Name.Decorator), + (r'(\()(' + stringdouble + ')', bygroups(Punctuation, String.Double)), + (r'(\()(' + stringsingle + ')', bygroups(Punctuation, String.Single)), + (r'(\,)(\s+)(' + stringdouble + ')', bygroups(Punctuation, Text, String.Double)), + (r'(\,)(\s+)(' + stringsingle + ')', bygroups(Punctuation, Text, String.Single)), + (r'\)', Punctuation), + (r'(\s+)(\%)', bygroups(Text, Name.Decorator), 'annotationname'), + (r'(\s+)(variable)(\s+)(\$)', bygroups(Text, Keyword.Declaration, Text, Name.Variable), 'varname'), + (r'(\s+)(function)(\s+)', bygroups(Text, Keyword.Declaration, Text), 'root') + ], 'varname': [ (r'\(:', Comment, 'comment'), - (qname, Name.Variable, 'operator'), + (r'(' + qname + ')(\()?', bygroups(Name, Punctuation), 'operator'), ], 'singletype': [ (r'\(:', Comment, 'comment'), @@ -406,7 +428,7 @@ def pushstate_operator_callback(lexer, match, ctx): 'itemtype': [ include('whitespace'), (r'\(:', Comment, 'comment'), - (r'\$', Punctuation, 'varname'), + (r'\$', Name.Variable, 'varname'), (r'(void)(\s*)(\()(\s*)(\))', bygroups(Keyword, Text, Punctuation, Text, Punctuation), 'operator'), (r'(element|attribute|schema-element|schema-attribute|comment|text|' @@ -415,11 +437,11 @@ def pushstate_operator_callback(lexer, match, ctx): # Marklogic specific type? (r'(processing-instruction)(\s*)(\()', bygroups(Keyword, Text, Punctuation), - ('occurrenceindicator', 'kindtestforpi')), + ('occurrenceindicator', 'kindtestforpi')), (r'(item)(\s*)(\()(\s*)(\))(?=[*+?])', bygroups(Keyword, Text, Punctuation, Text, Punctuation), 'occurrenceindicator'), - (r'\(\#', Punctuation, 'pragma'), + (r'(\(\#)(\s*)', bygroups(Punctuation, Text), 'pragma'), (r';', Punctuation, '#pop'), (r'then|else', Keyword, '#pop'), (r'(at)(\s+)(' + stringdouble + ')', @@ -437,9 +459,12 @@ def pushstate_operator_callback(lexer, match, ctx): bygroups(Keyword, Text, Keyword), 'singletype'), (r'(treat)(\s+)(as)', bygroups(Keyword, Text, Keyword)), (r'(instance)(\s+)(of)', bygroups(Keyword, Text, Keyword)), + (r'(case)(\s+)(' + stringdouble + ')', bygroups(Keyword, Text, String.Double), 'itemtype'), + (r'(case)(\s+)(' + stringsingle + ')', bygroups(Keyword, Text, String.Single), 'itemtype'), (r'case|as', Keyword, 'itemtype'), (r'(\))(\s*)(as)', bygroups(Operator, Text, Keyword), 'itemtype'), (ncname + r':\*', Keyword.Type, 'operator'), + (r'(function)(\()', bygroups(Keyword.Type, Punctuation)), (qname, Keyword.Type, 'occurrenceindicator'), ], 'kindtest': [ @@ -589,9 +614,9 @@ def pushstate_operator_callback(lexer, match, ctx): (r'(\d+)', Number.Integer, 'operator'), (r'(\.\.|\.|\))', Punctuation, 'operator'), (r'(declare)(\s+)(construction)', - bygroups(Keyword, Text, Keyword), 'operator'), + bygroups(Keyword.Declaration, Text, Keyword.Declaration), 'operator'), (r'(declare)(\s+)(default)(\s+)(order)', - bygroups(Keyword, Text, Keyword, Text, Keyword), 'operator'), + bygroups(Keyword.Declaration, Text, Keyword.Declaration, Text, Keyword.Declaration), 'operator'), (ncname + ':\*', Name, 'operator'), ('\*:'+ncname, Name.Tag, 'operator'), ('\*', Name.Tag, 'operator'), @@ -602,26 +627,29 @@ def pushstate_operator_callback(lexer, match, ctx): # NAMESPACE DECL (r'(declare)(\s+)(default)(\s+)(collation)', - bygroups(Keyword, Text, Keyword, Text, Keyword)), + bygroups(Keyword.Declaration, Text, Keyword.Declaration, Text, Keyword.Declaration)), (r'(module|declare)(\s+)(namespace)', - bygroups(Keyword, Text, Keyword), 'namespacedecl'), + bygroups(Keyword.Declaration, Text, Keyword.Declaration), 'namespacedecl'), (r'(declare)(\s+)(base-uri)', - bygroups(Keyword, Text, Keyword), 'namespacedecl'), + bygroups(Keyword.Declaration, Text, Keyword.Declaration), 'namespacedecl'), # NAMESPACE KEYWORD (r'(declare)(\s+)(default)(\s+)(element|function)', - bygroups(Keyword, Text, Keyword, Text, Keyword), 'namespacekeyword'), + bygroups(Keyword.Declaration, Text, Keyword.Declaration, Text, Keyword.Declaration), 'namespacekeyword'), (r'(import)(\s+)(schema|module)', bygroups(Keyword.Pseudo, Text, Keyword.Pseudo), 'namespacekeyword'), (r'(declare)(\s+)(copy-namespaces)', - bygroups(Keyword, Text, Keyword), 'namespacekeyword'), + bygroups(Keyword.Declaration, Text, Keyword.Declaration), 'namespacekeyword'), # VARNAMEs (r'(for|let|some|every)(\s+)(\$)', bygroups(Keyword, Text, Name.Variable), 'varname'), (r'\$', Name.Variable, 'varname'), (r'(declare)(\s+)(variable)(\s+)(\$)', - bygroups(Keyword, Text, Keyword, Text, Name.Variable), 'varname'), + bygroups(Keyword.Declaration, Text, Keyword.Declaration, Text, Name.Variable), 'varname'), + + # ANNOTATED GLOBAL VARIABLES AND FUNCTIONS + (r'(declare)(\s+)(\%)', bygroups(Keyword.Declaration, Text, Name.Decorator), 'annotationname'), # ITEMTYPE (r'(\))(\s+)(as)', bygroups(Operator, Text, Keyword), 'itemtype'), @@ -643,12 +671,13 @@ def pushstate_operator_callback(lexer, match, ctx): (r'(<)', pushstate_operator_starttag_callback), (r'(declare)(\s+)(boundary-space)', - bygroups(Keyword, Text, Keyword), 'xmlspace_decl'), + bygroups(Keyword.Declaration, Text, Keyword.Declaration), 'xmlspace_decl'), (r'(validate)(\s+)(lax|strict)', pushstate_operator_root_validate_withmode), (r'(validate)(\s*)(\{)', pushstate_operator_root_validate), (r'(typeswitch)(\s*)(\()', bygroups(Keyword, Text, Punctuation)), + (r'(typeswitch)(\s*)(\()', bygroups(Keyword, Text, Punctuation)), (r'(element|attribute)(\s*)(\{)', pushstate_operator_root_construct_callback), @@ -666,25 +695,25 @@ def pushstate_operator_callback(lexer, match, ctx): 'operator'), (r'(declare|define)(\s+)(function)', - bygroups(Keyword, Text, Keyword)), + bygroups(Keyword.Declaration, Text, Keyword.Declaration)), (r'(\{)', pushstate_operator_root_callback), (r'(unordered|ordered)(\s*)(\{)', pushstate_operator_order_callback), (r'(declare)(\s+)(ordering)', - bygroups(Keyword, Text, Keyword), 'declareordering'), + bygroups(Keyword.Declaration, Text, Keyword.Declaration), 'declareordering'), (r'(xquery)(\s+)(version)', bygroups(Keyword.Pseudo, Text, Keyword.Pseudo), 'xqueryversion'), - (r'(\(#)', Punctuation, 'pragma'), + (r'(\(#)(\s*)', bygroups(Punctuation, Text), 'pragma'), # sometimes return can occur in root state (r'return', Keyword), - (r'(declare)(\s+)(option)', bygroups(Keyword, Text, Keyword), + (r'(declare)(\s+)(option)', bygroups(Keyword.Declaration, Text, Keyword.Declaration), 'option'), # URI LITERALS - single and double quoted @@ -700,11 +729,16 @@ def pushstate_operator_callback(lexer, match, ctx): (r'then|else', Keyword), - # ML specific + # eXist specific XQUF + (r'(update)(\s*)(insert|delete|replace|value|rename)', bygroups(Keyword, Text, Keyword)), + (r'(into|following|preceding|with)', Keyword), + + # Marklogic specific (r'(try)(\s*)', bygroups(Keyword, Text), 'root'), (r'(catch)(\s*)(\()(\$)', bygroups(Keyword, Text, Punctuation, Name.Variable), 'varname'), + (r'(@'+qname+')', Name.Attribute), (r'(@'+ncname+')', Name.Attribute), (r'@\*:'+ncname, Name.Attribute), @@ -715,6 +749,7 @@ def pushstate_operator_callback(lexer, match, ctx): # STANDALONE QNAMES (qname + r'(?=\s*\{)', Name.Tag, 'qname_braren'), (qname + r'(?=\s*\([^:])', Name.Function, 'qname_braren'), + (r'(' + qname + ')(#)([0-9]+)', bygroups(Name.Function, Keyword.Type, Number.Integer)), (qname, Name.Tag, 'operator'), ] } @@ -731,9 +766,9 @@ class QmlLexer(RegexLexer): # JavascriptLexer above. name = 'QML' - aliases = ['qml'] - filenames = ['*.qml'] - mimetypes = ['application/x-qml'] + aliases = ['qml', 'qbs'] + filenames = ['*.qml', '*.qbs'] + mimetypes = ['application/x-qml', 'application/x-qt.qbs+qml'] # pasted from JavascriptLexer, with some additions flags = re.DOTALL | re.MULTILINE
pygments/lexers/x10.py+69 −0 added@@ -0,0 +1,69 @@ +# -*- coding: utf-8 -*- +""" + pygments.lexers.x10 + ~~~~~~~~~~~~~~~~~~~ + + Lexers for the X10 programming language. + + :copyright: Copyright 2006-2015 by the Pygments team, see AUTHORS. + :license: BSD, see LICENSE for details. +""" + +import re + +from pygments.lexer import RegexLexer +from pygments.token import Text, Comment, Operator, Keyword, Name, String, \ + Number, Punctuation, Error + +__all__ = ['X10Lexer'] + +class X10Lexer(RegexLexer): + """ + For the X10 language. + + .. versionadded:: 0.1 + """ + + name = 'X10' + aliases = ['x10', 'xten'] + filenames = ['*.x10'] + mimetypes = ['text/x-x10'] + + keywords = ( + 'as', 'assert', 'async', 'at', 'athome', 'ateach', 'atomic', + 'break', 'case', 'catch', 'class', 'clocked', 'continue', + 'def', 'default', 'do', 'else', 'final', 'finally', 'finish', + 'for', 'goto', 'haszero', 'here', 'if', 'import', 'in', + 'instanceof', 'interface', 'isref', 'new', 'offer', + 'operator', 'package', 'return', 'struct', 'switch', 'throw', + 'try', 'type', 'val', 'var', 'when', 'while' + ) + + types = ( + 'void' + ) + + values = ( + 'false', 'null', 'self', 'super', 'this', 'true' + ) + + modifiers = ( + 'abstract', 'extends', 'implements', 'native', 'offers', + 'private', 'property', 'protected', 'public', 'static', + 'throws', 'transient' + ) + + tokens = { + 'root': [ + (r'[^\S\n]+', Text), + (r'//.*?\n', Comment.Single), + (r'/\*(.|\n)*?\*/', Comment.Multiline), + (r'\b(%s)\b' % '|'.join(keywords), Keyword), + (r'\b(%s)\b' % '|'.join(types), Keyword.Type), + (r'\b(%s)\b' % '|'.join(values), Keyword.Constant), + (r'\b(%s)\b' % '|'.join(modifiers), Keyword.Declaration), + (r'"(\\\\|\\"|[^"])*"', String), + (r"'\\.'|'[^\\]'|'\\u[0-9a-fA-F]{4}'", String.Char), + (r'.', Text) + ], + }
pygments/styles/arduino.py+2 −1 modified@@ -16,7 +16,8 @@ class ArduinoStyle(Style): """ - The Arduino® language style. This style is designed to highlight the Arduino source code, so exepect the best results with it. + The Arduino® language style. This style is designed to highlight the + Arduino source code, so exepect the best results with it. """ background_color = "#ffffff"
pygments/util.py+2 −2 modified@@ -122,7 +122,7 @@ def text_analyse(text): def shebang_matches(text, regex): - """Check if the given regular expression matches the last part of the + r"""Check if the given regular expression matches the last part of the shebang if one exists. >>> from pygments.util import shebang_matches @@ -160,7 +160,7 @@ def shebang_matches(text, regex): if x and not x.startswith('-')][-1] except IndexError: return False - regex = re.compile('^%s(\.(exe|cmd|bat|bin))?$' % regex, re.IGNORECASE) + regex = re.compile(r'^%s(\.(exe|cmd|bat|bin))?$' % regex, re.IGNORECASE) if regex.search(found) is not None: return True return False
README.rst+1 −1 modified@@ -28,7 +28,7 @@ repository, tickets and pull requests can be viewed. Continuous testing runs on drone.io: .. image:: https://drone.io/bitbucket.org/birkenfeld/pygments-main/status.png - :target: https://drone.io/bitbucket.org/birkenfeld/pygments-main/ + :target: https://drone.io/bitbucket.org/birkenfeld/pygments-main The authors -----------
tests/examplefiles/batchfile.bat+0 −49 removed@@ -1,49 +0,0 @@ -rem this is a demo file. -@rem -@echo off - -call c:\temp.bat somearg -call :lab somearg -rem This next one is wrong in the vim lexer! -call c:temp.bat - -echo "Hi!" -echo hi -echo on -echo off -echo. -@echo off -if exist *.log echo The log file has arrived. -rem These are all escapes, also done incorrectly by the vim lexer -echo ^^ ^> ^< ^| - -x=beginning -setlocal -x = new text -endlocal - -echo testrem x -echo test rem x - -for %%var in (*.jpg) do echo %%var -for /D %%var in (a b c) do echo %%var -for /R C:\temp %%var in (*.jpg) do iexplore.exe %%var -rem Vim has this one wrong too. -for /L %%var in (10,-1,1) do echo %%var -for /F %%var in ("hi!") do echo %%var -for /F "eol=c,skip=1,usebackq" %%var in (`command`) do echo %%var %~l %~fl %~dl %~pl %~nl %~xl %~sl %~al %~tl %~zl %~$PATH:l %~dpl %~dp$PATH:l %~ftzal - -echo some file ?! > somefile.txt - -set PATH=%PATH%;c:\windows - -goto answer%errorlevel% - :answer0 - echo Hi it's zero - :answer1 - echo New - -if exist a del a -else echo A is missing! - -
tests/examplefiles/eg_example1.eg+155 −0 added@@ -0,0 +1,155 @@ +;; filepath tests +require: + library + ./awesome + ./variable/greatness/file + + +;; Strings +"Hello World!" +"Hello!\nWorld!" +'sasadads\ssdasdasd{ @method variable }' +var dstring = .dotted-string + +key-dash +but-can-it-do-ops + + yes-it-can: + 100 + + print "So cool!" + 100 + {that = "is awesome", you = "are hilarious"} + + jiminy-crickets: + oliver-twist each mr-bojangles + + kittens = + 12 + +check-it: + still-works: + {1, 2, 3}.reversed.reversed.awesome{}.that.sort().what.map with + x -> x * x + (1, 2, 3) + this + +if total awesomeness > great stupidity: + print "You've won!" + not-sure-what-this-does @@ but-it-wont-capture +else: + print "Keep trying!" + needs-work ++ you + +;; with tests +needs-obj('awesome') with {"greatness values"} +object.field.method with {1, 2, 3} + +;; object assignment +obj = { + key = "oh yeah" + tree = "schmoh cheah" +} +obj.field = { + that = super cool: "wowzers!" + thatFunc = {x} -> x +} + +;; match statements +match @awesome.great: + {head, *tail} -> [print head; tail] + Array[] -> convert(.arr) + String? -> convert(.str) + else -> throw E.error("This is an error!") + +unimpl-func = -> pass + +;; if / elif / else test +if coolness > 11: + add something: something-else +elif true: + add nothing: something-else +else: + add everything: the-castle + +;; nested if / elif / else test +mad-function = bananas -> + if bananas > 5: + print "You've got a lot of bananas!" + elif bananas == 5: + print "You might want to consider getting more bananas" + else: + print "Go get more bananas now!" + +;; class test +class Dog: + constructor(@name, @breed) = + this.awesomesauce = 100 + + unimpl-meth = -> pass + + bark(@, sd) + +;; error test +try: + throw E.test.my-error("This is my error.") +catch TypeError? e: + print "There was a type error." +catch E.my-error? e: + print "My error!" +catch e: + print "Some other error." +finally: + print "We are done." + +;; method shorthand operator +stream-of-twos = *-> + while [true]: yield 2 ;; need to fix literals with colons after them. + +;; gen shorthand operator +full-name = @-> + '{@first-name} {@last-name}' + +name-method = @-> + @name.parents.full-name() + +;; Keyword highlight test +key x +key +x; key @x; key .x ; key "x"; key 0; .asdasd +key (x); key [x]; key {x} +nokey.x(); nokey{x} +key x + y +key key x +x + key y +x - key y +nokey + x +nokey +key: x +key nokey: y +key x > nokey: z +x + key nokey: z +x and {y, z} +x + nokey: y +x mod nokey: y +x = key: y ;; comments work after keywords! +x each key: y +x each* k ;; +a.b{c.d, e.f} +a.bad-method(c.d, e.f) +#a{} +nokey mod: y ;; do not highlight nokey +;; because mod is an operator + +;; Highlight all of these: +await; break; chain; continue; else:; expr-value +match; return; yield + +;; Edge-case highlighting test +key-word: xyz +nokey - x: yz + +;; Some keywords may contain operators as a subpart. If your regexp +;; uses \b to single out operators like each, is or in, you may +;; fail to highlight these properly: +beaches ;; Do not highlight each inside the word beaches +each-thing ;; Do not highlight each +sleep-in ;; Do not highlight in +before-each: xyz ;; Highlight before-each as a keyword +is-great: xyz ;; Highlight is-great as a keyword + +send()
tests/examplefiles/es6.js+46 −0 added@@ -0,0 +1,46 @@ +// Most examples from https://github.com/rse/es6-features under MIT license +const PI = 3.141593; + +let callbacks = []; + +odds = evens.map(v => v + 1); + +nums.forEach(v => { + if (v % 5 === 0) + fives.push(v); +}) + +function f (x, y, ...a) { + return (x + y) * a.length; +} + +var params = [ "hello", true, 7 ]; +var other = [ 1, 2, ...params ]; // [ 1, 2, "hello", true, 7 ] +f(1, 2, ...params) === 9; + +var str = "foo"; +var chars = [ ...str ]; // [ "f", "o", "o" ] + +var customer = { name: "Foo" }; +var card = { amount: 7, product: "Bar", unitprice: 42 }; +message = `Hello ${customer.name}, +want to buy ${card.amount} ${card.product} for +a total of ${card.amount * card.unitprice} bucks?`; + +0b111110111 === 503; +0o767 === 503; + +for (let codepoint of "𠮷") console.log(codepoint); + +function* (); +*function(); +yield; + +export class Node { +} + +isFinite(); +isNaN(); +isSafeInteger(); +x = new Promise(...a); +x = new Proxy(...a);
tests/examplefiles/example.bat+205 −0 added@@ -0,0 +1,205 @@ +@ @@ echo off +::This is an example of the Windows batch language. + +setlocal EnableDelayedExpansion +(cls) +set/a^ +_te^ +sts^ +=0,^ +_"pa"^ +ssed=0^ +0 +set,/a title= Batch test +title=%,/a title% +echo^ %~nx0,^ the>,con comprehensive testing suite +ver +echo( + +if cmdextversion 2 goto =) +goto :fail + + :) +echo Starting tests at: +date/t & time/t +echo( + +if '%*'=='--help' ( + echo Usage: %~nx0 [--help] + echo --help: Display this help message and quit. + shift + goto :exit comment) else rem + +(call :comments) +call ::io+x +call:control:x +call::internal x + +:exit +if /i !_tests!==!_passed! ( + color 02 +) else if !*==* ( + color c + if not defined _exit^ +Code set _exit^ +Code=1 +) +set _percentage=NaN +if defined _tests ( + if !_tests! neq 0 (set/a_percentage=100*_passed/_tests) +) +echo( +if !_percentage!==NaN ( echo(There were no tests^^! & color e +) else ( echo Tests passed: %_passed%/%_tests% (%_percentage%%%^) ) +pause +color +title +endlocal +exit /b %_exitCode% + +x:fail +rem This should never happen. +echo Internal error 1>& 269105>>&2 +set /a _exitCode=0x69+(0105*1000) +break +goto :exit + +:comments +(rem )/? +) +rem "comment^ +(rem.) & set /a _tests+=1 +(rem) & goto :fail +(rem. ) & (rem. comment ) & echo Test %_tests%: Comments +rem ) +) +)|comment +)( +:: comment +goto :comments^^1:comment +:comments^^1 comment +if(1==1) goto :comments^ +^1 +rem^ /? +rem ^ +^ +goto :comments^ +2+comment +goto :fail +:comments2 +rem >^ +if 1==1 (goto :comments3) +:comments3) +goto :fail +:comments3 +rem comment^ +goto:fail +rem.comment comment^ +goto fail +rem "comment comment"^ +goto fail +rem comment comment^ +set /a _passed+=1 +GOTO :EOF +goto :fail + +:IO +SET /A _tests+=1 & Echo Test !_tests:*!==^!: I/O +verify on +pushd . +if exist temp echo temp already exists. & goto :eof +md temp +cd temp +mkdir 2>nul temp +chdir temp +>cd echo Checking drive... +>>cd echo must be C or else this won't work +for /f "tokens=* usebackq" %%G in ("cd +) do (<nul set /p="%%G ") +echo( +DEL cd +if not "%cd:~0,3%"=="C:\" ( + call call echo Wrong drive (should be C^): + vol + goto :test) +>test0^ +.bat echo rem Machine-generated; do not edit +call echo set /a _passed+=1 >>test0.bat +type test0.bat >"test 1.bat +ren "test 1.bat" test2.bat +rename test2.bat test.bat +caLL ^ +C:test +del test.bat 2>nul +2>NUL erase test0.bat +popd +rd temp\temp +rmdir temp +VERIFY OFF +goto:eof + +:control +set /a _tests+=1 +echo Test %_tests%: Control statements +set "_iterations=0">nul +for %%G in (,+,,-, +) do @( + for /l %%H in (,-1;;-1 -3,) do ( + for /f tokens^=1-2^,5 %%I in ("2 %%H _ _ 10") do ( + for /f "tokens=1 usebackq" %%L in ( `echo %%G%%J ``` ` +` ` ) do ( for /f "tokens=2" %%M in ('echo ' %%L0 ' +' ' ) do ( set /a _iterations+=(%%M%%M^) + ) + ) + ) + ) +) +if exist %~nx0 if not exist %~nx0 goto :fail +if exist %~nx0 ( + if not exist %~nx0 goto :fail +) else ( + if exist %~nx0 goto :fail +) +if /i %_iterations% gtr -2 ( + if /i %_iterations% geq -1 ( + if /i %_iterations% lss 1 ( + if /i %_iterations% leq 0 ( + if /i %_iterations% equ 0 ( + if 1 equ 01 ( + if 1 neq "01" ( + if "1" neq 01 ( + set /a _passed+=1)))))))) +) comment +goto :eof + +:internal +set /a _tests+=1 +echo Test %_tests%: Internal commands +keys on +mklink 2>nul +>nul path %path% +>nul dpath %dpath% +if not defined prompt prompt $P$G +prompt !prompt:~!rem/ $H? +echo on +rem/? +@echo off +rem(/?>nul +rem )/? >nul +(rem (/?) >nul +rem /?>nul +rem^/?>nul +if/?>nul || if^/^?>nul || if /?>nul || if x/? >nul +for/?>nul && for^/^?>nul && for /?>nul && for x/? >nul && for /?x >nul +goto/?>nul && goto^/? && goto^ /? && goto /^ +? && goto /?>nul && goto:/? >nul && goto ) /? ) >nul && (goto /? )>nul +=set+;/p extension'),=.bat +for /f "tokens=2 delims==" %%G in ( 'assoc %+;/p extension'),%' + ) do ( + assoc 2>nul %+;/p extension'),:*.=.%=%%G + ftype 1>nul %%G +) &>nul ver +if errorlevel 0 if not errorlevel 1 set /a _passed+=1 +goto :eof +:/? +goto :fail
tests/examplefiles/example.elm+58 −0 added@@ -0,0 +1,58 @@ +import Math.Vector3 (..) +import Math.Matrix4 (..) +import Graphics.WebGL (..) + +-- Create a mesh with two triangles + +type Vertex = { position:Vec3, color:Vec3 } + +mesh : [Triangle Vertex] +mesh = [ ( Vertex (vec3 0 0 0) (vec3 1 0 0) + , Vertex (vec3 1 1 0) (vec3 0 1 0) + , Vertex (vec3 1 -1 0) (vec3 0 0 1) + ) + ] + +-- Create the scene + +main : Signal Element +main = scene <~ foldp (+) 0 (fps 30) + +scene : Float -> Element +scene t = + webgl (400,400) + [ entity vertexShader fragmentShader mesh { view = view (t / 1000) } ] + +view : Float -> Mat4 +view t = + mul (makePerspective 45 1 0.01 100) + (makeLookAt (vec3 (4 * cos t) 0 (4 * sin t)) (vec3 0 0 0) (vec3 0 1 0)) + +-- Shaders + +vertexShader : Shader { attr | position:Vec3, color:Vec3 } { unif | view:Mat4 } { vcolor:Vec3 } +vertexShader = [glsl| + +attribute vec3 position; +attribute vec3 color; +uniform mat4 view; +varying vec3 vcolor; + +void main () { + gl_Position = view * vec4(position, 1.0); + vcolor = color; +} + +|] + +fragmentShader : Shader {} u { vcolor:Vec3 } +fragmentShader = [glsl| + +precision mediump float; +varying vec3 vcolor; + +void main () { + gl_FragColor = vec4(vcolor, 1.0); +} + +|]
tests/examplefiles/example.ezt+32 −0 added@@ -0,0 +1,32 @@ +* Easytrieve Plus example programm. + +* Environtment section. +PARM DEBUG(FLOW FLDCHK) + +* Library Section. +FILE PERSNL FB(150 1800) + NAME 17 8 A + EMP# 9 5 N * Note: '#' is a valid character for names. + DEPT 98 3 N. GROSS 94 4 P 2 + * ^ 2 field definitions in 1 line. + +* Call macro in example.mac. +FILE EXAMPLE FB(80 200) +%EXAMPLE SOMEFILE SOME + +* Activity Section. +JOB INPUT PERSNL NAME FIRST-PROGRAM START AT-START FINISH AT_FINISH + PRINT PAY-RPT +REPORT PAY-RPT LINESIZE 80 + TITLE 01 'PERSONNEL REPORT EXAMPLE-1' + LINE 01 DEPT NAME EMP# GROSS + +* Procedure declarations. +AT-START. PROC + DISPLAY 'PROCESSING...' +END-PROC + +AT-FINISH +PROC + DISPLAY 'DONE.' +END-PROC
tests/examplefiles/example.fish+580 −0 added@@ -0,0 +1,580 @@ +# ----------------------------------------------------------------------------- +# Fishshell Samples +# |- Theme / bobthefish +# |- Function / funced +# |- Configuration / config.fish +# ----------------------------------------------------------------------------- + +# name: bobthefish +# +# bobthefish is a Powerline-style, Git-aware fish theme optimized for awesome. +# +# You will probably need a Powerline-patched font for this to work: +# +# https://powerline.readthedocs.org/en/latest/fontpatching.html +# +# I recommend picking one of these: +# +# https://github.com/Lokaltog/powerline-fonts +# +# You can override some default options in your config.fish: +# +# set -g theme_display_user yes +# set -g default_user your_normal_user + +set -g __bobthefish_current_bg NONE + +# Powerline glyphs +set __bobthefish_branch_glyph \uE0A0 +set __bobthefish_ln_glyph \uE0A1 +set __bobthefish_padlock_glyph \uE0A2 +set __bobthefish_right_black_arrow_glyph \uE0B0 +set __bobthefish_right_arrow_glyph \uE0B1 +set __bobthefish_left_black_arrow_glyph \uE0B2 +set __bobthefish_left_arrow_glyph \uE0B3 + +# Additional glyphs +set __bobthefish_detached_glyph \u27A6 +set __bobthefish_nonzero_exit_glyph '! ' +set __bobthefish_superuser_glyph '$ ' +set __bobthefish_bg_job_glyph '% ' +set __bobthefish_hg_glyph \u263F + +# Python glyphs +set __bobthefish_superscript_glyph \u00B9 \u00B2 \u00B3 +set __bobthefish_virtualenv_glyph \u25F0 +set __bobthefish_pypy_glyph \u1D56 + +# Colors +set __bobthefish_lt_green addc10 +set __bobthefish_med_green 189303 +set __bobthefish_dk_green 0c4801 + +set __bobthefish_lt_red C99 +set __bobthefish_med_red ce000f +set __bobthefish_dk_red 600 + +set __bobthefish_slate_blue 255e87 + +set __bobthefish_lt_orange f6b117 +set __bobthefish_dk_orange 3a2a03 + +set __bobthefish_dk_grey 333 +set __bobthefish_med_grey 999 +set __bobthefish_lt_grey ccc + +set __bobthefish_dk_brown 4d2600 +set __bobthefish_med_brown 803F00 +set __bobthefish_lt_brown BF5E00 + +set __bobthefish_dk_blue 1E2933 +set __bobthefish_med_blue 275379 +set __bobthefish_lt_blue 326D9E + +# =========================== +# Helper methods +# =========================== + +function __bobthefish_in_git -d 'Check whether pwd is inside a git repo' + command which git > /dev/null 2>&1; and command git rev-parse --is-inside-work-tree >/dev/null 2>&1 +end + +function __bobthefish_in_hg -d 'Check whether pwd is inside a hg repo' + command which hg > /dev/null 2>&1; and command hg stat > /dev/null 2>&1 +end + +function __bobthefish_git_branch -d 'Get the current git branch (or commitish)' + set -l ref (command git symbolic-ref HEAD 2> /dev/null) + if [ $status -gt 0 ] + set -l branch (command git show-ref --head -s --abbrev |head -n1 2> /dev/null) + set ref "$__bobthefish_detached_glyph $branch" + end + echo $ref | sed "s-refs/heads/-$__bobthefish_branch_glyph -" +end + +function __bobthefish_hg_branch -d 'Get the current hg branch' + set -l branch (hg branch ^/dev/null) + set -l book " @ "(hg book | grep \* | cut -d\ -f3) + echo "$__bobthefish_branch_glyph $branch$book" +end + +function __bobthefish_pretty_parent -d 'Print a parent directory, shortened to fit the prompt' + echo -n (dirname $argv[1]) | sed -e 's|/private||' -e "s|^$HOME|~|" -e 's-/\(\.\{0,1\}[^/]\)\([^/]*\)-/\1-g' -e 's|/$||' +end + +function __bobthefish_git_project_dir -d 'Print the current git project base directory' + command git rev-parse --show-toplevel 2>/dev/null +end + +function __bobthefish_hg_project_dir -d 'Print the current hg project base directory' + command hg root 2>/dev/null +end + +function __bobthefish_project_pwd -d 'Print the working directory relative to project root' + echo "$PWD" | sed -e "s*$argv[1]**g" -e 's*^/**' +end + + +# =========================== +# Segment functions +# =========================== + +function __bobthefish_start_segment -d 'Start a prompt segment' + set_color -b $argv[1] + set_color $argv[2] + if [ "$__bobthefish_current_bg" = 'NONE' ] + # If there's no background, just start one + echo -n ' ' + else + # If there's already a background... + if [ "$argv[1]" = "$__bobthefish_current_bg" ] + # and it's the same color, draw a separator + echo -n "$__bobthefish_right_arrow_glyph " + else + # otherwise, draw the end of the previous segment and the start of the next + set_color $__bobthefish_current_bg + echo -n "$__bobthefish_right_black_arrow_glyph " + set_color $argv[2] + end + end + set __bobthefish_current_bg $argv[1] +end + +function __bobthefish_path_segment -d 'Display a shortened form of a directory' + if test -w "$argv[1]" + __bobthefish_start_segment $__bobthefish_dk_grey $__bobthefish_med_grey + else + __bobthefish_start_segment $__bobthefish_dk_red $__bobthefish_lt_red + end + + set -l directory + set -l parent + + switch "$argv[1]" + case / + set directory '/' + case "$HOME" + set directory '~' + case '*' + set parent (__bobthefish_pretty_parent "$argv[1]") + set parent "$parent/" + set directory (basename "$argv[1]") + end + + test "$parent"; and echo -n -s "$parent" + set_color fff --bold + echo -n "$directory " + set_color normal +end + +function __bobthefish_finish_segments -d 'Close open prompt segments' + if [ -n $__bobthefish_current_bg -a $__bobthefish_current_bg != 'NONE' ] + set_color -b normal + set_color $__bobthefish_current_bg + echo -n "$__bobthefish_right_black_arrow_glyph " + set_color normal + end + set -g __bobthefish_current_bg NONE +end + + +# =========================== +# Theme components +# =========================== + +function __bobthefish_prompt_status -d 'Display symbols for a non zero exit status, root and background jobs' + set -l nonzero + set -l superuser + set -l bg_jobs + + # Last exit was nonzero + if [ $status -ne 0 ] + set nonzero $__bobthefish_nonzero_exit_glyph + end + + # if superuser (uid == 0) + set -l uid (id -u $USER) + if [ $uid -eq 0 ] + set superuser $__bobthefish_superuser_glyph + end + + # Jobs display + if [ (jobs -l | wc -l) -gt 0 ] + set bg_jobs $__bobthefish_bg_job_glyph + end + + set -l status_flags "$nonzero$superuser$bg_jobs" + + if test "$nonzero" -o "$superuser" -o "$bg_jobs" + __bobthefish_start_segment fff 000 + if [ "$nonzero" ] + set_color $__bobthefish_med_red --bold + echo -n $__bobthefish_nonzero_exit_glyph + end + + if [ "$superuser" ] + set_color $__bobthefish_med_green --bold + echo -n $__bobthefish_superuser_glyph + end + + if [ "$bg_jobs" ] + set_color $__bobthefish_slate_blue --bold + echo -n $__bobthefish_bg_job_glyph + end + + set_color normal + end +end + +function __bobthefish_prompt_user -d 'Display actual user if different from $default_user' + if [ "$theme_display_user" = 'yes' ] + if [ "$USER" != "$default_user" -o -n "$SSH_CLIENT" ] + __bobthefish_start_segment $__bobthefish_lt_grey $__bobthefish_slate_blue + echo -n -s (whoami) '@' (hostname | cut -d . -f 1) ' ' + end + end +end + +function __bobthefish_prompt_hg -d 'Display the actual hg state' + set -l dirty (command hg stat; or echo -n '*') + + set -l flags "$dirty" + test "$flags"; and set flags "" + + set -l flag_bg $__bobthefish_lt_green + set -l flag_fg $__bobthefish_dk_green + if test "$dirty" + set flag_bg $__bobthefish_med_red + set flag_fg fff + end + + __bobthefish_path_segment (__bobthefish_hg_project_dir) + + __bobthefish_start_segment $flag_bg $flag_fg + echo -n -s $__bobthefish_hg_glyph ' ' + + __bobthefish_start_segment $flag_bg $flag_fg + set_color $flag_fg --bold + echo -n -s (__bobthefish_hg_branch) $flags ' ' + set_color normal + + set -l project_pwd (__bobthefish_project_pwd (__bobthefish_hg_project_dir)) + if test "$project_pwd" + if test -w "$PWD" + __bobthefish_start_segment 333 999 + else + __bobthefish_start_segment $__bobthefish_med_red $__bobthefish_lt_red + end + + echo -n -s $project_pwd ' ' + end +end + +# TODO: clean up the fugly $ahead business +function __bobthefish_prompt_git -d 'Display the actual git state' + set -l dirty (command git diff --no-ext-diff --quiet --exit-code; or echo -n '*') + set -l staged (command git diff --cached --no-ext-diff --quiet --exit-code; or echo -n '~') + set -l stashed (command git rev-parse --verify refs/stash > /dev/null 2>&1; and echo -n '$') + set -l ahead (command git branch -v 2> /dev/null | grep -Eo '^\* [^ ]* *[^ ]* *\[[^]]*\]' | grep -Eo '\[[^]]*\]$' | awk 'ORS="";/ahead/ {print "+"} /behind/ {print "-"}' | sed -e 's/+-/±/') + + set -l new (command git ls-files --other --exclude-standard); + test "$new"; and set new '…' + + set -l flags "$dirty$staged$stashed$ahead$new" + test "$flags"; and set flags " $flags" + + set -l flag_bg $__bobthefish_lt_green + set -l flag_fg $__bobthefish_dk_green + if test "$dirty" -o "$staged" + set flag_bg $__bobthefish_med_red + set flag_fg fff + else + if test "$stashed" + set flag_bg $__bobthefish_lt_orange + set flag_fg $__bobthefish_dk_orange + end + end + + __bobthefish_path_segment (__bobthefish_git_project_dir) + + __bobthefish_start_segment $flag_bg $flag_fg + set_color $flag_fg --bold + echo -n -s (__bobthefish_git_branch) $flags ' ' + set_color normal + + set -l project_pwd (__bobthefish_project_pwd (__bobthefish_git_project_dir)) + if test "$project_pwd" + if test -w "$PWD" + __bobthefish_start_segment 333 999 + else + __bobthefish_start_segment $__bobthefish_med_red $__bobthefish_lt_red + end + + echo -n -s $project_pwd ' ' + end +end + +function __bobthefish_prompt_dir -d 'Display a shortened form of the current directory' + __bobthefish_path_segment "$PWD" +end + +function __bobthefish_in_virtualfish_virtualenv + set -q VIRTUAL_ENV +end + +function __bobthefish_virtualenv_python_version -d 'Get current python version' + switch (readlink (which python)) + case python2 + echo $__bobthefish_superscript_glyph[2] + case python3 + echo $__bobthefish_superscript_glyph[3] + case pypy + echo $__bobthefish_pypy_glyph + end +end + +function __bobthefish_virtualenv -d 'Get the current virtualenv' + echo $__bobthefish_virtualenv_glyph(__bobthefish_virtualenv_python_version) (basename "$VIRTUAL_ENV") +end + +function __bobthefish_prompt_virtualfish -d "Display activated virtual environment (only for virtualfish, virtualenv's activate.fish changes prompt by itself)" + set flag_bg $__bobthefish_lt_blue + set flag_fg $__bobthefish_dk_blue + __bobthefish_start_segment $flag_bg $flag_fg + set_color $flag_fg --bold + echo -n -s (__bobthefish_virtualenv) $flags ' ' + set_color normal +end + + +# =========================== +# Apply theme +# =========================== + +function fish_prompt -d 'bobthefish, a fish theme optimized for awesome' + __bobthefish_prompt_status + __bobthefish_prompt_user + if __bobthefish_in_virtualfish_virtualenv + __bobthefish_prompt_virtualfish + end + if __bobthefish_in_git # TODO: do this right. + __bobthefish_prompt_git # if something is in both git and hg, check the length of + else if __bobthefish_in_hg # __bobthefish_git_project_dir vs __bobthefish_hg_project_dir + __bobthefish_prompt_hg # and pick the longer of the two. + else + __bobthefish_prompt_dir + end + __bobthefish_finish_segments +end + +# ----------------------------------------------------------------------------- +# funced - edit a function interactively +# +# Synopsis +# +# funced [OPTIONS] NAME +# +# Description +# +# funced provides an interface to edit the definition of the function NAME. +# ----------------------------------------------------------------------------- + +function funced --description 'Edit function definition' + set -l editor $EDITOR + set -l interactive + set -l funcname + while set -q argv[1] + switch $argv[1] + case -h --help + __fish_print_help funced + return 0 + + case -e --editor + set editor $argv[2] + set -e argv[2] + + case -i --interactive + set interactive 1 + + case -- + set funcname $funcname $argv[2] + set -e argv[2] + + case '-*' + set_color red + printf (_ "%s: Unknown option %s\n") funced $argv[1] + set_color normal + return 1 + + case '*' '.*' + set funcname $funcname $argv[1] + end + set -e argv[1] + end + + if begin; set -q funcname[2]; or not test "$funcname[1]"; end + set_color red + _ "funced: You must specify one function name +" + set_color normal + return 1 + end + + set -l init + switch $funcname + case '-*' + set init function -- $funcname\n\nend + case '*' + set init function $funcname\n\nend + end + + # Break editor up to get its first command (i.e. discard flags) + if test -n "$editor" + set -l editor_cmd + eval set editor_cmd $editor + if not type -f "$editor_cmd[1]" >/dev/null + _ "funced: The value for \$EDITOR '$editor' could not be used because the command '$editor_cmd[1]' could not be found + " + set editor fish + end + end + + # If no editor is specified, use fish + if test -z "$editor" + set editor fish + end + + if begin; set -q interactive[1]; or test "$editor" = fish; end + set -l IFS + if functions -q -- $funcname + # Shadow IFS here to avoid array splitting in command substitution + set init (functions -- $funcname | fish_indent --no-indent) + end + + set -l prompt 'printf "%s%s%s> " (set_color green) '$funcname' (set_color normal)' + # Unshadow IFS since the fish_title breaks otherwise + set -e IFS + if read -p $prompt -c "$init" -s cmd + # Shadow IFS _again_ to avoid array splitting in command substitution + set -l IFS + eval (echo -n $cmd | fish_indent) + end + return 0 + end + + set -q TMPDIR; or set -l TMPDIR /tmp + set -l tmpname (printf "$TMPDIR/fish_funced_%d_%d.fish" %self (random)) + while test -f $tmpname + set tmpname (printf "$TMPDIR/fish_funced_%d_%d.fish" %self (random)) + end + + if functions -q -- $funcname + functions -- $funcname > $tmpname + else + echo $init > $tmpname + end + if eval $editor $tmpname + . $tmpname + end + set -l stat $status + rm -f $tmpname >/dev/null + return $stat +end + +# ----------------------------------------------------------------------------- +# Main file for fish command completions. This file contains various +# common helper functions for the command completions. All actual +# completions are located in the completions subdirectory. +## ----------------------------------------------------------------------------- + +# +# Set default field separators +# + +set -g IFS \n\ \t + +# +# Set default search paths for completions and shellscript functions +# unless they already exist +# + +set -l configdir ~/.config + +if set -q XDG_CONFIG_HOME + set configdir $XDG_CONFIG_HOME +end + +# __fish_datadir, __fish_sysconfdir, __fish_help_dir, __fish_bin_dir +# are expected to have been set up by read_init from fish.cpp + +# Set up function and completion paths. Make sure that the fish +# default functions/completions are included in the respective path. + +if not set -q fish_function_path + set fish_function_path $configdir/fish/functions $__fish_sysconfdir/functions $__fish_datadir/functions +end + +if not contains $__fish_datadir/functions $fish_function_path + set fish_function_path[-1] $__fish_datadir/functions +end + +if not set -q fish_complete_path + set fish_complete_path $configdir/fish/completions $__fish_sysconfdir/completions $__fish_datadir/completions +end + +if not contains $__fish_datadir/completions $fish_complete_path + set fish_complete_path[-1] $__fish_datadir/completions +end + +# +# This is a Solaris-specific test to modify the PATH so that +# Posix-conformant tools are used by default. It is separate from the +# other PATH code because this directory needs to be prepended, not +# appended, since it contains POSIX-compliant replacements for various +# system utilities. +# + +if test -d /usr/xpg4/bin + if not contains /usr/xpg4/bin $PATH + set PATH /usr/xpg4/bin $PATH + end +end + +# +# Add a few common directories to path, if they exists. Note that pure +# console programs like makedep sometimes live in /usr/X11R6/bin, so we +# want this even for text-only terminals. +# + +set -l path_list /bin /usr/bin /usr/X11R6/bin /usr/local/bin $__fish_bin_dir + +# Root should also have the sbin directories in the path +switch $USER + case root + set path_list $path_list /sbin /usr/sbin /usr/local/sbin +end + +for i in $path_list + if not contains $i $PATH + if test -d $i + set PATH $PATH $i + end + end +end + +# +# Launch debugger on SIGTRAP +# +function fish_sigtrap_handler --on-signal TRAP --no-scope-shadowing --description "Signal handler for the TRAP signal. Lanches a debug prompt." + breakpoint +end + +# +# Whenever a prompt is displayed, make sure that interactive +# mode-specific initializations have been performed. +# This handler removes itself after it is first called. +# +function __fish_on_interactive --on-event fish_prompt + __fish_config_interactive + functions -e __fish_on_interactive +end
tests/examplefiles/example.jcl+31 −0 added@@ -0,0 +1,31 @@ +//IS198CPY JOB (PYGM-TEST-001),'PYGMENTS TEST JOB', +// CLASS=L,MSGCLASS=X,TIME=(00,10) +//* Copy 'OLDFILE' to 'NEWFILE'. +//COPY01 EXEC PGM=IEBGENER +//SYSPRINT DD SYSOUT=* +//SYSUT1 DD DSN=OLDFILE,DISP=SHR +//SYSUT2 DD DSN=NEWFILE, +// DISP=(NEW,CATLG,DELETE), +// SPACE=(CYL,(40,5),RLSE), Some comment +// DCB=(LRECL=115,BLKSIZE=1150) +//SYSIN DD DUMMY +/* +//* Test line continuation in strings. +//CONT01 EXEC PGM=IEFBR14,PARM='THIS IS A LONG PARAMETER WITHIN APOST +// ROPHES, CONTINUED IN COLUMN 15 OF THE NEXT RECORD' +//* Sort a couple of lines and show the result in the job log. +//SORT01 EXEC PGM=IEFBR14 +//SORTIN DD * +spam +eggs +ham +/* +//SORTOUT DD SYSOUT=* +/* +//* Test line continuation with comment at end of line continued by a +//* character at column 72 (in this case 'X'). +//STP4 EXEC PROC=BILLING,COND.PAID=((20,LT),EVEN), +// COND.LATE=(60,GT,FIND), +// COND.BILL=((20,GE),(30,LT,CHGE)) THIS STATEMENT CALLS THE X +// BILLING PROCEDURE AND SPECIFIES RETURN CODE TESTS FOR THREEX +// PROCEDURE STEPS.
tests/examplefiles/example.mac+6 −0 added@@ -0,0 +1,6 @@ +* Example Easytrieve macro declaration. For an example on calling this +* macro, see example.ezt. +MACRO FILENAME PREFIX +&FILENAME. +&PREFIX.-LINE 1 80 A +&PREFIX.-KEY 1 8 A
tests/examplefiles/example.pcmk+115 −0 added@@ -0,0 +1,115 @@ +node 167906355: sle12-a +node 167906357: sle12-c \ + description="The second node" \ + utilization memory=64 +node node1 \ + attributes mem=16G +node node2 utilization cpu=4 +primitive st stonith:ssh \ + params hostlist="node1 node2" \ + meta target-role="Started" \ + op start requires=nothing timeout=60s \ + op monitor interval=60m timeout=60s +primitive d1 ocf:pacemaker:Dummy \ + operations $id=d1-ops \ + op monitor interval=60m \ + op monitor interval=120m OCF_CHECK_LEVEL=10 +primitive fs1 Filesystem \ + params device="/dev/nfs-vg/fs1" directory="/srv/nfs" fstype=ext3 \ + op monitor interval=10s +primitive nfs-server nfsserver \ + params nfs_shared_infodir="/srv/nfs/state" nfs_ip=10.2.12.100 \ + op monitor interval=0 trace_ra=1 +primitive nfs-vg LVM \ + params volgrpname=nfs-vg +primitive p_drbd_nfs ocf:linbit:drbd \ + params drbd_resource=nfs \ + op monitor interval=15 role=Master \ + op monitor interval=30 role=Slave \ + op start interval=0 timeout=300 \ + op stop interval=0 timeout=120 +primitive s-libvirt stonith:external/libvirt \ + params hostlist="sle12-a sle12-c" hypervisor_uri="qemu+ssh://hex-10.suse.de/system?keyfile=/root/.ssh/xen" reset_method=reboot \ + op monitor interval=5m timeout=60s +primitive virtual-ip IPaddr2 \ + params ip=10.2.12.100 +primitive xen0 @vm_scheme1 xmfile=/etc/xen/vm/xen0 +primitive d7 Dummy \ + params rule inf: #uname eq node1 fake=1 \ + params rule inf: #uname eq node2 fake=2 +primitive very-primitive Dummy \ + params 3: rule #uname eq node1 interface=eth1 \ + params 2: rule #uname string:eq node2 interface=eth2 port=8888 \ + params 1: interface=eth0 port=9999 \ + operations $id-ref=those_other_ops +fencing_topology poison-pill power +fencing_topology \ + node-a: poison-pill power \ + node-b: ipmi serial +role nfs_admin \ + write meta:nfs-server:target-role \ + write meta:nfs-server:is-managed \ + write location:nfs-server \ + read ref:nfs-server +role basic-read \ + read status \ + read type:node attribute:uname \ + read type:node attribute:type \ + read property +role basic-read-basic \ + read cib +role d0-admin \ + write meta:d0:target-role \ + write meta:d0:is-managed \ + read xpath:"//nodes//attributes" \ + read ref:d0 +acl_target joe \ + nfs_admin +tag nfs: nfs-server nfs-vg +group nfs-disk nfs-vg fs1 +group nfs-srv virtual-ip nfs-server +ms ms_drbd_nfs p_drbd_nfs \ + meta notify=true clone-max=2 +location nfs-pref virtual-ip 100: sle12-a +location l1 nfs-srv 100: node1 +location l2 d1 \ + rule 100: #uname eq node1 +location l3 d1 \ + rule inf: #uname eq node1 and pingd gt 0 +location l4 d1 \ + rule -inf: not_defined pingd or pingd lte 0 +location l5 fs1 \ + rule -inf: not_defined pingd or pingd lte 0 \ + rule #uname eq node1 and pingd gt 0 \ + rule date lt 2009-05-26 and date in start=2009-05-26 end=2009-07-26 and date in start=2009-05-26 years=2009 and date spec years=2009 hours=09-17 +location l6 d1 \ + rule $id-ref=l2-rule1 +location l7 d1 \ + rule $id-ref=l2 +colocation c-nfs inf: nfs-server fs1 +colocation vg-with-drbd inf: nfs-vg ms_drbd_nfs:Master +# drbd device is the nfs-vg PV +order drbd-before-vg inf: ms_drbd_nfs:promote nfs-vg:start +# need fs1 for the NFS server +order o-nfs inf: fs1 nfs-server +rsc_ticket ticket-A_m6 ticket-A: d1 +rsc_ticket ticket-B_m6_m5 ticket-B: d1 d7 loss-policy=fence +rsc_ticket ticket-C_master ticket-C: d1 ms_drbd_nfs:Master loss-policy=fence +property cpset2: \ + maintenance-mode=true +property cib-bootstrap-options: \ + dc-version=1.1.12-ad083a8 \ + cluster-infrastructure=corosync \ + cluster-name=sle12-test3l-public \ + no-quorum-policy=ignore \ + startup-fencing=false \ + last-lrm-refresh=1415877622 \ + maintenance-mode=false +op_defaults op-options: \ + timeout=120s +rsc_defaults rsc-options: \ + failure-timeout=10m +op_defaults opsdef2: \ + rule 100: #uname eq node1 \ + record-pending=true +tag t1: d1 d7 opsdef2
tests/examplefiles/example.praat+245 −0 added@@ -0,0 +1,245 @@ +form Highlighter test + sentence Blank + sentence My_sentence This should all be a string + text My_text This should also all be a string + word My_word Only the first word is a string, the rest is invalid + boolean Binary 1 + boolean Text no + boolean Quoted "yes" + comment This should be a string + real left_Range -123.6 + positive right_Range_max 3.3 + integer Int 4 + natural Nat 4 +endform + +# External scripts +include /path/to/file +runScript: "/path/to/file" +execute /path/to/file + +# Predefined variables +a = praatVersion +a = e +a = pi +a$ = homeDirectory$ + tab$ + newline$ +a$ = temporaryDirectory$ +a$ = praatVersion$ +a$ = shellDirectory$ +a$ = homeDirectory$ +a$ = preferencesDirectory$ +a$ = defaultDirectory$ +nocheck selectObject: undefined + +# Arrays are not comments +a# = zero# (5, 6) +a [3], 5 = 7 +printline 'a[3,5]', 'a[3]' +a [1] = 2 +b [a [1]] = 3 +assert b [a [1]] = 3 +printline 'b[2]' + +# if-block with built-in variables +if windows + # We are on Windows +elsif unix = 1 or !macintosh + exitScript: "We are on Linux" +else macintosh == 1 + exit We are on Mac +endif + +string$ = "Strings can be 'interpolated'" +string$ = "But don't interpolate everything!" + +Text... 1 Right 0.2 Half many----hyphens +Text... 1 Right -0.4 Bottom aحبيبa +Text... 1 Right -0.6 Bottom 日本 +Draw circle (mm)... 0.5 0.5 i +x=1 + +rows = Object_'table'.nrow +value$ = Table_'table'$[25, "f0"] +fixed = Sound_10.xmin +fixed = Object_foo.xmin +fixed = Procrustes_foo.nx + +# old-style procedure call +call oldStyle "quoted" 2 unquoted string +assert oldStyle.local = 1 + +# New-style procedure call with parens +@newStyle("quoted", 2, "quoted string") +if praatVersion >= 5364 + # New-style procedure call with colon + @newStyle: "quoted", 2, "quoted string" +endif + +# inline if with inline comment +var = if macintosh = 1 then 0 else 1 fi ; This is an inline comment + +# for-loop with explicit from using local variable +# and paren-style function calls and variable interpolation +n = numberOfSelected("Sound") +for i from newStyle.local to n + name = selected$(extractWord$(selected$(), " ")) + sound'i' = selected("Sound", i) + sound[i] = sound'i' +endfor + +for i from 1 to n + # Different styles of object selection + select sound'i' + sound = selected() + sound$ = selected$("Sound") + select Sound 'sound$' + selectObject(sound[i]) + selectObject: sound + + # Pause commands + beginPause("Viewing " + sound$) + if i > 1 + button = endPause("Stop", "Previous", + ...if i = total_sounds then "Finish" else "Next" fi, + ...3, 1) + else + button = endPause("Stop", + ...if i = total_sounds then "Finish" else "Next" fi, + ...2, 1) + endif + editor_name$ = if total_textgrids then "TextGrid " else "Sound " fi + name$ + nocheck editor Sound 'editor_name$' + nocheck Close + nocheck endeditor + editor_id = editor: editor_name$ + Close + endeditor + + # New-style standalone command call + Rename: "SomeName" + + # Command call with assignment + duration = Get total duration + + # Multi-line command with modifier + pitch = noprogress To Pitch (ac): 0, 75, 15, "no", + ...0.03, 0.45, 0.01, 0.35, 0.14, 600 + + # do-style command with assignment + minimum = do("Get minimum...", 0, 0, "Hertz", "Parabolic") + + # New-style multi-line command call with broken strings + table = Create Table with column names: "table", 0, + ..."file subject speaker + ...f0 f1 f2 f3 " + + ..."duration response" + + # Function call with trailing space + removeObject: pitch, table + + # Picture window commands + selectObject: sound + # do-style command + do("Select inner viewport...", 1, 6, 0.5, 1.5) + Black + Draw... 0 0 0 0 "no" Curve + Draw inner box + Text bottom: "yes", sound$ + Erase all + + # Demo window commands + demo Erase all + demo Select inner viewport... 0 100 0 100 + demo Axes... 0 100 0 100 + demo Paint rectangle... white 0 100 0 100 + demo Text... 50 centre 50 half Click to finish + demoWaitForInput ( ) + demo Erase all + demo Text: 50, "centre", 50, "half", "Finished" +endfor + +switch$ = if switch == 1 then "a" else + ... if switch == 2 then "b" else + ... if switch == 3 then "c" else + ... if switch == 4 then "d" else + ... "default" fi fi fi fi + +# An old-style sendpraat block +# All these lines should be a string! +sendpraat Praat + ...'newline$' Create Sound as pure tone... "tone" 1 0 0.4 44100 440 0.2 0.01 0.01 + ...'newline$' Play + ...'newline$' Remove + +# A new-style sendpraat block +beginSendPraat: "Praat" + Create Sound as pure tone: "tone", 1, 0, 0.4, 44100, 440, 0.2, 0.01, 0.01 + duration = Get total duration + Remove +endSendPraat: "duration" +appendInfoLine: "The generated sound lasted for ", duration, "seconds" + +# Number types +a = 10% +a = -10 +a = +10 +a = 10.4 +a = 294e12 +a = 2.94e12 + +# Operators +a = 2 ^ -6 +a = -(1+1)^6 +a = 4^3 ^ 2 +a = 54 div 5.1 +a = 54.3 mod 5 +a = 3 ** 8 - 7 +a = 3 / (8 + 7) +a = (7 * (3 + 5)) / ((2 + 3) - 1) + +# Logical operators +assert (a = b) and c +assert a == (b or c) +assert a <= b not c +assert a >= b !c +assert a != b & c +assert a != b && c +assert a <> b || c +assert a < b | c +assert a > b +assert "hello" = "he" + "llo" +assert "hello" == "hello world" - " world" + +stopwatch +time = stopwatch +clearinfo +echo This script took +print 'time' seconds to +printline execute. + +# Old-style procedure declaration +procedure oldStyle .str1$ .num .str2$ + .local = 1 +endproc + +# New-style procedure declaration with parentheses +procedure newStyle (.str1$, .num, .str2$) + # Command with "local" variable + .local = Get total duration + .local = Get 'some' duration + .local = Get 'some[1]' value... hello 10 p[i] + .local = Get 'some[1,3]' value: "hello", 10, 'p[i]' + .local = Get 'some$' duration + .local = Get 'some$[1]' duration +endproc + +# New-style procedure declaration with colon +procedure _new_style: .str1$, .num, .str2$ + # Command with "local" variable + # Initial underscores in variables not allowed (unless interpolated) + _new_style.local = Get total duration +endproc + +asserterror Unknown symbol:'newline$'« _ +assert '_new_style.local' +
tests/examplefiles/example.rts+118 −0 added@@ -0,0 +1,118 @@ +# Example of a Riverbed TrafficScript (*.rts) file. + +http.setHeader( "Host", "secure.mysite.com" ); +$body = http.getBody( ); # get the POST data +$single = 'Hello \ +world'; +$double = "Hello \ +world"; +$pi = 3.14157; +$message = "The URL path is " . http.GetPath(); +$four = 2 + 2; +# Sets $ratio to "75%" (for example) +$ratio = ( $a / ($a + $b) * 100 ) . "%"; +$contentLength = http.getHeader( "Content-Length" ); +if( $contentLength > 1024 * 1024 ) { + log.warn( "Large request body: ".$contentLength ); +} +4 + 7.5 * $a +-$b / $c - 1 +7 % 3 # Returns 1 +"foo" && !0 # true +( 1 < 2 ) && ( 3 < 4 ) # true +$a || $b # true if $a or $b is true +0x1234 & 255 # 0x34 +1|2|4 #7 +1^3 #2 +~1 & 0xffff # 65534 +1 << 2 # 4 +2 >> 1 # 1 +$foo *= 5 # Product equals ($foo = $foo * 5) +$foo /= 2 # Quotient equals ($foo = $foo / 5) +$foo %= 2 # Modulo equals ($foo = $foo % 5) +$foo <<= 2 # Bit-shift left equals ($foo = $foo << 2) +$foo >>= 2 # Bit-shift right equals ($foo = $foo >> 2) +$foo &= 2 # Bitwise AND equals ($foo = $foo & 2) +$foo |= 2 # Bitwise OR equals ($foo = $foo | 2) +$foo ^= 2 # Bitwise XOR equals ($foo = $foo ^ 2) +$int = 10; +$double = 2.71828; +string.len( $double ); # casts to string, returns 7 +# Convert $string to a number, and add 4: +$r = $string + 4; # $r is 14 +if( string.startsWith( $path, "/secure" ) ) { + pool.use( "secure pool" ); +} else { + pool.use( "non-secure pool" ); +} + +for( $count = 0; $count < 10; $count++ ) { + log.info( "In loop, count = " . $count ); +} + +i$count = 0; +while( $count < 10 ) { + log.info( "In loop, count = " . $count ); + $count = $count + 1; +} + +$count = 0; +do { + log.info( "In loop, count = " . $count ); + $count = $count + 1; +} while( $count < 10 ); + +$mime = http.getResponseHeader( "Content-Type" ); +if( !string.startsWith( $mime, "text/html" )) break; +$array = [ "Alex", "Matt", "Oliver", "Laurence" ]; +$someone = $array[0]; +$arraylen = array.length($array); +log.info("My array has " . $arraylen . " elements.\n"); + +for ( $i = 0; $i < $arraylen; $i++ ){ + log.info ( "Element #" . $i . " " . $array[$i]); +} + +$hash = [ "orange" => "fruit", + "apple" => "fruit", + "cabbage" => "vegetable", + "pear" => "fruit" ]; + +foreach ( $key in hash.keys($hash)){ + log.info("Key: " . $key . "; Value: " . $hash[$key] . +";"); } + +# Declare a subroutine to calculate factorials +sub factorial( $n ) { + if( $n == 0 ) return 1; + return $n*factorial( $n-1 ); +} +# Put entries into the array +$c = 0; +while( $c <= 10 ) { + $msg = "Did you know that ". $c ."! is ". factorial( $c ) +."?" ; + data.set( "myarray".$c, $msg ); +$c++; } +# Look up several entries. Note: the 1000th entry is empty +$msg = ""; +$msg .= "Index 1000: ".data.get( "myarray1000" )."\n"; +# delete the entire array (but no other data stored by data.set) +data.reset( "myarray" ); +http.sendResponse( "200 OK", "text/plain", $msg, "" ); +sub headbug(){ + # Prints each header to the event log. + $headers = http.listHeaderNames(); + foreach ($header in $headers){ + log.info( $header . ": " . http.getheader($header)); +} } + +import foo; +foo.headbug(); +# Sets the regex string as ^192\.168\. ; the two examples +# below have the same effect +$regex = "^(192)\\.168\\."; +$regex = '^192\.168\.'; +if ( string.regexMatch( $ip, $regex ) ) { + # IP is on 192.168.* network +}
tests/examplefiles/example.scd+76 −0 added@@ -0,0 +1,76 @@ +Instr("cs.fm.BasicFM", { + arg freq = 440, + amp = 0.9, + gate = 0, + carrierFreqRatio = 1.0, + modulatorFreqRatio = 1.0, + // not sure if having these defaults here actually does anything. + modEnvShape = Env.adsr( + attackTime: 0.05, + decayTime: 0.1, + sustainLevel: 0.5 * amp, + releaseTime: 0.1, + peakLevel: amp, + curve: [4, -4, -2] + ), + carrierEnvShape = Env.adsr( + attackTime: 0.05, + decayTime: 0.1, + sustainLevel: 0.5 * amp, + releaseTime: 0.1, + peakLevel: amp, + curve: [4, -4, -2] + ); + + var carrier, + modulator, + carrierEnv, + modEnv, + out; + + modEnv = EnvGen.kr( + envelope: modEnvShape, + gate: gate + ); + + modulator = modEnv * SinOsc.ar(freq * modulatorFreqRatio); + + // carrier sustains until noteoff + carrierEnvShape.releaseNode = 2; + + carrierEnv = EnvGen.kr( + envelope: carrierEnvShape, + gate: gate + ); + + carrier = carrierEnv * SinOsc.ar( + (freq * carrierFreqRatio) + (modulator * freq) + ); + + // free synth when both carrier and modulator envelopes are done + FreeSelf.kr(Done.kr(carrierEnv) + Done.kr(modEnv) - 1); + + out = amp * carrier; +}, [ + \freq.asSpec(), + \amp.asSpec(), + \nil, + ControlSpec(0.1, 10), + ControlSpec(0.1, 10), + EnvSpec(Env.adsr( + attackTime: 0.05, + decayTime: 0.1, + sustainLevel: 0.8, + releaseTime: 0.1, + peakLevel: 1.0, + curve: [4, -4, -2] + )), + EnvSpec(Env.adsr( + attackTime: 0.05, + decayTime: 0.1, + sustainLevel: 0.8, + releaseTime: 0.1, + peakLevel: 1.0, + curve: [4, -4, -2] + )) +]);
tests/examplefiles/example.tap+37 −0 added@@ -0,0 +1,37 @@ +TAP version 13 +1..42 +1..13 A plan only supports directives so this text is wrong. +ok 1 A normal test line includes a number. +ok But a test line may also omit a number. + +A random line that does not look like a test or diagnostic should be ignored. + No matter how it is spaced out. + +Or if it is a totally blank line. + +not ok 3 This is a failing test line. + +# Diagnostics are any lines... +# ... beginning with a hash character. + +not ok 4 There are a couple of directives. # TODO is one of those directives. +not ok 5 # TODO: is invalid because the directive must be followed by a space. +ok 6 - Another directive line # toDO is not case sensitive. + +ok 7 A line that is a # SKIP +ok 8 Tests can be # skipped as long as the directive has the "skip" stem. +ok 9 The TODO directive must be followed by a space, but # skip: is valid. +1..0 # Skipped directives can show on a plan line too. + +Bail out! is a special phrase emitted when a TAP file aborted. + +not ok 10 Having TAP version 13 in the middle of a line is not a TAP version. +not ok 11 Having Bail out! in the middle of a line is not a bail out. + +ok 12 Here is an empty directive. # + +# The most basic valid test lines. +ok +not ok + +ok 15 Only the test number should look different. Not another 42, for example.
tests/examplefiles/example.tf+162 −0 added@@ -0,0 +1,162 @@ +variable "key_name" { + description = "Name of the SSH keypair to use in AWS." +} + +variable "key_path" { + description = "Path to the private portion of the SSH key specified." +} + +variable "aws_region" { + description = "AWS region to launch servers." + default = "us-west-2" + somevar = true +} + +# Ubuntu Precise 12.04 LTS (x64) +variable "aws_amis" { + default = { + eu-west-1 = "ami-b1cf19c6" + us-east-1 = "ami-de7ab6b6" + us-west-1 = "ami-3f75767a" + us-west-2 = "ami-21f78e11" + } +} + + + + + + +provider "aws" { + access_key = "${myvar}" + secret_key = "your aws secret key" + region = "us-east-1" +} +/* multiline + + comment + +*/ + + +# Single line comment +resource "aws_instance" "example" { + ami = "ami-408c7f28" + instance_type = "t1.micro" + key_name = "your-aws-key-name" +} + +# Create our Heroku application. Heroku will +# automatically assign a name. +resource "heroku_app" "web" {} + +# Create our DNSimple record to point to the +# heroku application. +resource "dnsimple_record" "web" { + domain = "${var.dnsimple_domain}" + + + # heroku_hostname is a computed attribute on the heroku + # application we can use to determine the hostname + value = "${heroku_app.web.heroku_hostname}" + + type = "CNAME" + ttl = 3600 +} + +# The Heroku domain, which will be created and added +# to the heroku application after we have assigned the domain +# in DNSimple +resource "heroku_domain" "foobar" { + app = "${heroku_app.web.name}" + hostname = "${dnsimple_record.web.hostname}" +} + + +# Specify the provider and access details +provider "aws" { + region = "${var.aws_region}" + value = ${file("path.txt")} +} + +# Our default security group to access +# the instances over SSH and HTTP +resource "aws_security_group" "default" { + name = "terraform_example" + description = "Used in the terraform" + + # SSH access from anywhere + ingress { + from_port = 22 + to_port = 22 + protocol = "tcp" + cidr_blocks = ["0.0.0.0/0"] + } + + # HTTP access from anywhere + ingress { + from_port = 80 + to_port = 80 + protocol = "tcp" + cidr_blocks = ["0.0.0.0/0"] + } +} + + +resource "aws_elb" "web" { + name = "terraform-example-elb" + + # The same availability zone as our instance + availability_zones = ["${aws_instance.web.availability_zone}"] + + listener { + instance_port = 80 + instance_protocol = "http" + lb_port = 80 + lb_protocol = "http" + } + + # The instance is registered automatically + instances = ["${aws_instance.web.id}"] +} + + +resource "aws_instance" "web" { + # The connection block tells our provisioner how to + # communicate with the resource (instance) + connection { + # The default username for our AMI + user = "ubuntu" + + # The path to your keyfile + key_file = "${var.key_path}" + } + + instance_type = "m1.small" + + # Lookup the correct AMI based on the region + # we specified + ami = "${lookup(var.aws_amis, var.aws_region)}" + + # The name of our SSH keypair you've created and downloaded + # from the AWS console. + # + # https://console.aws.amazon.com/ec2/v2/home?region=us-west-2#KeyPairs: + # + key_name = "${var.key_name}" + + # Our Security group to allow HTTP and SSH access + security_groups = ["${aws_security_group.default.name}"] + + # We run a remote provisioner on the instance after creating it. + # In this case, we just install nginx and start it. By default, + # this should be on port 80 + provisioner "remote-exec" { + inline = [ + "sudo apt-get -y update", + "sudo apt-get -y install nginx", + "sudo service nginx start" + ] + } +} +
tests/examplefiles/example.ttl+43 −0 added@@ -0,0 +1,43 @@ +@base <http://example.com> . +@prefix dcterms: <http://purl.org/dc/terms/>. @prefix xs: <http://www.w3.org/2001/XMLSchema> . +@prefix mads: <http://www.loc.gov/mads/rdf/v1#> . +@prefix skos: <http://www.w3.org/2004/02/skos/core#> . +@PREFIX dc: <http://purl.org/dc/elements/1.1/> # SPARQL-like syntax is OK +@prefix : <http://xmlns.com/foaf/0.1/> . # empty prefix is OK + +<http://example.org/#spiderman> <http://www.perceive.net/schemas/relationship/enemyOf> <http://example.org/#green-goblin> . + +<#doc1> a <#document> + dc:creator "Smith", "Jones"; + :knows <http://getopenid.com/jsmith> + dcterms:hasPart [ # A comment + dc:title "Some title", "Some other title"; + dc:creator "برشت، برتولد"@ar; + dc:date "2009"^^xs:date + ]; + dc:title "A sample title", 23.0; + dcterms:isPartOf [ + dc:title "another", "title" + ] ; + :exists true . + +<http://data.ub.uio.no/realfagstermer/006839> a mads:Topic, + skos:Concept ; + dcterms:created "2014-08-25"^^xsd:date ; + dcterms:modified "2014-11-12"^^xsd:date ; + dcterms:identifier "REAL006839" ; + skos:prefLabel "Flerbørstemarker"@nb, + "Polychaeta"@la ; + skos:altLabel "Flerbørsteormer"@nb, + "Mangebørstemark"@nb, + "Mangebørsteormer"@nb, + "Havbørsteormer"@nb, + "Havbørstemarker"@nb, + "Polycheter"@nb. + skos:inScheme <http://data.ub.uio.no/realfagstermer/> ; + skos:narrower <http://data.ub.uio.no/realfagstermer/018529>, + <http://data.ub.uio.no/realfagstermer/024538>, + <http://data.ub.uio.no/realfagstermer/026723> ; + skos:exactMatch <http://ntnu.no/ub/data/tekord#NTUB17114>, + <http://dewey.info/class/592.62/e23/>, + <http://aims.fao.org/aos/agrovoc/c_29110> .
tests/examplefiles/example.x10+9 −0 added@@ -0,0 +1,9 @@ +/** Example file for the X10 programming langauge (http://x10-lang.org). + */ +class Example { + + public static def main(Rail[String]) { + Console.OUT.println("Hello World!"); // say hello. + } + +}
tests/examplefiles/inform6_example+0 −0 renamedtests/examplefiles/main.cmake+1 −0 modified@@ -1,6 +1,7 @@ CMAKE_MINIMUM_REQUIRED(VERSION 2.6 FATAL_ERROR) SET( SOURCES back.c io.c main.c ) +SET( PATH $ENV{PATH} ) MESSAGE( ${SOURCES} ) # three arguments, prints "back.cio.cmain.c" MESSAGE( "${SOURCES}" ) # one argument, prints "back.c;io.c;main.c" MESSAGE( "" ) # one argument, prints "" an empty line
tests/examplefiles/pacman.ijs+1107 −0 added@@ -0,0 +1,1107 @@ +cocurrent 'jpacman' +coinsert 'j' + +BASELIB=: 'base library' +DATAMASK=: 0 +HWNDP=: '' +ISGUI=: 0 +ONLINE=: 0 +PKGDATA=: 0 7$a: +SECTION=: ,<'All' +SYSNAME=: 'Package Manager' +TIMEOUT=: 60 +WWWREV=: REV=: _1 + +IgnoreIOS=: 0 : 0 +api/jni +data/dbman +data/ddmysql +data/odbc +demos/isigraph +demos/wd +demos/wdplot +games/minesweeper +games/nurikabe +games/pousse +games/solitaire +general/pcall +general/sfl +graphics/d3 +graphics/fvj3 +graphics/gl2 +graphics/gnuplot +graphics/graph +graphics/graphviz +graphics/jturtle +graphics/print +graphics/tgsj +graphics/treemap +graphics/viewmat +gui/monthview +gui/util +ide/qt +math/tabula +media/animate +media/gdiplus +media/image3 +media/ming +media/paint +media/wav +) + +Ignore=: 3 : 0'' +if. IFIOS do. + <;._2 IgnoreIOS +else. + <'ide/ios' +end. +) +3 : 0'' +nc=. '--no-cache' +if. IFUNIX do. + if. UNAME-:'Darwin' do. + HTTPCMD=: 'curl -o %O --stderr %L -f -s -S %U' + elseif. do. + if. 'Android'-:UNAME do. nc=. '' + else. try. nc=. nc #~ 1 e. nc E. shell 'wget --help' catch. nc=. '' end. end. + HTTPCMD=: 'wget ',nc,' -O %O -o %L -t %t %U' + end. +else. + if. fexist exe=. jpath '~tools/ftp/wget.exe' do. exe=. '"',exe,'"' else. exe=. 'wget.exe' end. + try. nc=. nc #~ 1 e. nc E. shell exe,' --help' catch. nc=. '' end. + HTTPCMD=: exe,' ',nc,' -O %O -o %L -t %t -T %T %U' + if. fexist UNZIP=: jpath '~tools/zip/unzip.exe' do. UNZIP=: '"',UNZIP,'" -o -C ' else. UNZIP=: 'unzip.exe -o -C ' end. +end. +) +setfiles=: 3 : 0 +ADDCFG=: jpath '~addons/config/' +makedir ADDCFG +ADDCFGIJS=: ADDCFG,'config.ijs' +JRELEASE=: ({.~i.&'/') 9!:14'' +JRELEASE=: 'j802' +LIBTREE=: readtree'' +if. IFIOS do. + WWW=: '/jal/',JRELEASE,'/' +else. + WWW=: 'http://www.jsoftware.com/jal/',JRELEASE,'/' +end. +LIBVER=: jpath '~system/config/version.txt' +) +destroy=: codestroy +CFGFILES=: <;._2 (0 : 0) +addons.txt +library.txt +release.txt +revision.txt +zips.txt +) +LIBDESC=: 0 : 0 +This is the base library of scripts and labs included in the J system. + +Reinstalling or upgrading this library will overwrite files in the system subdirectory. Restart J afterwards. + +Files outside the system subdirectory, such as profile.ijs, are not changed. +) +cutjal=: ([: (* 4 > +/\) ' ' = ]) <;._1 ] +cutjsp=: ([: (* 5 > +/\) ' ' = ]) <;._1 ] +dquote=: '"'&, @ (,&'"') +fname=: #~ ([: *./\. ~:&'/') +hostcmd=: [: 2!:0 '(' , ] , ' || true)'"_ +ischar=: 2 = 3!:0 +rnd=: [ * [: <. 0.5 + %~ +sep2under=: '/' & (I.@('_' = ])}) +termLF=: , (0 < #) # LF -. {: +todel=: ; @: (DEL&, @ (,&(DEL,' ')) each) +tolist=: }. @ ; @: (LF&,@,@":each) +isjpkgout=: ((4 = {:) *. 2 = #)@$ *. 1 = L. +getintro=: ('...' ,~ -&3@[ {. ])^:(<#) +info=: smoutput +getnames=: 3 : 0 +select. L.y +case. 0 do. + if. +/ BASELIB E. y do. + y=. (<BASELIB), cutnames y rplc BASELIB;'' + else. + y=. cutnames y + end. +case. 1 do. + if. 2 = #$y do. + y=. {."1 y + else. + y=. ,y + end. +case. do. + '' return. +end. +y +) +curtailcaption=: 3 : 0 +idx=. <_1;~I. 45<#&>{:"1 y +y=. (45&getintro &.> idx{y) idx}y +) +deltree=: 3 : 0 +try. + res=. 0< ferase {."1 dirtree y + *./ res,0<ferase |.dirpath y +catch. 0 end. +) +fixjal=: 3 : 0 +if. 2 > #y do. i.0 5 return. end. +m=. _2 |. (LF,')',LF) E. y +r=. _2 }. each m <;._2 y +x=. r i.&> LF +d=. (x+1) }.each r +r=. x {.each r +r=. 3 {."1 cutjal &> ' ' ,each r +x=. d i.&> LF +c=. x {.each d +d=. (x+1) }.each d +r,.c,.d +) +fixjal2=: 3 : 0 +if. 2 > #y do. i.0 2 return. end. +cutjal &> ' ' ,each <;._2 y +) +fixjsp=: 3 : 0 +if. 2 > #y do. i.0 5 return. end. +m=. _2 |. (LF,')',LF) E. y +r=. _2 }. each m <;._2 y +x=. r i.&> LF +d=. (x+1) }.each r +r=. x {.each r +r=. ' ' ,each r +(cutjsp &> r),.d +) +fixlib=: 3 : 0 +msk=. (<LIBTREE) = 1 {"1 y +if. -. 1 e. msk do. ($0);'';0 return. end. +'ver fln siz'=. 2 4 5 { (msk i.1) { y +ver=. fixver ver +ver;fln;siz +) +fixlibs=: 3 : 0 +if. 2 > #y do. + i.0 6 return. +end. +fls=. <;._2 y +ndx=. fls i.&> ' ' +siz=. <&> 0 ". (ndx+1) }.&> fls +fls=. ndx {.each fls +zps=. <;._2 &> fls ,each '_' +pfm=. 3 {"1 zps +uname=. tolower UNAME +msk=. (uname -: ({.~ i.&'.')) &> pfm +if. 1 ~: +/msk do. msk=. 1,~ }:0*.msk end. +msk # zps,.fls,.siz +) +fixrev=: 3 : 0 +{. _1 ". :: _1: y -. CRLF +) +fixupd=: 3 : 0 +_1 ". :: _1: y -. CRLF +) +fixver=: 3 : 0 +if. ischar y do. + y=. y -. CRLF + y=. 0 ". ' ' (I. y='.') } y +end. +3 {. y +) +fixvers=: 3 : 0 +s=. $y +y=. ,y +3 {."1 [ 0 ". s $ ' ' (I. y e. './') } y +) +fmtjal=: 3 : 0 +if. 0 = #y do. '' return. end. +r=. (4 {."1 y) ,each "1 ' ',LF2 +r=. <@; "1 r +; r ,each ({:"1 y) ,each <')',LF +) +fmtjal2=: 3 : 0 +if. 0 = #y do. '' return. end. +; (2 {."1 y) ,each "1 ' ',LF +) +fmtdep=: 3 : 0 +}. ; ',' ,each a: -.~ <;._2 y +) +fmtjsp=: 3 : 0 +if. 0 = #y do. '' return. end. +r=. (4 {."1 y) ,each "1 ' ',LF +r=. <@; "1 r +; r ,each ({:"1 y) ,each <')',LF +) +fmtlib=: 3 : 0 +, 'q<.>,q<.>r<0>3.0,r<0>3.0' 8!:2 y +) +fmtver=: 3 : 0 +if. 0=#y do. '' return. end. +if. ischar y do. y return. end. +}. ; '.' ,each ": each y +) +fmtverlib=: 3 : 0 +fmtver y +) +fixzips=: 3 : 0 +if. 2 > #y do. i.0 5 return. end. +fls=. <;._2 y +ndx=. fls i.&> ' ' +siz=. 0 ". (ndx+1) }.&> fls +fls=. ndx {.each fls +zps=. <;._2 &> fls ,each '_' +zps=. zps,.fls,.<&>siz +pfm=. 3 {"1 zps +and=. (1 e. 'android'&E.) &> pfm +lnx=. (1 e. 'linux'&E.) &> pfm +mac=. (1 e. 'darwin'&E.) &> pfm +win=. mac < (1 e. 'win'&E.) &> pfm + +select. UNAME +case. 'Win' do. + zps=. win # zps +case. 'Linux' do. + zps=. lnx # zps +case. 'Android' do. + zps=. and # zps +case. 'Darwin' do. + zps=. mac # zps + zps=. zps /: 3 {"1 zps + zps=. (~: 3 {."1 zps) # zps +end. + +bit=. IF64 pick '64';'32' +pfm=. 3 {"1 zps +exc=. (1 e. bit&E.) &> pfm +zps=. zps \: exc +zps=. (~: 3 {."1 zps) # zps +fnm=. 0 {"1 zps +lnm=. 1 {"1 zps +ver=. 2 {"1 zps +pfm=. 3 {"1 zps +fls=. 4 {"1 zps +siz=. 5 {"1 zps +nms=. fnm ,each '/' ,each lnm +pfm=. (pfm i.&> '.') {.each pfm +ndx=. \: # &> pfm +sort ndx { nms,.pfm,.ver,.fls,.siz +) +fwritenew=: 4 : 0 +if. x -: fread y do. + 0 +else. + x fwrite y +end. +) +platformparent=: 3 : 0 +((< _2 {. y) e. '32';'64') # _2 }. y +) +makedir=: 1!:5 :: 0: @ < +plural=: 4 : 0 +y,(1=x)#'s' +) +sizefmt=: 3 : 0 +select. +/ y >: 1e3 1e4 1e6 1e7 1e9 +case. 0 do. + (": y), ' byte',(y~:1)#'s' +case. 1 do. + (": 0.1 rnd y%1e3),' KB' +case. 2 do. + (": 1 rnd y%1e3),' KB' +case. 3 do. + (": 0.1 rnd y%1e6),' MB' +case. 4 do. + (": 1 rnd y%1e6),' MB' +case. do. + (": 0.1 rnd y%1e9),' GB' +end. +) +shellcmd=: 3 : 0 +if. IFUNIX do. + hostcmd y +else. + spawn_jtask_ y +end. +) +subdir=: 3 : 0 +if. 0=#y do. '' return. end. +a=. 1!:0 y,'*' +if. 0=#a do. '' return. end. +a=. a #~ '-d' -:"1 [ 1 4 {"1 > 4 {"1 a +(<y) ,each ({."1 a) ,each '/' +) +testaccess=: 3 : 0 +f=. <jpath'~install/testaccess.txt' +try. + '' 1!:2 f + 1!:55 f + 1 +catch. + 0 +end. +) +toupper1=: 3 : 0 +if. 0=#y do. '' return. end. +(toupper {. y),tolower }. y +) +unzip=: 3 : 0 +'file dir'=. dquote each y +e=. 'Unexpected unzip error' +if. IFUNIX do. + notarcmd=. 0 + if. IFIOS +. UNAME-:'Android' do. + notarcmd=. _1-: 2!:0 ::_1: 'which tar' + if. (UNAME-:'Android') > '/mnt/sdcard'-:2!:5'EXTERNAL_STORAGE' do. notarcmd=. 1 end. + end. + if. notarcmd do. + require 'tar' + 'file dir'=. y + if. (i.0 0) -: tar 'x';file;dir do. e=. '' end. + else. + e=. shellcmd 'tar ',((IFIOS+:UNAME-:'Android')#(('Darwin'-:UNAME){::'--no-same-owner --no-same-permissions';'-o -p')),' -xzf ',file,' -C ',dir + end. + if. (0~:FHS) *. ('root'-:2!:5'USER') +. (<2!:5'HOME') e. 0;'/var/root';'/root';'';,'/' do. + shellcmd ::0: 'find ',dir,' -type d -exec chmod a+rx {} \+' + shellcmd ::0: 'find ',dir,' -type f -exec chmod a+r {} \+' + end. +else. + dir=. (_2&}. , '/' -.~ _2&{.) dir + e=. shellcmd UNZIP,' ',file,' -d ',dir +end. +e +) +zipext=: 3 : 0 +y, IFUNIX pick '.zip';'.tar.gz' +) +CHECKADDONSDIR=: 0 : 0 +The addons directory does not exist and cannot be created. + +It is set to: XX. + +You can either create the directory manually, or set a new addons directory in your profile script. +) +CHECKASK=: 0 : 0 +Read catalog from the server using Internet connection now? + +Otherwise the local catalog is used offline. +) +CHECKONLINE=: 0 : 0 +An active Internet connection is needed to install packages. + +Continue only if you have an active Internet connection. + +OK to continue? +) +CHECKREADSVR=: 0 : 0 +An active Internet connection is needed to read the server repository catalog. + +Continue only if you have an active Internet connection. + +OK to continue? +) +CHECKSTARTUP=: 0 : 0 +Setup repository using Internet connection now? + +Select No if not connected, to complete setup later. After Setup is done, repository can be used offline with more options in Tools menu and Preferences dialog. +) +checkaccess=: 3 : 0 +if. testaccess'' do. 1 return. end. +msg=. 'Unable to run Package Manager, as you do not have access to the installation folder.' +if. IFWIN do. + msg=. msg,LF2,'To run as Administrator, right-click the J icon, select Run as... and ' + msg=. msg,'then select Adminstrator.' +end. +info msg +0 +) +checkaddonsdir=: 3 : 0 +d=. jpath '~addons' +if. # 1!:0 d do. 1 return. end. +if. 1!:5 :: 0: <d do. + log 'Created addons directory: ',d + 1 return. +end. +info CHECKADDONSDIR rplc 'XX';d +0 +) +getonline=: 3 : 0 +ONLINE=: 2=3 2 wdquery y +) +getserver=: 3 : 0 +'rc p'=. httpgetr (WWW,'revision.txt');2 +if. rc do. 0 return. end. +write_lastupdate'' +WWWREV=: fixrev p +if. WWWREV = REV do. 1 return. end. +refreshweb'' +) +checkonline=: 3 : 0 +select. ReadCatalog_j_ +case. 0 do. + if. REV >: 0 do. + ONLINE=: 0 + log 'Using local copy of catalog. See Preferences to change the setting.' + 1 return. + end. + if. 0 = getonline 'Read Catalog from Server';CHECKREADSVR do. 0 return. end. +case. 1 do. + ONLINE=: 1 +case. 2 do. + if. REV >: 0 do. + if. 0 = getonline 'Read Catalog from Server';CHECKASK do. + log 'Using local copy of catalog. See Preferences to change the setting.' + 1 return. + end. + else. + if. 0 = getonline 'Setup Repository';CHECKSTARTUP do. 0 return. end. + end. +end. +log 'Updating server catalog...' +if. 0 = getserver'' do. + ONLINE=: 0 + log 'Working offline using local copy of catalog.' +else. + log 'Done.' +end. +1 +) +checkstatus=: 3 : 0 +if. 0 e. #LIBS do. '' return. end. +msk=. masklib PKGDATA +ups=. pkgups'' +libupm=. 1 e. msk *. ups +msk=. -. msk +addnim=. +/msk *. pkgnew'' +addupm=. +/msk *. pkgups'' +tot=. +/addnim,addupm,libupm +if. 0 = tot do. + 'All available packages are installed and up to date.' return. +end. +select. 0 < addnim,addupm +case. 0 0 do. + msg=. 'Addons are up to date.' +case. 0 1 do. + msg=. 'All addons are installed, ',(":addupm), ' can be upgraded.' +case. 1 0 do. + if. addnim = <:#PKGDATA do. + msg=. 'No addons are installed.' + else. + j=. ' addon',('s'#~1<addnim),' are not yet installed.' + msg=. 'Installed addons are up to date, ',(":addnim),j + end. +case. 1 1 do. + j=. (":addupm),' addon',('s'#~1<addupm),' can be upgraded, ' + msg=. j,(":addnim), ' addon',('s'#~1<addnim),' are not yet installed.' +end. +if. 0 = libupm do. + msg,LF,'The base library is up to date.' +else. + msg,LF,'There is a newer version of the base library.' +end. +) +write_lastupdate=: 3 : 0 +txt=. ": 6!:0 '' +txt fwrites ADDCFG,'lastupdate.txt' +) +checklastupdate=: 3 : 0 +if. _1 -: LASTUPD do. + res=. 'has never been updated.' +else. + res=. 'was last updated: ',timestamp LASTUPD +end. +'Local JAL information ',res +) +getdepend=: 3 : 0 +if. 0 = #y do. y return. end. +dep=. getdepend_console 1{"1 y +PKGDATA #~ (1{"1 PKGDATA) e. dep +) +getdepend_console=: 3 : 0 +if. 0 = #y do. y return. end. +old=. '' +ids=. 1{"1 PKGDATA +dep=. 6{"1 PKGDATA +res=. ~. <;._1 ; ',' ,each (ids e. y) # dep +whilst. -. res-:old do. + old=. res + res=. ~. res, <;._1 ; ',' ,each (ids e. res) # dep +end. +~. y, res -. a:, {."1 ADDINS +) +httpget=: 3 : 0 +'f t'=. 2 {. (boxxopen y),a: +n=. f #~ -. +./\. f e. '=/' +p=. jpath '~temp/',n +q=. jpath '~temp/httpget.log' +t=. ":{.t,3 +ferase p;q +fail=. 0 +cmd=. HTTPCMD rplc '%O';(dquote p);'%L';(dquote q);'%t';t;'%T';(":TIMEOUT);'%U';f +if. IFIOS +. UNAME-:'Android' do. + require 'socket' + 1!:55 ::0: <p + rc=. 0 [ e=. pp=. '' + whilst. 0 do. + 'rc sk'=. sdsocket_jsocket_'' + if. 0~:rc do. break. end. + rc=. sdconnect_jsocket_ sk;PF_INET_jsocket_;'23.21.67.48';80 + if. 0~:rc do. break. end. + 'rc sent'=. ('GET ',f,' HTTP/1.0',LF2) sdsend_jsocket_ sk;0 + if. 0~:rc do. break. end. + while. ((0=rc)*.(*#m)) [[ 'rc m'=. sdrecv_jsocket_ sk,4096 do. + pp=. pp,m + end. + end. + sdclose_jsocket_ sk + if. 0~:rc do. fail=. 1 + elseif. 1 -.@e. '200 OK' E. (20{.pp) do. fail=. 1 [ e=. ({.~ i.&LF) pp + elseif. #p1=. I. (CRLF,CRLF) E. 500{.pp do. p2=. 4 + elseif. #p1=. I. LF2 E. 500{.pp do. p2=. 2 + elseif. do. fail=. 1 + end. + if. 0=fail do. + ((p2+{.p1)}.pp) 1!:2 <p + else. + if. 0~:rc do. e=. sderror_jsocket_ rc end. + end. +elseif. do. + try. + e=. shellcmd cmd + catch. fail=. 1 end. +end. +if. fail +. 0 >: fsize p do. + if. _1-:msg=. freads q do. + if. 0=#msg=. e do. msg=. 'Unexpected error' end. end. + log 'Connection failed: ',msg + info 'Connection failed:',LF2,msg + r=. 1;msg + ferase p;q +else. + r=. 0;p + ferase q +end. +r +) +httpgetr=: 3 : 0 +res=. httpget y +if. 0 = 0 pick res do. + f=. 1 pick res + txt=. freads f + ferase f + 0;txt +end. +) +install=: 3 : 0 +dat=. getdepend y +'num siz'=. pmview_applycounts dat +many=. 1 < num +msg=. 'Installing ',(":num),' package',many#'s' +msg=. msg,' of ',(many#'total '),'size ',sizefmt siz +log msg +installdo 1 {"1 dat +log 'Done.' +readlocal'' +pacman_init 0 +) +install_console=: 3 : 0 + if. -. init_console 'server' do. '' return. end. + pkgs=. getnames y + if. pkgs -: ,<'all' do. pkgs=. 1 {"1 PKGDATA end. + pkgs=. pkgs (e. # [) ~. (<'base library'), ((pkgnew +. pkgups) # 1&{"1@]) PKGDATA + pkgs=. pkgs -. Ignore + pkgs=. getdepend_console pkgs + if. 0 = num=. #pkgs do. '' return. end. + many=. 1 < num + msg=. 'Installing ',(":num),' package',many#'s' + log msg + installdo pkgs + log 'Done.' + readlocal'' + pacman_init '' + checkstatus'' +) +upgrade_console=: 3 : 0 + if. -. init_console 'read' do. '' return. end. + pkgs=. getnames y + if. (0=#pkgs) +. pkgs -: ,<'all' do. pkgs=. 1{"1 PKGDATA end. + pkgs=. pkgs (e. # [) (pkgups # 1&{"1@])PKGDATA + install_console pkgs +) +installdo=: 3 : 0 +msk=. -. y e. <BASELIB +if. 0 e. msk do. + install_library'' +end. +install_addon each msk # y +) +install_addon=: 3 : 0 +ndx=. ({."1 ZIPS) i. <y +if. ndx = #ZIPS do. EMPTY return. end. +log 'Downloading ',y,'...' +f=. 3 pick ndx { ZIPS +'rc p'=. httpget WWW,'addons/',f +if. rc do. return. end. +log 'Installing ',y,'...' +msg=. unzip p;jpath'~addons' +ferase p +if. 0>:fsize jpath'~addons/',y,'/manifest.ijs' do. + log 'Extraction failed: ',msg + info 'Extraction failed:',LF2,msg + return. +end. +install_addins y +install_config y +) +install_addins=: 3 :0 +fl=. ADDCFG,'addins.txt' +ins=. fixjal2 freads fl +ins=. ins #~ (<y) ~: {."1 ins +ndx=. ({."1 ADDONS) i. <y +ins=. sort ins, 2 {. ndx { ADDONS +(fmtjal2 ins) fwrites fl +) +install_config=: 3 : 0 +ADDLABS=: '' +0!:0 :: ] < ADDCFGIJS +install_labs y +write_config'' +) +install_labs=: 3 : 0 +labs=. dirtree jpath '~addons/',y,'/*.ijt' +if. 0=#labs do. return. end. +pfx=. jpath '~addons/' +labs=. (#pfx) }.each {."1 labs +LABCATEGORY=: '' +0!:0 ::] <jpath '~addons/',y,'/manifest.ijs' +cat=. LABCATEGORY +if. 0 = #cat do. + cat=. toupper1 (y i. '/') {. y +end. +new=. labs ,each <' ',cat +txt=. sort ~. new,<;._2 ADDLABS +ndx=. 4 + (1 i.~ '.ijt'&E.) &> txt +msk=. fexist &> (<pfx) ,each ndx {.each txt +txt=. msk # txt +ADDLABS=: ; txt ,each LF +) +install_library=: 3 : 0 +log 'Downloading base library...' +f=. 1 pick LIB +'rc p'=. httpget WWW,'library/',f +if. rc do. return. end. +log 'Installing base library...' +unzip p;jpath'~system' +ferase p +readlin'' +) +write_config=: 3 : 0 +txt=. 'NB. Addon configuration',LF2 +txt=. txt,'ADDLABS=: 0 : 0',LF,ADDLABS,')',LF +txt fwrites ADDCFGIJS +) +show_console=: 4 : 0 + if. -. init_console 'read' do. '' return. end. + select. x + case. 'search' do. + pkgs=. getnames y + res=. (pkgsearch pkgs) # 1 2 3 4 {"1 PKGDATA + res=. curtailcaption res + case. 'show' do. + pkgs=. getnames y + if. pkgs -: ,<'all' do. pkgs=. 1 {"1 PKGDATA end. + res=. (msk=. pkgshow pkgs) # 5 {"1 PKGDATA + if. #res do. + res=. ,((<'== '), &.> msk # 1 {"1 PKGDATA) ,. res + res=. (2#LF) joinstring (70&foldtext)&.> res + end. + case. 'showinstalled' do. + dat=. (isjpkgout y) {:: (1 2 3 4 {"1 PKGDATA);<y + res=. (-.@pkgnew # ])dat + res=. curtailcaption res + case. 'shownotinstalled' do. + dat=. (isjpkgout y) {:: (1 2 3 4 {"1 PKGDATA);<y + res=. (pkgnew # 0 2 3&{"1@])dat + res=. curtailcaption res + case. 'showupgrade' do. + dat=. (isjpkgout y) {:: (1 2 3 4 {"1 PKGDATA);<y + res=. (pkgups # ])dat + res=. curtailcaption res + case. 'status' do. + res=. checklastupdate'' + res=. res,LF,checkstatus'' + end. + res +) +showfiles_console=: 4 : 0 + if. -. init_console 'read' do. '' return. end. + pkgs=. getnames y + pkgs=. pkgs (e. # [) (-.@pkgnew # 1&{"1@]) PKGDATA + pkgs=. pkgs -. <BASELIB + if. 0=#pkgs do. '' return. end. + fn=. (<'~addons/') ,&.> (pkgs) ,&.> <'/',x,(x-:'history'){::'.ijs';'.txt' + res=. res #~ msk=. (<_1) ~: res=. fread@jpath &.> fn + if. #res do. + res=. ,((<'== '), &.> msk#pkgs) ,. res + res=. (2#LF) joinstring res + end. +) +remove_console=: 3 : 0 + if. -. init_console 'edit' do. '' return. end. + pkgs=. getnames y + if. pkgs -: ,<'all' do. pkgs=. 1 {"1 PKGDATA end. + pkgs=. pkgs (e. # [) (-.@pkgnew # 1&{"1@]) PKGDATA + pkgs=. pkgs -. <BASELIB + if. 0 = num=. #pkgs do. '' return. end. + many=. 1 < num + msg=. 'Removing ',(":num),' package',many#'s' + log msg + remove_addon each pkgs + log 'Done.' + readlocal'' + pacman_init '' + checkstatus'' +) + +remove_addon=: 3 : 0 + log 'Removing ',y,'...' + treepath=. jpath '~addons/',y + if. ((0 < #@dirtree) *. -.@deltree) treepath do. + nf=. #dirtree treepath + nd=. <: # dirpath treepath + nd=. nd + (tolower treepath) e. dirpath jpath '~addons/', '/' taketo y + msg=. (":nd),' directories and ',(":nf),' files not removed.' + log 'Remove failed: ',msg + info 'Remove failed:',LF2,msg + return. + end. + remove_addins y + remove_config y +) +remove_addins=: 3 :0 + fl=. ADDCFG,'addins.txt' + ins=. fixjal2 freads fl + ins=. ins #~ (<y) ~: {."1 ins + (fmtjal2 ins) fwrites fl +) +remove_config=: 3 : 0 + ADDLABS=: '' + 0!:0 :: ] < ADDCFGIJS + remove_labs y + write_config'' +) +remove_labs=: 3 : 0 + txt=. <;._2 ADDLABS + txt=. txt #~ (<jpathsep y) ~: (#y)&{. each txt + ADDLABS=: ; txt ,each LF +) +LOG=: 1 +log=: 3 : 0 +if. LOG do. smoutput y end. +) +logstatus=: 3 : 0 +if. ONLINE do. + log checkstatus'' +end. +) +readlin=: 3 : 0 +LIN=: 6 1 1 >. fixver freads LIBVER +) +readlocal=: 3 : 0 +readlin'' +ADDONS=: fixjal freads ADDCFG,'addons.txt' +ADDINS=: fixjal2 freads ADDCFG,'addins.txt' +REV=: fixrev freads ADDCFG,'revision.txt' +LASTUPD=: fixupd freads ADDCFG,'lastupdate.txt' +LIBS=: fixlibs freads ADDCFG,'library.txt' +LIB=: fixlib LIBS +ZIPS=: fixzips freads ADDCFG,'zips.txt' +EMPTY +) +readtree=: 3 : 0 +f=. ADDCFG,'tree.txt' +tree=. LF -.~ freads f +if. -. (<tree) e. 'current';'stable' do. + tree=. 'current' + writetree tree +end. +tree +) +writetree=: 3 : 0 +y fwritenew ADDCFG,'tree.txt' +) +refreshweb=: 3 : 0 +if. 0 = refreshjal'' do. 0 return. end. +readlocal'' +1 +) +refreshaddins=: 3 : 0 +ADDLABS=: '' +f=. ADDCFG,'addins.txt' +p=. jpath '~addons/' +sd=. ;subdir each subdir p +if. 0=#sd do. + '' fwrite f + write_config'' return. +end. +r=. s=. '' +for_d. sd do. + mft=. freads (>d),'manifest.ijs' + if. mft -: _1 do. continue. end. + VERSION=: '' + 0!:100 mft + ver=. fmtver fixver VERSION + n=. }: (#p) }. >d + n=. '/' (I.n='\') } n + r=. r,n,' ',ver,LF + s=. s,d +end. +r fwritenew f +s=. (#p) }.each }: each s +install_labs each s +write_config'' +) +refreshjal=: 3 : 0 +'rc p'=. httpget WWW,zipext 'jal' +if. rc do. 0 return. end. +unzip p;ADDCFG +ferase p +if. *./ CFGFILES e. {."1 [ 1!:0 ADDCFG,'*' do. 1 return. end. +msg=. 'Could not install the local repository catalog.' +log msg +info msg +0 +) +updatejal=: 3 : 0 + log 'Updating server catalog...' + if. -. init_console 'server' do. '' return. end. + refreshaddins'' + readlocal'' + pacman_init'' + res=. checklastupdate'' + res,LF,checkstatus'' +) +RELIBMSG=: 0 : 0 +You are now using the XX base library, and can switch to the YY base library. + +This will download the YY version of the base library and overwrite existing files. Addons are not affected. + +OK to switch to the YY library? +) +prelib=: 3 : 0 +old=. LIBTREE +new=. (('stable';'current') i. <old) pick 'current';'beta' +msg=. RELIBMSG rplc ('XX';'YY'),.old;new +if. 0 = query SYSNAME;msg do. + info 'Not done.' return. +end. +switchlibrary 1 pick new +) +switchlibrary=: 3 : 0 +'' fwrite LIBVER +writetree LIBTREE=: y +refreshjal'' +readlocal'' +pmview_setpn'' +) +masklib=: 3 : 0 +(1 {"1 y) = <BASELIB +) +pkglater=: 3 : 0 +dat=. (s=.isjpkgout y){:: PKGDATA;<y +if. 0=#dat do. $0 return. end. +loc=. fixvers > (2-s) {"1 dat +srv=. fixvers > (3-s) {"1 dat +{."1 /:"2 srv ,:"1 loc +) +pkgnew=: 3 : 0 +dat=. (s=.isjpkgout y){:: PKGDATA;<y +if. 0=#dat do. $0 return. end. +0 = # &> (2-s) {"1 dat +) +pkgups=: pkgnew < pkglater +pkgsearch=: 3 : 0 + +./"1 +./ y E."1&>"(0 _) 1{"1 PKGDATA +) +pkgshow=: 3 : 0 + y e.~ 1{"1 PKGDATA +) +setshowall=: 3 : 0 +PKGDATA=: (<y) (<(I.DATAMASK);0) } PKGDATA +) +setshownew=: 3 : 0 +ndx=. I. DATAMASK *. pkgnew'' +PKGDATA=: (<y) (<ndx;0) } PKGDATA +) +setshowups=: 3 : 0 +ndx=. I. DATAMASK *. pkgups'' +PKGDATA=: (<y) (<ndx;0) } PKGDATA +) +splitlib=: 3 : 0 +if. 0=#y do. + 2 $ <y return. +end. +msk=. masklib y +(msk#y) ; <(-.msk)#y +) +pacman_init=: 3 : 0 +dat=. ADDONS #~ ({."1 ADDONS) e. {."1 ZIPS +if. 0=#dat do. + dat=. i.0 6 +else. + ndx=. ({."1 ADDINS) i. {."1 dat + ins=. ndx { (1 {"1 ADDINS),<'' + dat=. dat,.<'' + dat=. 0 5 1 3 4 2 {"1 dat + dat=. ins 1 }"0 1 dat +end. +lib=. 'base library';(fmtver LIN);(fmtver 0 pick LIB);'base library scripts';LIBDESC;'' +dat=. dat,lib +dat=. (<0),.dat +PKGDATA=: sort dat +nms=. 1 {"1 PKGDATA +nms=. ~. (nms i.&> '/') {.each nms +SECTION=: 'All';nms +DATAMASK=: (#PKGDATA) $ 1 +EMPTY +) +init_console=: 3 : 0 + if. 0=#y do. y=. 'read' end. + select. y + fcase. 'edit';'server' do. + if. -. checkaccess'' do. 0 return. end. + case. 'read' do. + if. -. checkaddonsdir'' do. 0 return. end. + setfiles'' + readlocal'' + pacman_init '' + res=. 1 + case. do. res=. 0 + end. + if. y -: 'server' do. res=. getserver'' end. + res +) +jpkg=: 4 : 0 + select. x + case. 'history';'manifest' do. + x showfiles_console y + case. 'install' do. + install_console y + case. 'reinstall' do. + remove_console y + install_console y + case. 'remove' do. + remove_console y + case. ;:'show search showinstalled shownotinstalled showupgrade status' do. + x show_console y + case. 'update' do. + updatejal '' + case. 'upgrade' do. + upgrade_console y + case. do. + msg=. 'Valid options are:',LF + msg=. msg,' history, install, manifest, remove, reinstall, show, search,',LF + msg=. msg,' showinstalled, shownotinstalled, showupgrade, status,',LF + msg,' update, upgrade' + end. +) +do_install=: 3 : 0 +if. -. checkaccess_jpacman_ '' do. return. end. +'update' jpkg '' +select. y +case. 'qtide';'angle' do. + 'install' jpkg 'base library ide/qt' + getqtbin (y-:'angle'){::0;'angle' + msg=. (+/ 2 1 * IFWIN,'Darwin'-:UNAME) pick 'jqt.sh';'the jqt icon';'jqt.cmd' + smoutput 'exit and restart J using ',msg +case. 'all' do. + 'install' jpkg 'all' + getqtbin 0 +end. +) +do_getqtbin=: 3 : 0 +smoutput 'Installing JQt binaries...' +if. 'Linux'-:UNAME do. + if. IFRASPI do. + z=. 'jqt-raspi-32.tar.gz' + else. + z=. 'jqt-',((y-:'slim') pick 'linux';'slim'),'-',(IF64 pick 'x86';'x64'),'.tar.gz' + end. + z1=. 'libjqt.so' +elseif. IFWIN do. + z=. 'jqt-win',((y-:'slim')#'slim'),'-',(IF64 pick 'x86';'x64'),'.zip' + z1=. 'jqt.dll' +elseif. do. + z=. 'jqt-mac',((y-:'slim')#'slim'),'-',(IF64 pick 'x86';'x64'),'.zip' + z1=. 'libjqt.dylib' +end. +'rc p'=. httpget_jpacman_ 'http://www.jsoftware.com/download/j802/qtide/',z +if. rc do. + smoutput 'unable to download: ',z return. +end. +d=. jpath '~bin' +if. IFWIN do. + unzip_jpacman_ p;d +else. + if. 'Linux'-:UNAME do. + if. (0~:FHS) do. + if. IFRASPI do. + d1=. '/usr/lib/arm-linux-gnueabihf/.' + elseif. IF64 do. + d1=. '/usr/lib/x86_64-linux-gnu/.' + elseif. do. + d1=. '/usr/lib/i386-linux-gnu/.' + end. + hostcmd_jpacman_ 'cd /usr/bin && tar --no-same-owner --no-same-permissions -xzf ',(dquote p), ' && chmod 755 jqt && chmod 644 libjqt.so && mv libjqt.so ',d1 + else. + hostcmd_jpacman_ 'cd ',(dquote d),' && tar xzf ',(dquote p) + end. + else. + hostcmd_jpacman_ 'unzip -o ',(dquote p),' -d ',dquote d + end. +end. +ferase p +if. #1!:0 ((0~:FHS)*.'Linux'-:UNAME){::(jpath '~bin/',z1);'/usr/bin/jqt' do. + m=. 'Finished install of JQt binaries.' +else. + m=. 'Unable to install JQt binaries.',LF + m=. m,'check that you have write permission for: ',LF,((0~:FHS)*.'Linux'-:UNAME){::(jpath '~bin');'/usr/bin' +end. +smoutput m +if. 'Linux'-:UNAME do. return. end. + +tgt=. jpath IFWIN{::'~install/Qt';'~bin/Qt5Core.dll' +y=. (*#y){::0;y +smoutput 'Installing Qt library...' +if. IFWIN do. + z=. 'qt53-',((y-:'angle') pick 'win';'angle'),'-',((y-:'slim')#'slim-'),(IF64 pick 'x86';'x64'),'.zip' +else. + z=. 'qt53-mac-',((y-:'slim')#'slim-'),(IF64 pick 'x86';'x64'),'.zip' +end. +'rc p'=. httpget_jpacman_ 'http://www.jsoftware.com/download/j802/qtlib/',z +if. rc do. + smoutput 'unable to download: ',z return. +end. +d=. jpath IFWIN{::'~install';'~bin' +if. IFWIN do. + unzip_jpacman_ p;d +else. + hostcmd_jpacman_ 'unzip -o ',(dquote p),' -d ',dquote d +end. +ferase p +if. #1!:0 tgt do. + m=. 'Finished install of Qt binaries.' +else. + m=. 'Unable to install Qt binaries.',LF + m=. m,'check that you have write permission for: ',LF,IFWIN{::tgt;jpath'~bin' +end. +smoutput m + +) +jpkg_z_=: 3 : 0 + 'help' jpkg y + : + a=. conew 'jpacman' + res=. x jpkg__a y + destroy__a'' + res +) +jpkgv_z_=: (<@:>"1@|:^:(0 ~: #))@jpkg \ No newline at end of file
tests/examplefiles/roboconf.graph+40 −0 added@@ -0,0 +1,40 @@ +################## +# A sample graph +################## + +import some-definition.graph; +import another-definition.graph; + +VM { + installer : target; + children: deployable; +} + +facet deployable { + # nothing +} + +# Sample deployables +mysql { + insTaller: puppet; + facets: deployable; + exports: ip, port = 3306; +} + +tomcat { + installer: bash; + facets: deployable; + exports: ip; + children: web-application; +} + +facet web-application { + exports: full-path = undefined; +} + +my-war-1 { + facets: web-application; + installer: file; + exports: full-path = apps/my-war-1; # the relative path + imports: mysql.*; +}
tests/examplefiles/roboconf.instances+24 −0 added@@ -0,0 +1,24 @@ + +# Deal with imports +import others.instances; + +instance of VM { + name: VM-mysql; + instance of mysql { + name: MySQL; + } +} + +instance of VM { + name: VM ; + count: 5; + + INSTANCE of tomcat { + name: Tomcat; + + instance of my-war-1 { + name: my-war-1; + full-path: apps/my-war; + } + } +}
tests/examplefiles/sparql.rq+30 −5 modified@@ -1,23 +1,48 @@ # This is a test SPARQL query +BASE <http://pygments.org/examples/sparql.rq> + PREFIX foaf: <http://xmlns.com/foaf/0.1/> PREFIX ex: <http://example.org/> PREFIX xsd: <http://www.w3.org/2001/XMLSchema#> PREFIX dcterms: <http://purl.org/dc/terms/> SELECT ?person (COUNT(?nick) AS ?nickCount) { + <#jonny> foaf:knows ?person . + ?person a foaf:Person . + ?person foaf:firstName "Freddy" . + ?person foaf:lastName "Smith" . + # predicate-object list ?person foaf:nick ?nick ; - foaf:lastName "Smith" ; - foaf:age "21"^^xsd:int ; + foaf:age "21"^^xsd:int ; # typed literal ex:title 'Mr' ; # single-quoted string + ex:width 2 ; # integer ex:height 1.80 ; # float - ex:distanceToSun +1.4e8 ; # float with exponent + ex:distanceToSun 1.4e8 ; # float with exponent ex:ownsACat true ; - dcterms:description "Someone with a cat called \"cat\"."@en . + ex:catName "Kitty", "Kitty_" ; # object list + # some other float values + ex:float1 .125 ; + ex:float2 +2.5e10 ; + ex:float3 2.5e+10 ; + ex:float4 -1.e-10 ; + ex:float5 .0e1 ; + ex:float6 5e11 ; + ex:float7 1. ; + ex:À "" ; + ex:豈 "" ; + ex:escape "\n\u00c0\U00010000"; + ex:catAge ?catage ; + dcterms:description "Someone with a cat called \"cat\"."@en . # language tag + ?person foaf:knows _:b0 . + _:b0 foaf:knows [ _:b1 a foaf:Person; foaf:name "Jonny" . ] . OPTIONAL { ?person foaf:isPrimaryTopicOf ?page } OPTIONAL { ?person foaf:name ?name { ?person foaf:depiction ?img } UNION { ?person foaf:firstName ?firstN } } FILTER ( bound(?page) || bound(?img) || bound(?firstN) ) -} GROUP BY ?person ORDER BY ?img + FILTER ( ?catage < 101 && ?catage > 9 && ?catage >= 10 && ?catage <= 100 && ?catage != 20 ) +} +GROUP BY ?person +ORDER BY ?img ASC(?firstN) DESC(?page)
tests/examplefiles/test-3.0.xq+185 −0 added@@ -0,0 +1,185 @@ +xquery version "3.0"; + +declare namespace other = "http://other"; + +declare variable $local:straight-var1 := 'one'; + +declare %private variable $local:private-var := 'secret'; +declare %public variable $local:public-var := 'not-secret'; +declare %other:annotation('param1', "param2") variable $local:some-var := 'anything'; + +declare variable $local:straight-var2 := 'two'; + + +(: Simple Map Operator example :) +declare function local:word-count($elms as element()*) as xs:integer { + sum($elms ! count(tokenize(., '\s+'))) +}; + +declare function local:add($a, $b) { + $a + $b +}; + +declare function local:dispatch($node as node()) as item()* { + typeswitch($node) + case text() return $node + case comment() return $node + case element(bill) return local:bill($node) + case element(btitle) return local:btitle($node) + case element(section-id) return local:section-id($node) + case element(bill-text) return local:bill-text($node) + case element(strike) return local:strike($node) + default return local:passthru($node) +}; + +(: `switch` expression example :) +declare function local:noise($animal) { + let $duck := "Duck", + $quack := "Quack" + return + switch ($animal) + case "Cow" return "Moo" + case 'Cat' return 'Meow' + case $duck return $quack + default return "What's that odd noise?" +}; + +(: `group by` expression with binding example :) +declare function local:a-to-z() { + let $data as element()* := ( + <item>Apples</item>, + <item>Bananas</item>, + <item>Apricots</item>, + <item>Pears</item>, + <item>Brambles</item> + ) return + <GroupedItems>{ + for $item in $data + group by $key := upper-case(substring($item, 1, 1)) + order by $key + return + <Group key="{$key}">{$item}</Group> + }</GroupedItems> +}; + +(: `group by` expression example :) +declare function local:plays-by-character() { + let $plays := ( + document { + <play> + <title>Hamlet</title> + <characters> + <character>Hamlet</character> + <character>Claudius</character> + <character>Polonius</character> + <character>Rosencrantz</character> + <character>Guildenstern</character> + <character>Francisco</character> + <character>Reynaldo</character> + </characters> + </play> + }, + document { + <play> + <title>Rosenkrantz and Guildenstern are Dead</title> + <characters> + <character>Alfred</character> + <character>Rosencrantz</character> + <character>Guildenstern</character> + <character>Hamlet</character> + <character>Claudius</character> + </characters> + </play> + } + ) return + + for $play in $plays/play + let $title := $play/title + for $character in $play/characters/character + group by $character + return + <character name="{$character}"> + { + $title ! <play>{ . }</play> + } + </character> +}; + +declare + %other:a + %private + %other:b('1') + %other:c("1", "2", "3", "4") +function local:very-annotated() { + let $thing := "thing" + return + $thing +}; + +declare %public function local:slightly-annotated() { + let $nothing := () + return + $nothing +}; + +declare function local:ordered() { + for $hit in doc("/db/doc-with-indexes.xml")//tei:p[other:query(., $search-expression)] + let $score as xs:float := other:score($hit) + order by $score descending + return ( + <p>Score: {$score}:</p>, + other:summarize($hit, <config width="40"/>) + ) +}; + +declare function local:concat-expr($postfix) { + + let $concatenated := other:uri() || "/" || $postfix + return + $concatenated +}; + +declare function local:human-units($bytes) { + let $unit := if($bytes > math:pow(1024, 3)) then + (math:pow(1024, 3), "GB") + else if($bytes > math:pow(1024, 2)) then + (math:pow(1024, 2), "MB") + else + (1024, "KB") + return + format-number($bytes div $unit[1], ".00") || " " || $unit[2] +}; + +declare function local:merge-simple($a as xs:string+, $b as xs:string+) as xs:string+ { + ($a, $b) +}; + +(: higher order function example 1 :) +declare function local:apply($func, $value) { + $func($value) +}; + +(: higher order function example 2 :) +declare function local:apply-all($func, $list) { + $list ! $func(.) +}; + +(: higher order function example 3 :) +declare function local:apply-all-long($func as function(xs:string) as xs:string, $list) { + $list ! $func(.) +}; + +(: higher order function example 4 :) +declare function local:merge($func as function(xs:string+, xs:string+) as xs:string+, $a as xs:string+, $b as xs:string+) as xs:string+ { + $func($a, $b) +}; + +let $to-upper := upper-case#1 +let $to-upper-long as function(xs:string) as xs:string := upper-case#1 +return + <case> + { + local:apply-all($to-upper, ("Hello", "world!")) ! <upper>{.}</upper>, + local:apply-all-long(lower-case#1, ("Hello", "world!")) ! <lower>{.}</lower> + } + </case>
tests/examplefiles/test.bpl+140 −0 added@@ -0,0 +1,140 @@ +/* + * Test Boogie rendering +*/ + +const N: int; +axiom 0 <= N; + +procedure foo() { + break; +} +// array to sort as global array, because partition & quicksort have to +var a: [int] int; +var original: [int] int; +var perm: [int] int; + +// Is array a of length N sorted? +function is_sorted(a: [int] int, l: int, r: int): bool +{ + (forall j, k: int :: l <= j && j < k && k <= r ==> a[j] <= a[k]) +} + +// is range a[l:r] unchanged? +function is_unchanged(a: [int] int, b: [int] int, l: int, r: int): bool { + (forall i: int :: l <= i && i <= r ==> a[i] == b[i]) +} + +function is_permutation(a: [int] int, original: [int] int, perm: [int] int, N: int): bool +{ + (forall k: int :: 0 <= k && k < N ==> 0 <= perm[k] && perm[k] < N) && + (forall k, j: int :: 0 <= k && k < j && j < N ==> perm[k] != perm[j]) && + (forall k: int :: 0 <= k && k < N ==> a[k] == original[perm[k]]) +} + +function count(a: [int] int, x: int, N: int) returns (int) +{ if N == 0 then 0 else if a[N-1] == x then count(a, x, N - 1) + 1 else count(a, x, N-1) } + + +/* +function count(a: [int] int, x: int, N: int) returns (int) +{ if N == 0 then 0 else if a[N-1] == x then count(a, x, N - 1) + 1 else count(a, x, N-1) } + +function is_permutation(a: [int] int, b: [int] int, l: int, r: int): bool { + (forall i: int :: l <= i && i <= r ==> count(a, a[i], r+1) == count(b, a[i], r+1)) +} +*/ + +procedure partition(l: int, r: int, N: int) returns (p: int) + modifies a, perm; + requires N > 0; + requires l >= 0 && l < r && r < N; + requires ((r+1) < N) ==> (forall k: int :: (k >= l && k <= r) ==> a[k] <= a[r+1]); + requires ((l-1) >= 0) ==> (forall k: int :: (k >= l && k <= r) ==> a[k] > a[l-1]); + + /* a is a permutation of the original array original */ + requires is_permutation(a, original, perm, N); + + ensures (forall k: int :: (k >= l && k <= p ) ==> a[k] <= a[p]); + ensures (forall k: int :: (k > p && k <= r ) ==> a[k] > a[p]); + ensures p >= l && p <= r; + ensures is_unchanged(a, old(a), 0, l-1); + ensures is_unchanged(a, old(a), r+1, N); + ensures ((r+1) < N) ==> (forall k: int :: (k >= l && k <= r) ==> a[k] <= a[r+1]); + ensures ((l-1) >= 0) ==> (forall k: int :: (k >= l && k <= r) ==> a[k] > a[l-1]); + + /* a is a permutation of the original array original */ + ensures is_permutation(a, original, perm, N); +{ + var i: int; + var sv: int; + var pivot: int; + var tmp: int; + + i := l; + sv := l; + pivot := a[r]; + + while (i < r) + invariant i <= r && i >= l; + invariant sv <= i && sv >= l; + invariant pivot == a[r]; + invariant (forall k: int :: (k >= l && k < sv) ==> a[k] <= old(a[r])); + invariant (forall k: int :: (k >= sv && k < i) ==> a[k] > old(a[r])); + + /* a is a permutation of the original array original */ + invariant is_permutation(a, original, perm, N); + + invariant is_unchanged(a, old(a), 0, l-1); + invariant is_unchanged(a, old(a), r+1, N); + invariant ((r+1) < N) ==> (forall k: int :: (k >= l && k <= r) ==> a[k] <= a[r+1]); + invariant ((l-1) >= 0) ==> (forall k: int :: (k >= l && k <= r) ==> a[k] > a[l-1]); + { + if ( a[i] <= pivot) { + tmp := a[i]; a[i] := a[sv]; a[sv] := tmp; + tmp := perm[i]; perm[i] := perm[sv]; perm[sv] := tmp; + sv := sv +1; + } + i := i + 1; + } + + //swap + tmp := a[i]; a[i] := a[sv]; a[sv] := tmp; + tmp := perm[i]; perm[i] := perm[sv]; perm[sv] := tmp; + + p := sv; +} + + +procedure quicksort(l: int, r: int, N: int) + modifies a, perm; + + requires N > 0; + requires l >= 0 && l < r && r < N; + requires ((r+1) < N) ==> (forall k: int :: (k >= l && k <= r) ==> a[k] <= a[r+1]); + requires ((l-1) >= 0) ==> (forall k: int :: (k >= l && k <= r) ==> a[k] > a[l-1]); + + /* a is a permutation of the original array original */ + requires is_permutation(a, original, perm, N); + + ensures ((r+1) < N) ==> (forall k: int :: (k >= l && k <= r) ==> a[k] <= a[r+1]); + ensures ((l-1) >= 0) ==> (forall k: int :: (k >= l && k <= r) ==> a[k] > a[l-1]); + + ensures is_unchanged(a, old(a), 0, l-1); + ensures is_unchanged(a, old(a), r+1, N); + ensures is_sorted(a, l, r); + + /* a is a permutation of the original array original */ + ensures is_permutation(a, original, perm, N); +{ + var p: int; + + call p := partition(l, r, N); + + if ((p-1) > l) { + call quicksort(l, p-1, N); + } + + if ((p+1) < r) { + call quicksort(p+1, r, N); + } +}
tests/examplefiles/test-exist-update.xq+75 −0 added@@ -0,0 +1,75 @@ +xquery version "3.0"; + +declare function local:add-log-message($message as xs:string) as empty-sequence()? +{ + let $logfile-collection := "/db/apps/exist101/log" + let $logfile-name := "exist101-log.xml" + let $logfile-full := concat($logfile-collection, '/', $logfile-name) + let $logfile-created := + if(doc-available($logfile-full))then + $logfile-full + else + xmldb:store($logfile-collection, $logfile-name, <eXist101-Log/>) + return + update insert + <LogEntry timestamp="{current-dateTime()}">{$message}</LogEntry> + into doc($logfile-full)/* +}; + +declare function local:insert-attributes() { + let $elm as element() := doc('/db/Path/To/Some/Document.xml')/* + return ( + update insert <NEW/> into $elm, + update insert attribute x { 'y' } into $elm/*[last()], + update insert attribute a { 'b' } into $elm/*[last()] + ) +}; + +declare function local:insert-elem() { + let $elm as element() := doc('/db/Path/To/Some/Document.xml')/* + return + update insert <NEW x="y" a="b"/> into $elm +}; + +declare function local:insert-elem2() { + let $elm as element() := doc('/db/Path/To/Some/Document.xml')/* + let $new-element as element() := <NEW x="y" a="b"/> + return + update insert $new-element into $elm +}; + +declare function local:insert-single() { + update insert <LogEntry>Something happened...</LogEntry> into doc('/db/logs/mainlog.xml')/* +}; + + +declare function local:trim-insert() { + let $document := doc('/db/logs/mainlog.xml') + let $newentry := <LogEntry>Something happened...</LogEntry> + return + update delete $document/*/LogEntry[position() ge 10], + if(exists($document/*/LogEntry[1]))then + update insert $newentry preceding $document/*/LogEntry[1] + else + update insert $newentry into $document/* +}; + + +declare function local:attempt-document-node-insert() { + + (: This is invalid: :) + let $document as document-node() := <Root><a/></Root> + return + update insert <b/> into $document/* +}; + +declare function local:attempt-attr-update-with-node() { + update replace doc('/db/test/test.xml')/*/@name with + <a>aaa<b>bbb</b></a> +}; + + +(# exist:batch-transaction #) { + update delete $document/*/LogEntry[position() ge 10], + update insert $newentry preceding $document/*/LogEntry[1] +} \ No newline at end of file
tests/examplefiles/test.orc+257 −0 added@@ -0,0 +1,257 @@ +// This is a Csound orchestra file for testing a Pygments <http://pygments.org> +// lexer. Csound single-line comments can be preceded by a pair of forward +// slashes... +; ...or a semicolon. + +/* Block comments begin with /* and end with */ + +// Orchestras begin with a header of audio parameters. +nchnls = 1 +nchnls_i = 1 +sr = 44100 +0dbfs = 1 +ksmps = 10 + +// The control rate kr = sr / ksmps can be omitted when the number of audio +// samples in a control period (ksmps) is set, but kr may appear in older +// orchestras. +kr = 4410 + +// Orchestras contain instruments. These begin with the keyword instr followed +// by a comma-separated list of numbers or names of the instrument. Instruments +// end at the endin keyword and cannot be nested. +instr 1, N_a_M_e_, +Name + // Instruments contain statements. Here is a typical statement: + aSignal oscil 0dbfs, 440, 1 + // Statements are terminated with a newline (possibly preceded by a comment). + // To write a statement on several lines, precede the newline with a + // backslash. + prints \ + "hello, world\n";comment + + // Csound 6 introduced function syntax for opcodes with one or zero outputs. + // The oscil statement above is the same as + aSignal = oscil(0dbfs, 440, 1) + + // Instruments can contain control structures. + kNote = p3 + if (kNote == 0) then + kFrequency = 220 + elseif kNote == 1 then // Parentheses around binary expressions are optional. + kFrequency = 440 + endif + + // Csound 6 introduced looping structures. + iIndex = 0 + while iIndex < 5 do + print iIndex + iIndex += 1 + od + iIndex = 0 + until iIndex >= 5 do + print iIndex + iIndex += 1 + enduntil + // Both kinds of loops can be terminated by either od or enduntil. + + // Single-line strings are enclosed in double-quotes. + prints "string\\\r\n\t\"" + // Multi-line strings are enclosed in pairs of curly braces. + prints {{ + hello, + + world + }} + + // Instruments often end with a statement containing an output opcode. + outc aSignal +endin + +// Orchestras can also contain user-defined opcodes (UDOs). Here is an +// oscillator with one audio-rate output and two control-rate inputs: +opcode anOscillator, a, kk + kAmplitude, kFrequency xin + aSignal vco2 kAmplitude, kFrequency + xout aSignal +endop +instr TestOscillator + outc(anOscillator(0dbfs, 110)) +endin + +// Python can be executed in Csound +// <http://www.csounds.com/manual/html/pyrun.html>. So can Lua +// <http://www.csounds.com/manual/html/lua.html>. +pyruni {{ +import random + +pool = [(1 + i / 10.0) ** 1.2 for i in range(100)] + +def get_number_from_pool(n, p): + if random.random() < p: + i = int(random.random() * len(pool)) + pool[i] = n + return random.choice(pool) +}} + +// The Csound preprocessor supports conditional compilation and including files. +#ifdef DEBUG +#undef DEBUG +#include "filename.orc" +#endif + +// The preprocessor also supports object- and function-like macros. This is an +// object-like macro that defines a number: +#define A_HZ #440# + +// This is a function-like macro: +#define OSCIL_MACRO(VOLUME'FREQUENCY'TABLE) #oscil $VOLUME, $FREQUENCY, $TABLE# + +// Bodies of macros are enclosed in # and can contain newlines. The arguments of +// function-like macros are separated by single-quotes. Uses of macros are +// prefixed with a dollar sign. +instr TestMacro + aSignal $OSCIL_MACRO(1'$A_HZ'1) + // Not unlike PHP, macros expand in double-quoted strings. + prints "The frequency of the oscillator is $A_HZ Hz.\n" + out aSignal +endin + +// Here are other things to note about Csound. + +// There are two bitwise NOT operators, ~ and ¬ (U+00AC). The latter is common +// on keyboards in the United Kingdom +// <https://en.wikipedia.org/wiki/British_and_American_keyboards>. +instr TestBitwiseNOT + print ~42 + print ¬42 +endin + +// Csound uses # for bitwise XOR, which the Csound manual calls bitwise +// non-equivalence <http://www.csounds.com/manual/html/opnonequiv.html>. +instr TestBitwiseXOR + print 0 # 0 + print 0 # 1 + print 1 # 0 + print 1 # 1 +endin + +// Loops and if-then statements are relatively recent additions to Csound. There +// are many flow-control opcodes that involve goto and labels. +instr TestGoto + // This... + if p3 > 0 goto if_label + goto else_label +if_label: + prints "if branch\n" + goto endif_label +else_label: + prints "else branch\n" +endif_label: + + // ...is the same as this. + if p3 > 0 then + prints "if branch\n" + else + prints "else branch\n" + endif + + // This... + iIndex = 0 +loop_label: + print iIndex + iIndex += 1 + if iIndex < 10 goto loop_label + + // ...is the same as this... + iIndex = 0 +loop_lt_label: + print iIndex + loop_lt iIndex, 1, 10, loop_lt_label + + // ...and this. + iIndex = 0 + while iIndex < 10 do + print iIndex + iIndex += 1 + od +endin + +// The prints and printks opcodes +// <https://github.com/csound/csound/blob/develop/OOps/ugrw1.c#L831>, arguably +// the primary methods of logging output, treat certain sequences of characters +// different from printf in C. +instr TestPrints + // ^ prints an ESCAPE character (U+001B), not a CIRCUMFLEX ACCENT character + // (U+005E). ^^ prints a CIRCUMFLEX ACCENT. + prints "^^\n" + // ~ prints an ESCAPE character (U+001B) followed by a [, not a TILDE + // character (U+007E). ~~ prints a TILDE. + prints "~~\n" + // \A, \B, \N, \R, and \T correspond to the escaped lowercase characters (that + // is, BELL (U+0007), BACKSPACE (U+0008), new line (U+000A), CARRIAGE RETURN + // (U+000D), and tab (U+0009)). + prints "\T\R\N" + // %n, %r, and %t are the same as \n, \r, and \t, as are %N, %R, and %T. + prints "%t%r%n" + // %! prints a semicolon. This is a hold-over from old versions of Csound that + // allowed comments to begin in strings. + prints "; %!\n" +endin + +// The arguments of function-like macros can be separated by # instead of '. +// These two lines define the same macro. +#define OSCIL_MACRO(VOLUME'FREQUENCY'TABLE) #oscil $VOLUME, $FREQUENCY, $TABLE# +#define OSCIL_MACRO(VOLUME#FREQUENCY#TABLE) #oscil $VOLUME, $FREQUENCY, $TABLE# + +// Uses of macros can optionally be suffixed with a period. +instr TestMacroPeriodSuffix + aSignal $OSCIL_MACRO.(1'$A_HZ'1) + prints "The frequency of the oscillator is $A_HZ.Hz.\n" + out aSignal +endin + +// Csound has @ and @@ operator-like macros that, when followed by a literal +// non-negative integer, expand to the next power of 2 and the next power of 2 +// plus 1: +// @x = 2^(ceil(log2(x + 1))), x >= 0 +// @@0 = 2 +// @@x = 2^(ceil(log2(x))) + 1, x > 0 +// These macros are in +// <https://github.com/csound/csound/blob/develop/Engine/csound_orc.l#L542> (and +// <https://github.com/csound/csound/blob/develop/Engine/csound_sco.lex#L202>) +// and are described at <http://www.csounds.com/manual/html/ScoreEval.html>. +instr TestAt + prints "%d %2d %2d\n", 0, @0, @@0 + prints "%d %2d %2d\n", 1, @1, @@1 + prints "%d %2d %2d\n", 2, @2, @@2 + prints "%d %2d %2d\n", 3, @3, @@3 + prints "%d %2d %2d\n", 4, @4, @@4 + prints "%d %2d %2d\n", 5, @5, @@5 + prints "%d %2d %2d\n", 6, @6, @@6 + prints "%d %2d %2d\n", 7, @7, @@7 + prints "%d %2d %2d\n", 8, @8, @@8 + prints "%d %2d %2d\n", 9, @9, @@9 +endin + +// Including newlines in macros can lead to confusing code, but it tests the +// lexer. +instr MacroAbuse + if 1 == 1 then + prints "on\n" +#define FOO# +BAR +#endif // This ends the if statement. It is not a preprocessor directive. +endin + +scoreline_i {{ +f 1 0 16384 10 1 +i "N_a_M_e_" 0 2 +i "TestOscillator" 2 2 +i "TestBitwiseNOT" 0 1 +i "TestBitwiseXOR" 0 1 +i "TestGoto" 0 1 +i "TestMacroPeriodSuffix" 4 1 +i "TestAt" 0 1 +i "MacroAbuse" 0 1 +e +}}
tests/examplefiles/test.psl+182 −0 added@@ -0,0 +1,182 @@ +// This is a comment + +// 1. Basics + +// Functions +func Add(X : Univ_Integer; Y : Univ_Integer) -> Univ_Integer is + return X + Y; +end func Add; +// End of line semi-colons are optional +// +, +=, -, -=, *, *=, /, /= +// all do what you'd expect (/ is integer division) + +// If you find Univ_Integer to be too verbose you can import Short_Names +// which defines aliases like Int for Univ_Integer and String for Univ_String +import PSL::Short_Names::*, * + +func Greetings() is + const S : String := "Hello, World!" + Println(S) +end func Greetings +// All declarations are 'const', 'var', or 'ref' +// Assignment is :=, equality checks are ==, and != is not equals + +func Boolean_Examples(B : Bool) is + const And := B and #true // Parallel execution of operands + const And_Then := B and then #true // Short-Circuit + const Or := B or #false // Parallel execution of operands + const Or_Else := B or else #false // Short-Cirtuit + const Xor := B xor #true + var Result : Bool := #true; + Result and= #false; + Result or= #true; + Result xor= #false; +end func Boolean_Examples +// Booleans are a special type of enumeration +// All enumerations are preceded by a sharp '#' + +func Fib(N : Int) {N >= 0} -> Int is + if N <= 1 then + return N + else + // Left and right side of '+' are computed in Parallel here + return Fib(N - 1) + Fib(N - 2) + end if +end func Fib +// '{N >= 0}' is a precondition to this function +// Preconditions are built in to the language and checked by the compiler + +// ParaSail does not have mutable global variables +// Instead, use 'var' parameters +func Increment_All(var Nums : Vector<Int>) is + for each Elem of Nums concurrent loop + Elem += 1 + end loop +end func Increment_All +// The 'concurrent' keyword in the loop header tells the compiler that +// iterations of the loop can happen in any order. +// It will choose the most optimal number of threads to use. +// Other options are 'forward' and 'reverse'. + +func Sum_Of_Squares(N : Int) -> Int is + // The type of Sum is inferred + var Sum := 0 + for I in 1 .. N forward loop + Sum += I ** 2 // ** is exponentiation + end loop +end func Sum_Of_Squares + +func Sum_Of(N : Int; Map : func (Int) -> Int) -> Int is + return (for I in 1 .. N => <0> + Map(I)) +end func Sum_Of +// It has functional aspects as well +// Here, we're taking an (Int) -> Int function as a parameter +// and using the inherently parallel map-reduce. +// Initial value is enclosed with angle brackets + +func main(Args : Basic_Array<String>) is + Greetings() // Hello World + Println(Fib(5)) // 5 + // Container Comprehension + var Vec : Vector<Int> := [for I in 0 .. 10 {I mod 2 == 0} => I ** 2] + // Vec = [0, 4, 16, 36, 64, 100] + Increment_All(Vec) + // Vec = [1, 5, 17, 37, 65, 101] + // '|' is an overloaded operator. + // It's usually used for concatenation or adding to a container + Println("First: " | Vec[1] | ", Last: " | Vec[Length(Vec)]); + // Vectors are 1 indexed, 0 indexed ZVectors are also available + + Println(Sum_Of_Squares(3)) + + // Sum of fibs! + Println(Sum_Of(10, Fib)) +end func main + +// Preceding a type with 'optional' allows it to take the value 'null' +func Divide(A, B, C : Real) -> optional Real is + // Real is the floating point type + const Epsilon := 1.0e-6; + if B in -Epsilon .. Epsilon then + return null + elsif C in -Epsilon .. Epsilon then + return null + else + return A / B + A / C + end if +end func Divide + +// 2. Modules +// Modules are composed of an interface and a class +// ParaSail has object orientation features + +// modules can be defined as 'concurrent' +// which allows 'locked' and 'queued' parameters +concurrent interface Locked_Box<Content_Type is Assignable<>> is + // Create a box with the given content + func Create(C : optional Content_Type) -> Locked_Box; + + // Put something into the box + func Put(locked var B : Locked_Box; C : Content_Type); + + // Get a copy of current content + func Content(locked B : Locked_Box) -> optional Content_Type; + + // Remove current content, leaving it null + func Remove(locked var B : Locked_Box) -> optional Content_Type; + + // Wait until content is non-null, then return it, leaving it null. + func Get(queued var B : Locked_Box) -> Content_Type; +end interface Locked_Box; + +concurrent class Locked_Box is + var Content : optional Content_Type; +exports + func Create(C : optional Content_Type) -> Locked_Box is + return (Content => C); + end func Create; + + func Put(locked var B : Locked_Box; C : Content_Type) is + B.Content := C; + end func Put; + + func Content(locked B : Locked_Box) -> optional Content_Type is + return B.Content; + end func Content; + + func Remove(locked var B : Locked_Box) -> Result : optional Content_Type is + // '<==' is the move operator + // It moves the right operand into the left operand, + // leaving the right null. + Result <== B.Content; + end func Remove; + + func Get(queued var B : Locked_Box) -> Result : Content_Type is + queued until B.Content not null then + Result <== B.Content; + end func Get; +end class Locked_Box; + +func Use_Box(Seed : Univ_Integer) is + var U_Box : Locked_Box<Univ_Integer> := Create(null); + // The type of 'Ran' can be left out because + // it is inferred from the return type of Random::Start + var Ran := Random::Start(Seed); + + Println("Starting 100 pico-threads trying to put something in the box"); + Println(" or take something out."); + for I in 1..100 concurrent loop + if I < 30 then + Println("Getting out " | Get(U_Box)); + else + Println("Putting in " | I); + U_Box.Put(I); + + // The first parameter can be moved to the front with a dot + // X.Foo(Y) is equivalent to Foo(X, Y) + end if; + end loop; + + Println("And the winner is: " | Remove(U_Box)); + Println("And the box is now " | Content(U_Box)); +end func Use_Box;
tests/examplefiles/test.shen+137 −0 added@@ -0,0 +1,137 @@ +(package pygments-test [some symbols] + +\* multiline + comment +*\ + +\\ With vars as functions + +(define super + [Value Succ End] Action Combine Zero -> + (if (End Value) + Zero + (Combine (Action Value) + (super [(Succ Value) Succ End] + Action Combine Zero)))) + +(define for + Stream Action -> (super Stream Action (function do) 0)) + +(define filter + Stream Condition -> + (super Stream + (/. Val (if (Condition Val) [Val] [])) + (function append) + [])) + +(for [0 (+ 1) (= 10)] (function print)) + +(filter [0 (+ 1) (= 100)] + (/. X (integer? (/ X 3)))) + + +\\ Typed functions + +(define typed-map + { (A --> B) --> (list A) --> (list B) } + F X -> (typed-map-h F X [])) + +(define typed-map-h + { (A --> B) --> (list A) --> (list B) \\ comment + --> (list B) } + _ [] X -> (reverse X) + F [X | Y] Z -> (typed-map-h F Y [(F X) | Z])) + +(define append-string + { string --> string \* comment *\ --> string } + S1 S2 -> (cn S1 S2)) + +(let X 1 + Y 2 + (+ (type X number) (type Y number))) + +\\ Yacc + +(defcc <st_input> + <lrb> <st_input1> <rrb> <st_input2> + := (package-macro (macroexpand <st_input1>) <st_input2>); + <lcurly> <st_input> := [{ | <st_input>]; + <rcurly> <st_input> := [} | <st_input>]; + <bar> <st_input> := [bar! | <st_input>]; + <semicolon> <st_input> := [; | <st_input>]; + <colon> <equal> <st_input> := [:= | <st_input>]; + <colon> <minus> <st_input> := [:- | <st_input>]; + <colon> <st_input> := [: | <st_input>]; + <comma> <st_input> := [(intern ",") | <st_input>]; + <e> := [];) + +(defcc <lsb> + 91 := skip;) + +\\ Pattern matching + +(define matches + 1 X 3 -> X + X Y Z -> Y where (and (= X 1) (= Z 3)) + true false _ -> true + (@p a X c) (@s X "abc") (@v 1 2 3 <>) -> true + [X | Rest] [] [a b c] -> true + [(@p a b)] [[[1] 2] X] "string" -> true + _ _ _ -> false) + + +\\ Prolog + +(defprolog th* + X A Hyps <-- (show [X : A] Hyps) (when false); + X A _ <-- (fwhen (typedf? X)) (bind F (sigf X)) (call [F A]); + (mode [F] -) A Hyp <-- (th* F [--> A] Hyp); + (mode [cons X Y] -) [list A] Hyp <-- (th* X A Hyp) (th* Y [list A] Hyp); + (mode [@s X Y] -) string Hyp <-- (th* X string Hyp) (th* Y string Hyp); + (mode [lambda X Y] -) [A --> B] Hyp <-- ! + (bind X&& (placeholder)) + (bind Z (ebr X&& X Y)) + (th* Z B [[X&& : A] | Hyp]); + (mode [type X A] -) B Hyp <-- ! (unify A B) (th* X A Hyp);) + +\\ Macros + +(defmacro log-macro + [log N] -> [log N 10]) + +\\ Sequent calculus + +(datatype rank + + if (element? X [ace 2 3 4 5 6 7 8 9 10 jack queen king]) + ________ + X : rank;) + +(datatype suit + + if (element? Suit [spades hearts diamonds clubs]) + _________ + Suit : suit;) + +(datatype card + + Rank : rank; Suit : suit; + _________________ + [Rank Suit] : card; + + Rank : rank, Suit : suit >> P; + _____________________ + [Rank Suit] : card >> P;) + +(datatype card + + Rank : rank; Suit : suit; + ================== + [Rank Suit] : card;) + +\\ String interpolation and escape sequences + +"abc~A ~S~R ~% blah + c#30;c#31;blah" + +)
tests/examplefiles/yahalom.cpsa+34 −0 added@@ -0,0 +1,34 @@ +(herald "Yahalom Protocol with Forwarding Removed") + +(defprotocol yahalom basic + (defrole init + (vars (a b c name) (n-a n-b text) (k skey)) + (trace (send (cat a n-a)) + (recv (enc b k n-a n-b (ltk a c))) + (send (enc n-b k)))) + (defrole resp + (vars (b a c name) (n-a n-b text) (k skey)) + (trace (recv (cat a n-a)) + (send (cat b (enc a n-a n-b (ltk b c)))) + (recv (enc a k (ltk b c))) + (recv (enc n-b k)))) + (defrole serv + (vars (c a b name) (n-a n-b text) (k skey)) + (trace (recv (cat b (enc a n-a n-b (ltk b c)))) + (send (enc b k n-a n-b (ltk a c))) + (send (enc a k (ltk b c)))) + (uniq-orig k))) + +(defskeleton yahalom + (vars (a b c name) (n-b text)) + (defstrand resp 4 (a a) (b b) (c c) (n-b n-b)) + (non-orig (ltk b c) (ltk a c)) + (uniq-orig n-b)) + +;;; Ensure encryption key remains secret. +(defskeleton yahalom + (vars (a b c name) (n-b text) (k skey)) + (defstrand resp 4 (a a) (b b) (c c) (n-b n-b) (k k)) + (deflistener k) + (non-orig (ltk b c) (ltk a c)) + (uniq-orig n-b))
tests/test_basic_api.py+4 −7 modified@@ -98,13 +98,10 @@ def verify(cls): inst = cls(stripall=True) ensure(inst.get_tokens(' \n b\n\n\n'), 'b\n') # some lexers require full lines in input - if cls.__name__ not in ( - 'PythonConsoleLexer', 'RConsoleLexer', 'RubyConsoleLexer', - 'SqliteConsoleLexer', 'MatlabSessionLexer', 'ErlangShellLexer', - 'BashSessionLexer', 'LiterateHaskellLexer', 'LiterateAgdaLexer', - 'PostgresConsoleLexer', 'ElixirConsoleLexer', 'JuliaConsoleLexer', - 'RobotFrameworkLexer', 'DylanConsoleLexer', 'ShellSessionLexer', - 'LiterateIdrisLexer', 'LiterateCryptolLexer'): + if ('ConsoleLexer' not in cls.__name__ and + 'SessionLexer' not in cls.__name__ and + not cls.__name__.startswith('Literate') and + cls.__name__ not in ('ErlangShellLexer', 'RobotFrameworkLexer')): inst = cls(ensurenl=False) ensure(inst.get_tokens('a\nb'), 'a\nb') inst = cls(ensurenl=False, stripall=True)
tests/test_irc_formatter.py+30 −0 added@@ -0,0 +1,30 @@ +# -*- coding: utf-8 -*- +""" + Pygments IRC formatter tests + ~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + + :copyright: Copyright 2006-2015 by the Pygments team, see AUTHORS. + :license: BSD, see LICENSE for details. +""" + +from __future__ import print_function + +import re +import unittest + +from pygments.util import StringIO +from pygments.lexers import PythonLexer +from pygments.formatters import IRCFormatter + +import support + +tokensource = list(PythonLexer().get_tokens("lambda x: 123")) + +class IRCFormatterTest(unittest.TestCase): + def test_correct_output(self): + hfmt = IRCFormatter() + houtfile = StringIO() + hfmt.format(tokensource, houtfile) + + self.assertEqual(u'\x0302lambda\x03 x: \x0302123\x03\n', houtfile.getvalue()) +
tests/test_latex_formatter.py+1 −1 modified@@ -44,7 +44,7 @@ def test_valid_output(self): po.stdout.close() except OSError: # latex not available - pass + raise support.SkipTest else: if ret: print(output)
tests/test_lexers_other.py+19 −4 modified@@ -6,14 +6,12 @@ :copyright: Copyright 2006-2015 by the Pygments team, see AUTHORS. :license: BSD, see LICENSE for details. """ - import glob import os import unittest from pygments.lexers import guess_lexer -from pygments.lexers.scripting import RexxLexer - +from pygments.lexers.scripting import EasytrieveLexer, JclLexer, RexxLexer def _exampleFilePath(filename): return os.path.join(os.path.dirname(__file__), 'examplefiles', filename) @@ -36,7 +34,24 @@ def _testCanRecognizeAndGuessExampleFiles(self, lexer): self.assertEqual(guessedLexer.name, lexer.name) def testCanRecognizeAndGuessExampleFiles(self): - self._testCanRecognizeAndGuessExampleFiles(RexxLexer) + LEXERS_TO_TEST = [ + EasytrieveLexer, + JclLexer, + RexxLexer, + ] + for lexerToTest in LEXERS_TO_TEST: + self._testCanRecognizeAndGuessExampleFiles(lexerToTest) + + +class EasyTrieveLexerTest(unittest.TestCase): + def testCanGuessFromText(self): + self.assertLess(0, EasytrieveLexer.analyse_text('MACRO')) + self.assertLess(0, EasytrieveLexer.analyse_text('\nMACRO')) + self.assertLess(0, EasytrieveLexer.analyse_text(' \nMACRO')) + self.assertLess(0, EasytrieveLexer.analyse_text(' \n MACRO')) + self.assertLess(0, EasytrieveLexer.analyse_text('*\nMACRO')) + self.assertLess(0, EasytrieveLexer.analyse_text( + '*\n *\n\n \n*\n MACRO')) class RexxLexerTest(unittest.TestCase):
tests/test_shell.py+26 −0 modified@@ -61,3 +61,29 @@ def testParsedSingle(self): ] self.assertEqual(tokens, list(self.lexer.get_tokens(fragment))) + def testShortVariableNames(self): + fragment = u'x="$"\ny="$_"\nz="$abc"\n' + tokens = [ + # single lone $ + (Token.Name.Variable, u'x'), + (Token.Operator, u'='), + (Token.Literal.String.Double, u'"'), + (Token.Text, u'$'), + (Token.Literal.String.Double, u'"'), + (Token.Text, u'\n'), + # single letter shell var + (Token.Name.Variable, u'y'), + (Token.Operator, u'='), + (Token.Literal.String.Double, u'"'), + (Token.Name.Variable, u'$_'), + (Token.Literal.String.Double, u'"'), + (Token.Text, u'\n'), + # multi-letter user var + (Token.Name.Variable, u'z'), + (Token.Operator, u'='), + (Token.Literal.String.Double, u'"'), + (Token.Name.Variable, u'$abc'), + (Token.Literal.String.Double, u'"'), + (Token.Text, u'\n'), + ] + self.assertEqual(tokens, list(self.lexer.get_tokens(fragment)))
tests/test_terminal_formatter.py+51 −0 added@@ -0,0 +1,51 @@ +# -*- coding: utf-8 -*- +""" + Pygments terminal formatter tests + ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + + :copyright: Copyright 2006-2015 by the Pygments team, see AUTHORS. + :license: BSD, see LICENSE for details. +""" + +from __future__ import print_function + +import unittest +import re + +from pygments.util import StringIO +from pygments.lexers.sql import PlPgsqlLexer +from pygments.formatters import TerminalFormatter + +DEMO_TEXT = '''\ +-- comment +select +* from bar; +''' +DEMO_LEXER = PlPgsqlLexer +DEMO_TOKENS = list(DEMO_LEXER().get_tokens(DEMO_TEXT)) + +ANSI_RE = re.compile(r'\x1b[\w\W]*?m') + +def strip_ansi(x): + return ANSI_RE.sub('', x) + +class TerminalFormatterTest(unittest.TestCase): + def test_reasonable_output(self): + out = StringIO() + TerminalFormatter().format(DEMO_TOKENS, out) + plain = strip_ansi(out.getvalue()) + self.assertEqual(DEMO_TEXT.count('\n'), plain.count('\n')) + print(repr(plain)) + + for a, b in zip(DEMO_TEXT.splitlines(), plain.splitlines()): + self.assertEqual(a, b) + + def test_reasonable_output_lineno(self): + out = StringIO() + TerminalFormatter(linenos=True).format(DEMO_TOKENS, out) + plain = strip_ansi(out.getvalue()) + self.assertEqual(DEMO_TEXT.count('\n') + 1, plain.count('\n')) + print(repr(plain)) + + for a, b in zip(DEMO_TEXT.splitlines(), plain.splitlines()): + self.assertTrue(a in b)
Vulnerability mechanics
Generated by null/stub on May 9, 2026. Inputs: CWE entries + fix-commit diffs from this CVE's patches. Citations validated against bundle.
References
12- github.com/advisories/GHSA-fff8-4w9p-7v76ghsaADVISORY
- nvd.nist.gov/vuln/detail/CVE-2015-8557ghsaADVISORY
- packetstormsecurity.com/files/133823/Pygments-FontManager._get_nix_font_path-Shell-Injection.htmlnvdWEB
- seclists.org/fulldisclosure/2015/Oct/4nvdWEB
- www.debian.org/security/2016/dsa-3445nvdWEB
- www.openwall.com/lists/oss-security/2015/12/14/17nvdWEB
- www.openwall.com/lists/oss-security/2015/12/14/6nvdWEB
- www.oracle.com/technetwork/topics/security/bulletinjan2016-2867206.htmlnvdWEB
- www.ubuntu.com/usn/USN-2862-1nvdWEB
- github.com/pygments/pygments/commit/db6dd826f8624179e563aaded391efe824462f51ghsaWEB
- github.com/pypa/advisory-database/tree/main/vulns/pygments/PYSEC-2016-32.yamlghsaWEB
- security.gentoo.org/glsa/201612-05nvdWEB
News mentions
0No linked articles in our index yet.