Metadata-Version: 2.1
Name: Mathics-Scanner
Version: 1.3.0
Summary: Character Tables and Tokenizer for Mathics and the Wolfram Language.
Home-page: https://mathics.org/
Maintainer: Mathics Group
License: GPL-3.0-only
Keywords: Mathematica,Wolfram,Interpreter,Shell,Math,CAS
Classifier: Intended Audience :: Developers
Classifier: Intended Audience :: Science/Research
Classifier: License :: OSI Approved :: GNU General Public License v3 (GPLv3)
Classifier: Programming Language :: Python
Classifier: Programming Language :: Python :: 3.6
Classifier: Programming Language :: Python :: 3.7
Classifier: Programming Language :: Python :: 3.8
Classifier: Programming Language :: Python :: 3.9
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: Implementation :: CPython
Classifier: Programming Language :: Python :: Implementation :: PyPy
Classifier: Topic :: Scientific/Engineering
Classifier: Topic :: Scientific/Engineering :: Mathematics
Classifier: Topic :: Scientific/Engineering :: Physics
Classifier: Topic :: Software Development :: Interpreters
Description-Content-Type: text/x-rst
Provides-Extra: dev
Provides-Extra: full
License-File: COPYING.txt
License-File: AUTHORS.txt

|Workflows| |Pypi Installs| |Latest Version| |Supported Python Versions|

|Packaging status|

Mathics Character Tables and Scanner
=====================================

This repository really contains two things:

* extensive tables describing WL symbols and operators their properties
* a tokenizer or scanner portion for the Wolfram Language.

With respect to the first item, there is a commented YAML that contains a
full set of translation between:

* Wolfram Language named characters,
* Unicode/ASCII equivalents and Unicode and WL code-points,
* AMSLaTeX equivalent names,
* Operator name (if symbol is an operator),
* Operator precedence (if an operator)
* Keyboard escape sequences for the symbol

Uses
----

The scanner and character tables are used inside `Mathics <https://mathics.org>`_. However information can
also be used by other programs for tokenizing and formatting Wolfram Language code.

For example, tables are used in `mathics-pygments <https://pypi.org/project/Mathics-Scanner/>`_, a Pygments-based
lexer and highlighter for Mathematica/Wolfram Language source code.

This library may be useful if you need to work with Wolfram Language
named character and convert them to various formats.

Usage
-----

- For tokenizing and scanning Wolfram Language code, use the
  ``mathics_scanner.tokenizer.Tokenizer`` class.
- To convert between Wolfram Language named characters and Unicode/ASCII, use
  the ``mathics_scanner.characters.replace_wl_with_plain_text`` and
  ``mathics_scanner.characters.replace_unicode_with_wl`` functions.
- To convert between qualified names of named characters (such ``FormalA`` for
  ``\[FormalA]``) and Wolfram's internal representation use the
  ``m̀athics_scanner.characters.named_characters`` dictionary.

To regenerate JSON-format tables run:

::

   $ mathics-generate-json-table

Without options ``mathics-generate-json-table`` produces the maximum set of correspondences.

In most applications though you may need just a few of these. The
``--field`` option can be used to narrow the list of entries to output in JSON. Run
``mathics-generate-json-table --help`` for a full list of fields.


Implementation
--------------

For notes on the implementation of the packages or details on the conversion
scheme please read `Implementation <https://mathics-scanner.readthedocs.io/en/latest/implementation.html>`_.

Contributing
------------

Please feel encouraged to contribute to this package or Mathics! Create your
own fork, make the desired changes, commit, and make a pull request.

License
-------

Mathics is released under the GNU General Public License Version 3 (GPL3).

.. |Workflows| image:: https://github.com/Mathics3/mathics-scanner/workflows/Mathics%20(ubuntu)/badge.svg
.. |Packaging status| image:: https://repology.org/badge/vertical-allrepos/mathics-scanner.svg
			    :target: https://repology.org/project/mathics-scanner/versions
.. |Latest Version| image:: https://badge.fury.io/py/Mathics-Scanner.svg
		 :target: https://badge.fury.io/py/Mathics-Scanner
.. |Pypi Installs| image:: https://pepy.tech/badge/Mathics-Scanner
.. |Supported Python Versions| image:: https://img.shields.io/pypi/pyversions/Mathics-Scanner.svg

