overview

This is a work-in-progress demonstrator of a Mathematical Expression Recognizer (Last update: September, 2016).

The recognition engine is based on SESHAT, an open-source system for recognizing handwritten math expressions.

instructions

  1. Write an expression on the 'draw here' area.
  2. Click on the 'send' link to translate it to a LaTeX equation.

publications

  1. 2014, ICPR - Offline Features for Classifying Handwritten Math Symbols with Recurrent Neural Networks
  2. 2013, DocEng - A shape-based layout descriptor for classifying spatial relationships in handwritten math
  3. 2013, ICDAR - Classification of On-line Mathematical Symbols with Hybrid Features and Recurrent Neural Networks
  4. 2012, Pattern Recognition Letters - Recognition of On-line Handwritten Mathematical Expressions Using 2D Stochastic Context-Free Grammars and Hidden Markov Models
  5. 2011, ICDAR - Recognition of Printed Mathematical Expressions Using Two-dimensional Stochastic Context-Free Grammars
  6. 2010, ICPR - Comparing Several Techniques for Offline Recognition of Printed Mathematical Symbols

awards

  1. 2014, CROHME: Competition on Recognition of Online Handwritten Mathematical Expressions
  2. 2013, CROHME: Competition on Recognition of Online Handwritten Mathematical Expressions
  3. 2011, CROHME: Competition on Recognition of Online Handwritten Mathematical Expressions

aplications

  1. 2015, ╬╝captcha: Solving captchas on touchscreens made easy

accepted expressions

At present, the system is trained to recognize more than 100 symbols:
a-z A-Z 0-9 < > ≤ ≥ = ≠ - + ± × ! | . , ... ( ) [ ] { } ′ → ∃ ∀ ∈ ∫ √ ∞ α β Δ γ λ μ φ π σ Σ ∏ θ lim log cos sin tan

The system expects complete equations, hence, an expression like "\(2+2=\)" would not be recognized because an equal sign requires a right-hand operator (e.g. "\(2+2=4\)").

Most of math expressions are accepted, containing subscripts (\(a_1\)), superscripts (\(x^2\)), fractions (\(\displaystyle\frac{a+b}{c}\)), square roots (\(\sqrt{\pi^2}\) or \(\sqrt[3]{4y}\)), integral (\(\displaystyle\int_a^b x\)), summation (\(\displaystyle\sum_{i=0}^n x+i\)), limits (\(\displaystyle\lim_{x \rightarrow \infty}\frac{1}{x}\)), functions (\(\sin^2 \alpha\)), ...

On the other hand, matrices \(\displaystyle\left(\begin{matrix} 1 & 2 \\ 3 & 4 \end{matrix}\right)\), left scripts (\({}_nC_m\)) or piecewise functions (\( f(n) = \left\{ \begin{array}{l l} n & \quad n \geq 0\\ 1-n & \quad n < 0 \end{array} \right. \)) are not recognized yet.

It is expected to increase the number of symbols and accepted expressions in the short-term, as well as improving the recognition system and the expression accepted.