Multimodal Interactive Handwritten Transcription

Introduction

Choose Demo

First of all, you must choose one of the available demos by clicking on the transcribe button. By clicking on use custom server? link, you can specify a custom CATTI server while your session is active.

The demos are defined in XML files. All XML files are validated against our custom DTD. If an XML file is not valid, you cannot work on that demo.

Choose Page Thumbnail

Once you have selected a demo, you can navigate to any page.

Layout Description

The center block is the page to transcribe itself. The tidy menu on the right side is a pagination item. With that pagination item you can browse all the pages quickly.

You can browse all the pages visually by using the page browser, located above the center page. You may notice four different colors used on such a list:

Light gray

Non-transcribed segments.

Light green

Partially transcribed segments.

Dark green

Fully validated segments.

Light red

Locked segments (the segment cannot be edited; reload the page or try it later).

The bottom menu is intended to help you with common tasks: close session, change demo, view application shorcuts, or explore the API.

Loading a Transcription

By clicking on an image line, the system will propose an inital, automatic transcription. If you were working on that line previously, the trascription will be loaded from a log file. From here onwards you can do several operations to work on that image line.

Stylus Interaction

This is the interaction mode by default. You can use the application with any kind of pointing device, such as a stylus or a PC tablet. You can also use your computer mouse, but this option is discouraged. That is because the computer mouse, unfortunately, is not too good at drawing with precision.

Single Click Validation

By clicking on a word you will accept as prefix all precedent words, but not including the clicked word.

This gesture can be understood as a Rejection operation, since it behaves in that way.

By using your mouse wheel you will receive other transcription. The mouse wheel represents a reject gesture, very similar to the single click validation, but including some advantages:

1. A forward wheel scroll will ask a new translation and the previous one will be stored on an internal buffer.
2. A backward wheel scroll will load the previous translation from the internal buffer (if available).

Substitution

Place your stylus over an incorrect word and write down the correct text.

Insertion

Draw a vertical line between two words and then write down the text that you wish to insert.

Deletion

Draw a diagonal line over an incorrect word and it will be deleted. The deletion gesture must begin outside the text field boundaries. It is very important for the system to identify correctly this gesture. You can click on the debug link from the bottom menu to toggle the debug mode and see clearly the word boundaries. See this image for a visual clarification.

Accept Final Transcription

Draw a verification gesture (v-like) after the last corrected word to accept the full proposed transcription. The next image line will be loaded automatically.

Keyboard Interaction

To switch to keyboard mode you must press any key over the application interface. It is important to remark that while you move your stylus pen over the words, the focus is updated. So, since the keys work as expected, take these examples when switching to keyboard mode:

1. Clicking the space or delete key will delete the text.
2. Clicking the TAB key will change to the next word in the tab loop.
3. Clicking any alphanumeric or punctuation key will overwrite the word text.
4. Clicking non-character keys (UP, CTRL, CAPS, ALT, etc.) will give focus to the current word.

Rejection

This is a Mouse Click gesture emulation. Since you are using the keyboard and not the stylus pen, the interaction is different:

1. CRTL + UP will ask a new translation and the previous one will be stored on an internal buffer.
2. CRTL + DOWN will load the previous translation from the internal buffer.

You can also use the mouse wheel to reject sufixes:

1. A forward wheel scroll will ask a new translation and the previous one will be stored on an internal buffer.
2. A backward wheel scroll will load the previous translation from the internal buffer (if available).

Substitution

Once you have changed the word text, press ENTER or go to the next word (with the TAB key) to send the transcription prefix.

Insertion

Hold down CTRL + SPACE to insert a new text field to the right of your selected word.

If the SHIFT key is also pressed, the text field will be inserted to the left of your selected word instead.

Deletion

Hold down CTRL + DEL to delete the selected text field.

Accept Final Transcription

Hold down CTRL + ENTER to accept the full proposed transcription. The final transcription will be until the text field with focus. The next image line will be loaded automatically.

System Errors

You may experience connection problems due to firewall restrictions. The MM-CATTI system is able to connect both by direct socket and regular HTTP requests over 80 port. However, the fastest connection is achieved through direct socket.

If you experience a really slow connection try to open the required ports — you can click on use custom server? label on index page to see which port are used on each demo.

These are the errors that you may find when a connection is not possible:

Socket IO error

That means that a direct connection to CAT server is not available.

This error is dispatched when an input/output error occurs that causes a send or load operation to fail.

Socket Security error

It means that is not possible to connect to a socket outside the caller's security sandbox.

This error is dispatched if a socket attempts to connect either to a server outside the caller's security sandbox or to a port lower than 1024.

Socket Timeout

It indicates that it has been too long to wait for a connection.

If the connection doesn't succeed within the specified time, the connection fails. At this point, the system will switch the connection channel, that is, the socket connection is replaced for a traditional HTTP request over 80 port. Additionally, the system will use the HTTP connection channel for successive connections until the corpus changes.

Dispatcher IO error

That means that the web server tried to connect to CAT server, with no success.

This error is dispatched when an HTTP CAT request over 80 port is not possible.

Dispatcher Security error

It is a sanbox security error, generally caused by a cross-domain request.

This error is dispatched when an external domain tries to communicate with our MM-CATTI system.

Dispatcher Timeout exceed

It means that the server dispatcher is not responding.

At this point, the system will not be able to connect. You may try it later.