On the canvas, chips may be easily moved, resized, and ‘pushed’ onto the canvas (i.e., the symbols on the chip are added to the canvas, and the chip disappears).Ĭhips have two possible states: ‘recognized’ chips containing a string, and ‘template’ chips representing only symbols on a canvas. All formula chips have their creation history automatically recorded, and are stored in a ‘history’ menu in the symbol palette panel. In addition to the formula creation operations described above, chips can be created by selecting symbols on the canvas and ‘popping’ them up into a chip. Chips can be created and used in a number of ways. Figure 1 shows a chip in the query bar, and there are two ‘favorite’ chips in a list at bottom-right. MathSeer introduces a new model for formula reuse, flexible containers that we call formula ‘chips’. Users may also want to avoid re-entering formulas, and to share formulas with others. 1 handwriting is slow, and accurate recognition is challenging.
Handwritten formula entry is convenient for small expressions, but for large expressions such as Eq. 1).įormula Containers and Reuse (‘Chips’). In order to help the user add additional information for their query, MathSeer also supports keywords in their search queries (see Fig. found that users search for math with keywords or in the context of a question. Formulas in the query bar can be chosen for editing by clicking on them, allowing for quick switching between formulas. Robust undo/redo operations are provided to easily reverse operations. Users can freely alternate between drawing and manipulating symbols on the canvas, uploading images, and editing the panel contents. A line-of-sight graph-based parsing technique is used to recognize formula images and handwritten formulas. This produces a formula ‘chip’ on the canvas, which can be used directly in a query, edited, or used in constructing other formulas. Images may be dragged-and-dropped on the canvas or uploaded using a button that presents a file navigation pop-up window. At bottom-right is a panel for ‘favorite’ formulas (two are shown here), the formula history, and palettes for symbols and structures to insert in the panel. Formulas are created by manipulating symbols on the canvas, uploading formula images, and editing in the panel at bottom left. Query formulas and keywords are ‘chips’ at top left keywords are entered using the box at top right. At bottom-right, palettes containing symbols and structures may be used to insert corresponding at the cursor position in the panel. The can then be edited, with a rendering of the formula updated in real-time (e.g., to quickly change ‘p’ to ‘P’). Pressing a button recognizes formula structure, and copies the result into the panel at the bottom-left of the interface. 2 In MathSeer handwritten symbols are recognized each time a user stops drawing for a short time. 1) allows formula input using a combination of typing, uploading formula images, and drawing formulas by hand. To address these issues, our MathSeer search interface (see Fig. They also expressed formulas with handwriting that they could not using a keyboard (e.g., ). In one study, a majority of the undergraduate participants reported preferring drawing over typing formulas given a choice between the two. These are key design elements in the MathSeer interface. Appealing solutions to these issues are handwritten formula input, formula image upload, and supporting the analogy of physically moving symbols around on a page. Two challenges for text-based input are (1) most users are unfamiliar with (even fewer know MathML), and (2) rendered formulas are shown separately from input, leading to users having difficulty locating entry errors.
Text input is used by most online math-aware systems, including DLMF, WebMIAS, Math WebSearch, Wolfram Alpha, SymboLab, SearchOnMath, and the (now-defunct) Springer Search. Many users find template editors confining, and so the text box approach is the most common, often in combination with a palette used to insert symbols and structures in the entry box. Despite this, most math-aware search engines are restricted to two forms of input: (1) (or MathML) entry in text boxes, and (2) visual template editors similar to the Microsoft Equation Editor. Are large, complex, and contain symbols that many non-experts cannot name let alone express in a query.