What is Magic Shape?

Magic Shape is the final project for my Human Computer Interaction class at Baylor. It is a solid geometry edutainment application for children using tangible augmented reality.

This article describes the development of the Magic Shape prototype. The purpose of Magic Shape is to teach the following concepts:

  • The recognition of basic 3D shapes such as cube, cylinder, sphere, cone and pyramid.
  • The number of faces of basic 3D shapes.
  • The number of edges of basic 3D shapes.
  • Volume and surface area of basic 3D shapes.

There are three modes for Magic Shape

  • Practice Mode
  • Testing Mode
  • Game Mode

AR Interface

The interaction interface is a combination of AR and TUI. Using AR, the system can render different types of 3D shapes, such as a cube, cylinder or sphere. All they need is a piece of paper. Tangible cubes are also used in the system. The reason for that is we want children to learn about the concepts of volume and surface. By using these physical cubes, children can also build their own 3D shapes.

In the initial prototype, there are 8 tangible objects being tracked as shown in the following figure.

  • A cube: view a 3D cube
  • A cylinder: view a 3D cylinder
  • Scissors: cut the 3D shape into 2D diagram
  • A wireframe box: display the wireframe
  • A green pen: change the shape color to green
  • A purple pen: change the shape color to purple • A zoom-in tool: make the 3D shape bigger
  • A zoom-out tool: make the 3D shape smaller

All of these objects are attached with fiducial markers. The markers are default markers from the ALVAR tracking library and are rated highly for tracking performance.

Tangible User Interface

Magic Shape has a set of physical cubes as shown in the following figure.


With these physical cubes, children can build and create any shape they want and these new shapes can be recognized by the system in real-time. Also Magic Shape can calculate the volume and the surface area of these new shapes and display them to the children. By default, each physical cube is considered one unit cubed. So the volume simply equals to the number of cubes. On the other hand, calculating the surface area of the shapes is much harder. In our prototype, we use augmented reality and computer vision technology to support the tangible user interface. So the system only knows whether the cube is present or not and it has no idea whether the shape is connected or not. To make things simple, the system assumes that each shape is connected and only one side of a cube is connected to another one. In this case, the system computes the surface area using the following formula:
Number of Cubes × 6 − 2 × Number of Connected Sides

Practice Mode

In practice mode, the system can render five types of shapes: cube, cylinder, cone, pyramid and sphere. Each of these shapes can be manipulated using the following operations:

  • Rotate: children can rotate the 3D shape by rotating the marker.
  • Move: children can move the 3D shape by moving the marker.
  • Scale: children can make the shape bigger by showing the zoom in marker and make the shape smaller by showing the zoom out marker.
  • Change color: children can change the color of the shape by showing the different color pen marker.
  • Cut: children can cut the 3D shape into a 2D diagram by showing the scissors marker. It allowschildren to count the number of faces
  • Wireframe: children can see the wireframe of the 3D shape by showing the wireframe marker. It allows children to count the number of edges

Children can also explore the volume and surface area of different shapes they build in this mode. Magic Shape will give real-time feedback to whatever shapes children build in the system. In this way, children can observe the impact of changing a shape on its volume and surface and it also reinforces the relationships of the number of blocks to volume and the number of visible sides to surface area.

Testing Mode

In testing mode, we use augmented paper to present questions and children are asked to answer these questions accordingly. Basically, this is an Q/A interface and Magic Shape will tell children whether they answer correctly or not. For the initial prototype, there are two kinds of questions in this mode. One scenario is that children are asked to build any shapes that match the given volume or surface area. Another scenarios is that the system lets children build a new shape using the tangible cubes and then asks them to calculate the volume and surface area of the new shape. One example of the first scenario is presented in the following figure.

Game Mode

In game mode, children can play an augmented reality marble game. But before that, they have to build a maze and obstacles using the shapes that they have learned in Magic Shape. They can freely build a combination of shapes to form a maze and obstacles and then play the AR marble game in this maze they build. In the game, children must guide a virtual ball through the maze and obstacles that they build on the board by tilting and translating the board.

Implementation

Magic Shape is implemented using GoblinXNA, which is a platform for research on 3D user interfaces, including augmented reality and virtual reality. Marker tracking is done using ALVAR tracking library developed by VTT technical Research Center. The prototype runs on a thinkpad laptop with Windows7 operating system and it is equipped with a Logitech Pro 9000 webcam. During all modes of the system, Magic Shape runs at approximately 60 fps.