Mindstorm:n-point image deformation tool

From GIMP Developer Wiki
Revision as of 07:45, 15 July 2013 by Korek (Talk | contribs)

Jump to: navigation, search


The deformation method uses two meshes. One is called 'reference pose' mesh, the other is called 'current pose' mesh. Reference pose mesh is always hidden and current pose mesh can be visible. Only current pose mesh is deformed.

Mesh (lattice) for this method usually consists of squares. The deformation method is not limited to square mesh but square mesh can be constructed in an easier way, usually with less elements (than triangle mesh), and with small tweaks we can achieve similar precision as with triangular mesh.

For this method it holds that with decreasing lattice square size (i.e. increasing number of mesh elements), precision of result is increasing, but inefficiency of the algorithm is increasing as well. User should be able to adjust the balance.

Lattice with control points.png

At the beginning, user selects n-point deformation tool. When there is a selected region, deformation is performed only on it.

Mesh above the image is created automatically with use of alpha channel. When there isn't any alpha channel (or selection), then mesh is created above the whole image (or selection).

Control points manipulation

Add control points

With a click at some position on image, control point is added. User can add control points at arbitrary positions on image (except position [or some radius around it which depends on mesh square size] where a point is already placed) – mesh will be adjusted according to that.

Selection, deselection, multi-selection

More or less the same as sections with same names in Cage tool specification.

Delete control points

Selected control points are deleted by <delete> key. Last added point can be deleted by <backspace> key.

Deformation of the image

When user moves (click – drag – release) a control point (or multi-selected control points), the image is deformed.

Depth of control point

There should be an option to adjust which part of overlapping mesh/image is visible – e.g. we have an image of a person (as in the picture below) and we want to move her arm so that it is in front of (or behind) her body.

Depth of control point.png

Every control point has its depth which affects a part of mesh under the control point.

User can change the depth using two buttons – one increases the depth, the other decreases it – or using two keys - which?.

When more control points are selected, every one's depth is increased/decreased.

Mesh manipulation

Mesh visibility

In some situations it's useful to see how the mesh looks like. User can toggle mesh visibility using a check button (or switch).

Mesh element size

User can choose the size of mesh element using a slider – but only before he starts adding control points. It is because of the fact that when changing 'reference pose' mesh square size, it's often impossible to recreate 'current pose' mesh so that it creates the same deformation as previous 'current pose' mesh. Note that this option changes both 'reference pose' and 'current pose' mesh element size.

What would be a suitable name for this option? Density or precision?

Deformation control

Deformation algorithm control

The deformation algorithm iteratively solves a least squares problem described in the paper. During that process we periodically display current state of the solution and user can see that the deformation is evolving somehow. Usually hundreds of iterations are performed during 1 FPS. We don't know how much time the solving will take and at the same time we often don't want to find the final solution – we're interested just in the process.

Hence there should be an option to temporary stop the process at some state or let it continue with solving further. How to do that in the best way? It could be done for example by <space> key.

Deformation termination

When user is satisfied with result, he presses <enter> key. Deformed image is redrawn using selected interpolation method. The deformation ends.

When user wants to cancel the deformation, he presses <escape> key. Deformed image is discarded, original image appears on the canvas. The deformation ends.




Deformation mode

The method allows 3 types of deformation:

  • rigid (known as as-rigid-as-possible)
  • similar (known as as-similar-as-possible)
  • stretch (ARAP combined with weights from Moving Least Squares method)

'Rigid' type is very good for bending objects or manipulating articulated objects. It's not so good in stretching or shrinking of objects. 'Similar' type allows changing of scale. Only translation, rotation and change of scale is included in deformation. It can be seen in the example. 'Stretch' type allows better stretching of objects. It can be seen in the example.

User can select the deformation type using drop down list or radio buttons.

Number of deformation iterations

User can select using slider, the number of iterations that are performed during 1 FPS. The larger the number the more rigid the deformation behaves.

How to call this option? It can be called for example rigidity or flexibility.

Type of interpolation

User can select the type of interpolation which is used when the deformation is terminated in favor of deformed image.

Types of interpolation:

  • nearest neighbor (none)
  • linear
  • cubic
  • sinc