Specs:Action Search Dialog
- 1 Text-based Intent driven Tool (TITo)
- 1.1 Background
- 1.2 Overview
- 1.3 User Interaction
- 1.4 Action History
- 1.5 GUI Memory
- 1.6 Preferences
- 1.7 Action Search Algorithm
Text-based Intent driven Tool (TITo)
This is a first draft of specifications for an Action Search dialog box. The feature has been originally proposed by Srihari and improved from there. The current code is available in the branch "tito" on the GNOME git repository of GIMP.
Unless bugs are found, the following specifications is implemented in this branch.
GIMP provides a huge list of individual commands (actions in GTK+ terminology): generic actions (open/close/save an image...), pixel processing (GEGL operations), plugins, etc. Currently you can run any of these actions either through scripts, shortcuts or the UI if a widget has been created to run it (usually a menu item).
Advanced users want to work fast. Searching through menus can be painfully slow in many cases, in particular when you need to do it very often (users who use a lot of filters). Creating shortcuts is usually a good solution, but only for a limited set of actions, the ones you really use the most often, if not all the time. Indeed shortcuts are limited by the number of possibility in key associations, but also because you may forget the shortcuts if you don't use them often enough. By default, GIMP is provided with hundreds of actions, and you can create an infinity (some very spread plugins, the ones providing many additional filters in particular, would add dozens of them).
On the other hand, if you may forget an association of meaningless keys, you won't usually forget natural language keywords, like parts of the name of the filter, the type of the filter, what it does, etc. This feature is proposing to be able to search any action at any time by such keywords.
The "tito" feature would be named more generically: "Action Search".
The correspondent action is named "dialogs-action-search". It would pop-up a dialog box, displaying a simple text entry widget where a user can type keywords, thus automatically updating a list of proposed actions. The proposed actions can be browsed and activated either by mouse or keyboard.
Similar Feature in other Software
Blender has a similar feature called the "space-menu" (its shortcut being "Space"). It can be run from everywhere through Blender's UI and would provide a list of all available commands.
This is a very practical feature in Blender.
- The default shortcut would be '/', common shortcut for quick-search in many software (text editors, web browsers, etc.), and not assigned yet in GIMP. '/' is a 1-letter shortcut which makes it indeed a very nice default for a "quick" search.
- Note: 'ctrl-f' was also considered (being also a very common shortcut for a "search" feature in most software) but it was already used for "plug-in-repeat" (Repeat Last Filter).
- The "action-search" is also available as a menu item in "Help" > "Search and Run a Command".
The action-search dialog can be closed either by hitting Esc, or by closing the dialog with any usual way provided by the user's windows manager (usually hitting a window cross on top-right, with common Alt-F4 shortcut, or any alternative which depends on the desktop and is not controlled by GIMP).
The dialog can be called back up again at any time by the previous methods (shortcut, menu...).
Proposed Action List
- The proposed action list order is relevant. The top item is computed as being the most relevant action to the search, the second slightly less, and so on, until the last, which is still considered relevant, but less than all the others. See the section `Action Search Algorithm` for details on how relevance is computed.
- By default, the proposed action list shows only the actions available in the current context (the actions which can actually be run at this time). For instance if no file is currently opened, the "view-close" action won't appear in the list of actions because there are no views to close. Nevertheless you can set up the action-search to also display the unavailable actions (see the "Preferences" section). But even by doing so, the unavailable actions will be listed, but will be grayed and not selectable.
Single Action Item
Here is a list of values relevant to an action:
- Action Name: the action name is the actual GTK+ naming, usually derived from the English language, though it should not be considered as "Natural Language", but as "Machine Language". This information is not displayed in the list.
- Action Label: each action in the proposed action list shows a human-language label (localized to user's chosen locale). This information is the minimum available for all actions.
For instance, the label for the action named view-close is: "Close View". This label will likely be different if GIMP is localized in another language.
- Action Stock Icon: if an action has an associated stock icon, it will be displayed in small version to the left of the label.
As a special exception, a toggle action (for instance "windows-use-single-window-mode") would display a OK/NO icon to characterize its current state.
- Action Shortcut/Accelerator: if the action has a shortcut assigned, it is displayed to the right of the label, in a smaller font size, after a pipe sign. This allows a user to assimilate and learn shortcuts of actions commonly run.
- Action Description/Tooltip: if the action has a longer description, it is displayed below the Action Name, in a smaller and lighter font style.
Example: our action view-close will display like this in the proposed action list:
X Close View | Ctrl + W Close the active image view
- The proposed action list would appear as soon as you type a sign in the text entry. It is automatically updated at each change of the entered text.
When the proposed action list is updated, the top item of the list (judged the most relevant, see below) is selected by default, making it easy to immediately run it with Enter.
- The proposed action list disappears when the text entry gets empty again (by hitting backspace repetively, or selection/delete, or any other mean).
When the proposed action list disappears, any previous selection is unselected, avoiding accidental and unintuitive action run when pressing Enter on an empty keyword.
- When the text entry is empty, typing the "down arrow" would force-show the full list of all actions (this full list is different whether or not you set up to show unavailable actions) and focus and select the first item.
- When the text entry has the focus, hitting the down arrow would focus and select the top proposition (most relevant one). When a proposition is highlighted, hitting the down arrow again, would highlight the next one, and so on, until the last proposition is highlighted, at which point hitting the down arrow again won't do anything.
- When the first action is highlighted, hitting the up arrow gives focus back to the text entry.
- When an action is highlighted, other than the first one, hitting the up arrow highlights the immediately upper action.
- When an action is highlighted, hitting Enter would run this action and close the action-search dialog.
- When no action is highlighted, hitting Enter would do nothing.
- At any time, hitting Esc would close the action-search dialog without any action being run.
- Hitting any other key would give focus back to the text entry, with the pointer and text selection being the exact same as they were before the text entry lost focus. This means that you can always easily update your keywords without having to repetitively use the arrow keys, or the mouse.
- The proposed action list will show a scroll bar if the dialog is not big enough to show the whole list of proposed actions on screen. You can scroll through the whole list with your mouse with the usual usage of scroll bar interface.
- Clicking an action in the list would select this action.
- Double-clicking an action in the list would run this action and close the action-search dialog.
The feature comes alongside a separate feature: action calls are logged. Only the number of times they have been run is memorized. They are logged however the action is run, whether through menu, shortcut, script or action-search itself.
Moreover only a limited number of actions are memorized. This default of "action history length" is 100 actions. A value of 100 means that the 100 most used actions will be kept in memory (and not the last 100 run actions).
Some actions are not logged:
- "dialogs-action-search" itself to avoid circular calling;
- any action "*-menu" because they are run each time you open a menu, which is very informative;
- any action "*-popup";
- any action "context-*" because they happen too often in normal usage (select a tool, etc.);
- "plug-in-repeat", "plug-in-reshow" and any action "plug-in-recent-*" because they are only redirection of actual actions.
If you often use the search-action dialog, you will likely appreciate not having to displace and resize it each time on the screen (for instance if you want to see your image, or any other reason). For this reason, the dialog remembers where it was positioned last, as well as the width of the text entry, and the height of the proposed action list.
The preferences propose 2 preferences in "Help System" tab, "Action Search" frame:
- the "Maximum History Size" spin button (see "Action History" section), integer, default 100;
- the "Show Unavailable Actions" checkbox (see "Proposed Action List" section), default unchecked.
It also proposes a button to "Clear Action History".
Action Search Algorithm
Note 1: in the following algorithm, all text is case-insensitive, which means that searching "WoRd" and "word" would get the same result.
Note 2: depending on the show-unavailable preference value, the meaning of "any action" differs. If show unavailable is unchecked, it actually means "any action which is available".
Note 3: the entry text is stripped out of any leading and trailing spaces before searching.
(1) If the entry text has a content (a "keyword"), the algorithm shows first any action in the history, for which, in this order:
- if and only if the entry text is exactly 2 letters, the first letter of the keyword is the first letter of the first word of the Action Label; and the second letter of the keyword is the first letter of the second word of the Action Name.
Example: "gb" selects "Gaussian Blur".
- the searched keyword appears at the start of the Action Name;
Example: "gauss" selects "Gaussian Blur".
- the searched keyword appears anywhere else in the Action Name;
Example: "lur" selects "Gaussian Blur"
- if and only if the entry text is more than 2 letters, the search keyword appears anywhere in the Action Description.
Example: "averag" displays "Gaussian Blur" (because its tooltip is: "Performs an averaging of neighboring pixels with the normal distribution as weighting").
Note: when several results from the history are in the same "section", their memorized number of calls is further used to weight the upper action. For instance the search "blur" would select both "Gaussian Blur", "Selective Gaussian Blur" and "Tileable Blur..." in the same section "the searched keyword appears anywhere else in the Action Name". Nevertheless if the history recorded that "Selective Gaussian Blur" had been run 20 times whereas "Tileable Blur" had been run only 10 times and "Gaussian Blur" 5 times, then "Selective Gaussian Blur" would be on top, followed by "Tileable Blur", and finally "Gaussian Blur". Additionally if 2 actions have the exact same number of calls, the last one called is considered more relevant.
(2) If the entry text has a content (a "keyword"), after any action from (1) has been shown, the algorithm shows any action *not* in the history, for which, in this order:
- if and only if the entry text is exactly 2 letters, the first letter of the keyword is the first letter of the first word of the Action Name; and the second letter of the keyword is the first letter of the second word of the Action Name.
- the search keyword appears at the start of the Action Name;
- the searched keyword appears anywhere else in the Action Name;
- if and only if the entry text is more than 2 letters, the search keyword appears anywhere in the Action Description;
(3) If the entry text has a content (a "keyword"), after any action from (1) and (2) has been shown, the algorithm shows any action (in history or not), for which:
- individual letters in the search keywords appear in the same order, but with optional intermediate letters, in the Action Name.
This is called fuzzy search. It allows powerful research when you search some action whose name is well known. For instance if you were to convert your image to 16 bit floating point (gamma), you could search '1bfpg + Enter' which would return only this response, because you know exactly what you search, and is a much faster process. Or searching "cdrgb" would return "Convert to Default RGB" as only response.
But it may also display totally irrelevant responses on common searches. This is why its result are always at the end of the list. For instance, when you are searching for color balance actions, you may begin to type "bala", and you certainly don't want to get "Set Brush Size to Default Value" being considered more relevant than "White Balance" or "Color Balance" actions.