OCR

The OCR action A feature that supplies the instructions in a botflow. in a botflow A file that stores the actions and variables designed to automate tasks. takes a picture of a target and extracts the text from the image using Optical Character Recognition (OCR) technology.

Note: Use OCR when the Get Value action can't extract text from a target.

Using the OCR action, you can:

  • Get text from a PDF.
  • Copy text from a text editor.
  • Copy values from online forms.

Some examples of target applications for the OCR action using the Selector tool are:

  • PDFs.
  • Nintex Forms.
  • Browser windows.

You can add an OCR action to a botflow using the:

Add an OCR action to a botflow using the Project Pane

This example provides the steps necessary to add an OCR action to a botflow using the Project Pane.

To add an OCR action to a botflow.

  1. On the Actions list, click Images and then click OCR from the Images list.

    2. The OCR Action Builder window displays.

  2. Set the OCR settings.
    • Preview: Provides a preview of the target.
    • Color/B&W (Slide bar): The Color/B&W toggle switches the Preview between color and black-and-white. Use the slide bar to adjust black-and-white contrast.
    • Left, Top, Width, Height: Enter a numeric value to adjust the OCR capture box area.
    • Preview: Click Refresh to refresh the Preview and populate the Preview Field.
    • Save to: Type the Save to destination.
  1. If needed, adjust the Rule settings.
    • Image Editor: Select to use the OCR action with the Image Editor.
    • Image File: Select to use the OCR action with an image file of your choosing.
  2. If needed, adjust the Run Error Task settings and Timing settings to fine tune how the action runs.
  3. View the action Detailssettings.
  4. If needed, adjust the Run Switch setting to create the action without running it.
  5. Optionally, add an action note.
  6. Click OK to add the OCR action to the botflow.

Add an OCR action to a botflow using the Selector Tool

This example provides the steps necessary to add an OCR action to a botflow using the Selector Tool.

To add an OCR action to a botflow.

  1. Drag the Selector tool from the Build Pane Area to add new actions and edit existing actions. and drop it onto an application to display the Target Preview.
  2. Select OCR from the Images actions list.

    3. The OCR Action Builder window displays.

  3. Set the OCR settings.
    • Preview: Provides a preview of the target.
    • Color/B&W (Slide bar): The Color/B&W toggle switches the Preview between color and black-and-white. Use the slide bar to adjust black-and-white contrast.
    • Left, Top, Width, Height: Enter a numeric value to adjust the OCR capture box area.
    • Preview: Click Refresh to refresh the Preview and populate the Preview Field.
    • Save to: Type the Save to destination.
  1. If needed, adjust the Rule settings to modify how Nintex RPA finds the application target.
  2. If needed, adjust the Run Error Task settings and Timing settings to fine tune how the action runs.
  3. View the action Details settings.
  4. If needed, adjust the Run Switch setting to create the action without running it.
  5. Optionally, add an action note.
  6. Click OK to add the OCR action to the botflow.

Black & White Mode

Use Black & White Mode when detecting control values such as a check box check or a radio button. Different colored pixels make up each of these controls and can change with different versions of selected Windows themes.

 Black & White example

Color Mode

Use Color Mode when an exact pixel color is needed to make a specific determination such as determining if a target is enabled or disabled.

Color example

Toolbar button enabled (color) versus disabled

         

OCR action field and button descriptions

Field or button Description
Selector Tool

The Selector Tool allows you to choose the target element the botflow interacts with, such as a button, text field or check box.

Ensure your machine's Windows Settings (Start Menu > Settings) are set as follows when using the Selector Tool:

  • Windows Display Scale and layout set to 100% (Settings->System->Display).
  • Windows Display Make text bigger set to 100% (Settings->Ease of Access->Display).

You can use the Selector tool to do things such as:

  • Click a button in an application.
  • Send information to a web page or Windows application.
  • Copy information from a web or Windows application.

Drag the Selector tool from the Task Pane Area to create and manage actions within a specific task. onto the desired application target An element within an application Nintex RPA interacts with, such as a button, text field, or checkbox. and choose the appropriate action from the Build Pane Area to add new actions and edit existing actions..

Selector Tool
Build Pane

The Build Pane displays after you select a targeted element in a Windows application.

The Build Pane lets you:

Selected target elements display in the Build Pane with a list of actions supported by the targeted item. The Build Pane also displays how long the item has been targeted.
Recent Target

The most recent target may be accessed by clicking the Recent Target button to the right of the Selector.

This is useful when creating Excel, Emulator, and Web Table Targets allowing you to create new actions with existing targets.
Field or button Description
OCR Options/Settings General

  • Preview: Provides a preview of the target.
  • Color/B&W (Slide bar): The Color/B&W toggle switches the Preview between color and black-and-white. Use the slide bar to adjust black-and-white contrast using variables or other token values.
  • Left, Top, Width, Height: Enter a numeric value to adjust the OCR capture box area, use the up/down arrows to increase/decrease the numeric value, or use the Expression Builder to build your action rules using variables or other token values.

Tip: Click the capture box in Preview to manually move or reset the capture area.

Result

  • Preview: Click Refresh to refresh the Preview and populate the Preview Field.
  • Save to: Type the Save to destination or use the Expression Builder to build the action settings using variables or other token values.
Rules (Project Pane)

Image Editor: Select to use the OCR action with the Image Editor. Be sure to have an image open in the Image Editor.

Image File: Select to use the OCR action with an image file of your choosing. Type in the file path to the image, click the Folder icon to select the image, or use the Expression Builder to build your action rules using variables or other token values.
Rules (Selector Tool)

Some actions have rules required to locate a target and vary based on the targeted application.

Rules of a targeted application element are configured and saved with the action. The rules are then used to locate the proper targeted element, such as an Microsoft Excel workbook.

To set the most common rules type:

  1. Click Rules from the Build Pane Area to add new actions and edit existing actions. or the Action Builder The Action Builder window provides access to the tools needed to create and modify Actions or instructions within a botflow. window.

    2. The Rules window displays the Rules available for the selected targeted application.

  2. Adjust the available rules as required.
  1. Click OK to save the Rules.
  • Some targets provide no rules.
  • Most targets provide at least the text of the target, such as 'Log in' for a log in screen action.
  • Some targets provide a name to identify the target (a unique developer-defined control name).
  • Some targets provide an ID (a unique identifier that locates an item based on how it appears in the target).
  • All targets use the following rules:
    • Ignore: Ignores the value of the selected rule.
    • Exact Match: Uses an exact match when considering the value of the rule.
    • Best Match: Uses a best match when considering the value of the rule and uses a variety of information to locate the target.
    • Left Match: Uses leading characters when considering the value of the rule based on a specified number of leading characters.
    • Right Match: Uses trailing characters when considering the value of the rule based on a specified number of trailing characters.
Preview All actions created using the Selector Tool include a Preview of the targeted element.
Details

When directed by Nintex Support, click Details from the Preview Panel of the Action Builder. If needed, Nintex Support will supply additional information.
Timing Settings

Some actions have timing settings and vary based on the targeted application. The timing settings of a targeted application element are configured and saved with the action. The timing settings are then used to control when the action starts.

  1. Click Wait up to 1 sec.
  1. Open the Time drop-down list and select the required setting:
    • Do not wait: Immediately attempts to execute the action and applies any error settings or rules.
    • Wait up to: Waits up to one (1) second to execute the action per any configured rules. The time specified is the wait time prior to continuing the botflow. In the event of an application being unavailable, the action adheres to the error settings. Time units include numeric settings for seconds, minutes, and hours.
    • Wait forever: Waits until the target application is in a ready state and applies any error settings or rules.
  1. Click X to close the drop-down list and save the setting.
Run error task
  1. Click Run error task.
  2. Open the On Error drop-down list and select the required setting:
    • Run error task: A botflow containing an error task automatically runs the error task in the event of an error. If the botflow does not contain an error task, the botflow stops in the event of an error.
    • Stop on error: The botflow stops in the event of an error.
    • Ignore errors: The action is skipped in the event of an error and the botflow attempts to execute the next action
  1. Click X to close the drop-down and save the setting.
Note

(Optional) Type a Note to document any special instructions or functionality.    
Run Switch

The Run Switch toggle controls how Nintex RPA runs an action when adding or editing an action in a botflow.

  • Run: To run the action automatically when you click OK, turn the toggle to Run.

  • Don't Run: If you do not want to run the action automatically when you click OK, turn the toggle to Don't Run.
OK/Cancel Click OK to save the action or click Cancel to discard the action or any changes.