Heindl Solutions - Using Basler cameras with HALCON

Introduction and hardware setup

Basler cameras and MVTec HALCON make for a good combination for industrial machine vision applications.

Imagine the following hardware setup: Two cameras, one on each side of a conveyor belt. There are some objects which have to be inspected from the left and from the right side simultaneously. The inspection is triggered by a GPIO of each of the cameras:

Let's start with a single camera first. This task should be straightforward to complete:

acquire_basler_00.hdev

* acquire_basler_00.hdev

dev_update_off ()
dev_open_window (0, 0, 512, 512, 'black', WindowHandleLeft)

InterfaceName := 'GigEVision2' 1
Device := 'default' 2
open_framegrabber (InterfaceName, 0, 0, 0, 0, 0, 0, 'default', -1, 'default', -1, 'false', 'default', Device, 0, -1, AcqHandleLeft)

while (true)
    grab_image (ImageLeft, AcqHandleLeft) ⚡ 3
    dev_display (ImageLeft)
    ProcessLeft (ImageLeft, ResultLeft)
    dev_disp_text ('Left: ' + ResultLeft, 'window', 'top', 'left', 'black', [], [])
    wait_seconds (0.200)
endwhile

close_framegrabber (AcqHandleLeft)

Patience you must have, my young Padawan. This program works, but can be improved:

It is often a good idea to separate some arguments into local variables. This way a future reader of the script code will have an easier time in parsing all the 'default' arguments of open_framegrabber. The separate definition of InterfaceName and Device makes the code a bit more readable. Nevertheless, this has a major drawback in this special case: If the interface driver name is not written directly as string argument for open_framegrabber HDevelop will not be able to provide helpful suggestions for exactly the GigEVision2 interface and a keypress of F1 will not show the specific documentation for the GigEVision2 interface. We will thus move the InterfaceName directly into the call to open_framegrabber.

'default' works as long as there is only a single camera attached to the interface. But in general it is a bad idea to count on this simplification: Using 'default', the script depends on the fact that no other device can be found, but this will break as soon as a further camera becomes somehow visible to this machine.

It seems a naive usage of grab_image is almost never a good idea unless you do not care about performance at all. We will switch to grab_image_async later.

Once we change

open_framegrabber (InterfaceName, …)

open_framegrabber ('GigEVision2', …)

we can use HDevelop's autocompletion to specifically select a certain camera. Autocompleting single arguments in a complex operator call like open_framegrabber works best in the Operator Window. This window was visible by default up to HDevelop 20.05 but nowadays this window has to be shown manually using the menu entry Window ➡ Open Qperator Window. We can now change the Device:

* acquire_basler_01.hdev
⋮
open_framegrabber ('GigEVision2', 0, 0, 0, 0, 0, 0, 'default', -1, 'default', -1, 'false', 'default', \
                   ' | device:00306347d7ab_Basler_acA160020gc | unique_name:00306347d7ab_Basler_acA160020gc | interface:Esen_ITF_80fa5b400f14d0a7003effffff00 | producer:Esen', \
                   0, -1, AcqHandleLeft)
⋮

We obviously didn't want to write the device name by hand here!

An appropriate next step would be to set a custom name for the camera. This can be done using Basler's "pylon Viewer". Go to pylon Viewer ➡ menu Tools ➡ pylon IP Configurator and select your camera. Then enter a Device User ID and click on Save. Make sure to close the camera in pylon Viewer when switching back to HDevelop.

As an alternative, the device name can be set directly in HDevelop calling once:

set_framegrabber_param (AcqHandle, 'DeviceUserID', 'GigE_Cam1').

Now autosuggestion for the device name suggests:

open_framegrabber ('GigEVision2', 0, 0, 0, 0, 0, 0, 'default', -1, 'default', -1, 'false', 'default', \
                   'GigE_Cam1', 0, -1, AcqHandleLeft)

Sometimes a more complicated device name will be suggested like

' | device:GigE_Cam1 | unique_name:00306347d7ab_Basler_acA160020gc | user_name:GigE_Cam1 | interface:Esen_ITF_80fa5b400f14d0a7003effffff00 | producer:Esen'

which we can abbreviate manually to 'GigE_Cam1'.

Alternative acquisition interfaces

We do not use the 'pylon' interface. It is outdated. The latest version is "pylon Interface for Basler Cameras Rev. 13.0.4 (x64-win64) (Release date: 2019-08-26) ".

Also remember to use 'GigEVision2': 'GigEVision' is deprecated/outdated as well.

If 'GigEVision2' is not available in your HALCON installation, you can download it from MVTec. Don't be surprised if the version is 20.11.xx: This version denotes the lowest HALCON version still supported. So for example an interface 20.11.xx is perfectly fine for HALCON version 22.11.

Setting basic camera parameters

Until now we did not specify any settings for the camera. In this case the camera will return results according to (we have to assume to some degree) random internal settings of the camera. This is of course unreliable and we might want to set parameters explicitly on our own:

* acquire_basler_01.hdev

dev_update_off ()
dev_open_window (0, 0, 512, 512, 'black', WindowHandleLeft)

open_framegrabber ('GigEVision2', 0, 0, 0, 0, 0, 0, 'default', -1, 'default', -1, 'false', 'default', \
                   'GigE_Cam1', 0, -1, AcqHandleLeft)
set_framegrabber_param (AcqHandleLeft, 'ExposureTimeAbs', 50000) // microseconds

while (true)
    grab_image (ImageLeft, AcqHandleLeft) ⚡   // better use grab_image_async, see comment in acquire_basler_00.hdev
    dev_display (ImageLeft)
    ProcessLeft (ImageLeft, ResultLeft)
    dev_disp_text ('Left: ' + ResultLeft, 'window', 'top', 'left', 'black', [], [])
endwhile

close_framegrabber (AcqHandleLeft)

This might work for now and the resulting acquired images are good:

Until someday the algorithm fails and the investigation shows that the acquired image now looks like:

An acquired image showing the letter A very faintly

What happened? By setting only certain parameters (like ExposureTimeAbs in this example) there will be lots of other parameters persisted on the camera until the next power cycle. There might be another script (a legacy or test implementation) still laying around on this machine and this will sooner or later be executed by someone: This can very well change some persisted parameters on the camera! Or settings are changed directly in Pylon Viewer! Thus suddenly the Gain (for example) will be at some other random value. To make our application more robust, we recommend to always reset the camera to factory defaults directly after opening and then change the appropriate settings manually (Software Persistence):

* acquire_basler_02.hdev

dev_update_off ()
dev_open_window (0, 0, 512, 512, 'black', WindowHandleLeft)

open_framegrabber ('GigEVision2', 0, 0, 0, 0, 0, 0, 'default', -1, 'default', -1, 'false', 'default', \
                   'GigE_Cam1', 0, -1, AcqHandleLeft)

* reset the camera to factory settings
set_framegrabber_param (AcqHandleLeft, 'UserSetSelector', 'Default')
set_framegrabber_param (AcqHandleLeft, 'UserSetLoad','')

* set specific parameters needed for our application
set_framegrabber_param (AcqHandleLeft, 'ExposureTimeAbs', 50000) // microseconds

while (true)
    grab_image (ImageLeft, AcqHandleLeft) ⚡    // better use grab_image_async
    dev_display (ImageLeft)
    ProcessLeft (ImageLeft, ResultLeft)
    dev_disp_text ('Left: ' + ResultLeft, 'window', 'top', 'left', 'black', [], [])
    wait_seconds (0.200)
endwhile

close_framegrabber (AcqHandleLeft)

We do not like alternatives like persisting settings in User Sets on the camera that much: This hides essential knowledge from the script developer, makes version control harder and causes problems if a camera has to be exchanged.

Note: It seems that a 'UserSetLoad' does not reset all parameters. At least for our camera, after a power cycle the default for 'AcquisitionMode' is set to 'Continuous', but this won't be reset with a call to 'UserSetLoad'!

Enabling software triggering

Let's have a look at the timing so far:

acquire_basler_timing.hdev

⋮
while (true)
    * Assume trigger signal here
    
    count_seconds (TriggerTime)
    grab_image (ImageLeft, AcqHandleLeft)
    count_seconds (Now)
    DurationMillis := 1000.0 * (Now - TriggerTime)
    dev_display (ImageLeft)
    dev_disp_text ('Duration: ' + DurationMillis + ' ms', 'window', 'bottom', 'left', 'black', [], [])
⋮
endwhile

close_framegrabber (AcqHandleLeft)

With my test setup the duration between enter and finish of grab_image is approximately 330 milliseconds. Using Data Chunks we can see that multiple images were acquired internally by the camera. In my case the 'ChunkFramecounter' returns 3 most of the time. This is caused by the setting 'AcquisitionMode' which is 'Continuous' by default.

One could change the acquisition mode from 'Continuous' to 'SingleFrame':

set_framegrabber_param (AcqHandle, 'AcquisitionMode', 'SingleFrame').

but this still leaves as with an execution time of 300 milliseconds for the grab_image operator call.

To fix this and to decrease the time that we have to wait for new images to arrive and in anticipation that we want to add a 2nd camera later grab_image should be replaced by a combination of grab_image_start / grab_image_async. This way we have a chance to execute two acquisitions simultaneously in a later step: acquire_basler_async1.hdev

⋮
grab_image_start (AcqHandleLeft, -1)
while (true)
    * Assume trigger signal here
    
    count_seconds (TriggerTime)
    * ATTENTION: Do not use! Returns an outdated image!
    grab_image_async (ImageLeft, AcqHandleLeft, -1) ⚡
    count_seconds (Now)
    DurationMillis := 1000.0 * (Now - TriggerTime)
    dev_display (ImageLeft)
    dev_disp_text ('Duration: ' + DurationMillis + ' ms', 'window', 'bottom', 'left', 'black', [], [])
⋮
endwhile

close_framegrabber (AcqHandleLeft)

In using grab_image_async this way, we will receive the previously taken image (in most cases)! Of course for our application, we need to take an image of the current scene as fast as possible as soon as our software event triggers the acquisition part of the script!

We need to enable soft triggering:
acquire_basler_with_softtrigger.hdev

⋮
* set specific parameters needed for our application
set_framegrabber_param (AcqHandleLeft, 'ExposureTimeAbs', 50000) // microseconds

* enable software triggering
set_framegrabber_param (AcqHandleLeft, 'TriggerMode', 'On') 1
set_framegrabber_param (AcqHandleLeft, 'TriggerSelector', 'FrameStart')
set_framegrabber_param (AcqHandleLeft, 'TriggerSource', 'Software')
dev_get_system ('engine_environment', EngineEnvironment) 2
if (EngineEnvironment == 'HDevelop')
    * during development an infinite timeout would be
    * very inconvenient
    set_framegrabber_param (AcqHandleLeft, 'grab_timeout', 1000) // milliseconds
endif

grab_image_start (AcqHandleLeft, -1)
while (true)
    * Assume trigger signal here
    
    count_seconds (TriggerTime)
    set_framegrabber_param (AcqHandleLeft, 'TriggerSoftware', '') 3
    grab_image_async (ImageLeft, AcqHandleLeft, -1)
    count_seconds (Now)
    DurationMillis := 1000.0 * (Now - TriggerTime)
    dev_display (ImageLeft)
    dev_disp_text ('Duration: ' + DurationMillis + ' ms', 'window', 'bottom', 'left', 'black', [], [])
⋮
endwhile

close_framegrabber (AcqHandleLeft)

Now the time between a trigger signal and the finish of the acquisition decreases to approximately 100 milliseconds which seems to stem from 50 milliseconds exposure time and some 50 milliseconds other overhead (maybe data transfer).

Enable soft trigger mode.

It might happen that no trigger is occurring during a long time, especially during development. In order to be able to continue development in this case, a grab timeout of 1000 milliseconds is enabled. After this time, grab_image_async will return with an error. This is only activated during a development session in HDevelop.

Execute a software trigger! The value '' is ignored.

For grab_image_async it is not important which of the following two acquisition modes is active:

set_framegrabber_param (AcqHandle, 'AcquisitionMode', 'SingleFrame')

set_framegrabber_param (AcqHandle, 'AcquisitionMode', 'Continuous')

Switching to USB3 cameras

Beware of some potential issues if instead of GigEVision you want to utilize USB3 cameras. Probably PylonViewer and HALCON are installed on the same machine. PylonViewer installs its own driver which appears as USB Composite Device / Basler ace USB3 Vision Camera in Windows' Device Manager. This way, the camera can be used within Pylon Viewer.

We want to use the camera from HALCON as well. If you start up HDevelop's Image Acquisition Assistant it will detect 'USB3Vision' devices and give you a warning like Screenshot of the assistant warning

We do NOT recommend to follow this suggestion! This would install the HALCON USB3Vision driver. Afterwards HDevelop/HALCON would be able to use the camera (good), but PylonViewer would be no more (bad).

The solution is to use the 'GenICamTL' acquisition interface instead! Now HALCON as well as the PylonSDK can use the camera.

Install the Pylon Driver

If you want to use the Pylon Driver but the libusbx driver is already installed (Windows):

Device Manager: uninstall libusbx driver (and remove driver!)
unplug+reconnect device
install Pylon Driver for "USB3 Vision Device" from C:\Program Files\Basler\pylon 6\Applications\Win32\share\Tools\x64
driver will be shown in Device Manager as USB Hub / USB Composite Device / Basler ace USB3 Vision Camera

Afterwards you can use

open_framegrabber ('GenICamTL', …)

Install the libusbx driver (not recommended)

If you really want to use the libusbx driver but the Pylon Driver is already installed (Windows) you can do the following:

see also section "USB3 Vision Device Access" at https://www.mvtec.com/products/interfaces/documentation/view/1309-standard-13-mvtecdoc-usb3vision/#DEVICE_ACCESS
install libusbx driver:
uninstall existing driver (and remove driver!). Also remove driver for the parent "USB Composite device "!
unplug+reconnect device
open HDevelop Image Acquisition Assistant, it will automatically prompt you to add the correct generic parameter to install the driver if necessary
unplug+reconnect device

Afterwards you can use

open_framegrabber ('USB3Vision', …)

Two cameras

acquire_basler_2cam.hdev

dev_update_off ()
open_framegrabber ('GenICamTL', 0, 0, 0, 0, 0, 0, 'default', -1, 'default', -1, 'false', 'default', 'BaslerUSB3A', 0, -1, AcqHandleLeft)
open_framegrabber ('GenICamTL', 0, 0, 0, 0, 0, 0, 'default', -1, 'default', -1, 'false', 'default', 'Basler_USB3_AceU', 0, -1, AcqHandleRight)
AcqHandles := [AcqHandleLeft, AcqHandleRight]

* setup all open cameras
for IndexCam0 := 0 to |AcqHandles|-1 by 1 1
    AcqH := AcqHandles[IndexCam0]
    
    dev_open_window (0, IndexCam0 * 520, 512, 512, 'black', WindowHandle1) 2
    WindowHandles[IndexCam0] := WindowHandle1
    
    * reset the camera to factory settings
    set_framegrabber_param (AcqH, 'UserSetSelector', 'Default')
    set_framegrabber_param (AcqH, 'UserSetLoad','')
    
    * set specific parameters needed for our application
    set_framegrabber_param (AcqH, 'ExposureTime', 50000) // microseconds 3
    
    * enable software triggering
    set_framegrabber_param (AcqH, 'TriggerMode', 'On')
    set_framegrabber_param (AcqH, 'TriggerSelector', 'FrameStart')
    set_framegrabber_param (AcqH, 'TriggerSource', 'Software')
    dev_get_system ('engine_environment', EngineEnvironment)
    if (EngineEnvironment == 'HDevelop')
        * during development an infinite timeout would be
        * very inconvenient
        set_framegrabber_param (AcqH, 'grab_timeout', 1000) // milliseconds
    endif
    
    grab_image_start (AcqH, -1)
endfor

while (true)
    * Assume trigger signal here

    count_seconds (TriggerTime)
    for IndexCam0 := 0 to |AcqHandles|-1 by 1 4
        AcqH := AcqHandles[IndexCam0]
        set_framegrabber_param (AcqH, 'TriggerSoftware', '')
    endfor
    grab_image_async (ImageLeft, AcqHandleLeft, -1)
    grab_image_async (ImageRight, AcqHandleRight, -1)
    count_seconds (Now)
    DurationMillis := 1000.0 * (Now - TriggerTime)


…
    dev_set_window (WindowHandles[0])
    dev_display (ImageLeft)
    dev_set_window (WindowHandles[1])
    dev_display (ImageRight)
    
    dev_set_window (WindowHandles[0])
    dev_disp_text ('Duration: ' + DurationMillis + ' ms', 'window', 'bottom', 'left', 'black', [], [])
    ProcessLeft (ImageLeft, ResultLeft)
    dev_disp_text ('Left: ' + ResultLeft, 'window', 'top', 'left', 'black', [], [])

    dev_set_window (WindowHandles[1])
    dev_disp_text ('Duration: ' + DurationMillis + ' ms', 'window', 'bottom', 'left', 'black', [], [])
    ProcessRight (ImageRight, ResultRight)
    dev_disp_text ('Right: ' + ResultRight, 'window', 'top', 'left', 'black', [], [])    
    wait_seconds (0.200)

endwhile
close_framegrabber (AcqHandleLeft)

Assume that all two cameras use the same settings. If for example the exposure is different then move the corresponding setting out of the loop body.

Open a window for each camera. This will be used to display the current image per camera. The window column position will be increased for each camera.

ExposureTimeAbs has to be changed to ExposureTime for our USB3Vision cameras as ExposureTimeAbs is not supported.

All cameras are soft-triggered simultaneously.

Hardware trigger

Software triggers do not have the strict timing properties that are often required. Software triggering introduces additional processing and communication delays, leading to variable trigger-to-exposure times. This variability can result in inconsistent image acquisition timing, especially in high-speed applications where precise synchronization is crucial. Therefore after initial experiments using the software triggering one often wants to switch to hardware-triggering mechanisms. Many industrial cameras have GPIO pins which can be used for that. In our example, we use the pin 'Line1' to trigger acquisitions:

acquire_basler_hardwaretrigger.hdev

⋮
* enable H/W trigger on Line1 input
set_framegrabber_param (AcqHandle, 'TriggerMode', 'On')
set_framegrabber_param (AcqHandle, 'LineSelector', 'Line1')
set_framegrabber_param (AcqHandle, 'TriggerActivation', 'FallingEdge')
set_framegrabber_param (AcqHandle, 'grab_timeout', 1000) // milliseconds
⋮

Downloads

Summary

We have shown various typical tasks and sometimes necessary hacks for using Basler GigEVision and Basler USB3 cameras in HALCON.

Please feel free to contact us if you have any questions or need help with your project.

Using Basler cameras with HALCON - Best practices

2024-06-01 by Andreas Heindl