Hand segmentation in thermal spectrum


Ing. Jiří Mekyska, prof. Ing. Zdeněk Smékal, CSc.


It is possible to download software here.

Publication to be cited

MEKYSKA, J.; FONT-ARAGONES, X.; FAÚNDEZ ZANUY, M.; HERNÁNDEZ-MINGORANCE, R.; MORALES, A.; FERRER-BALLESTER, M. Thermal hand image segmentation for biometric recognition. In Proceedings of 45th annual IEEE International Carnahan Conference on Security Technology. 2011. pp. 26-30. ISBN: 978-1-4577-0901-2.


This toolbox contains functions that can be used for the hand segmentation in thermal spectrum. There were proposed two basic methods of hand segmentation in TH spectrum. Both are dependent on the presence of the image in VIS spectrum. Moreover M1 takes into account also the empirically found settings of segmentation function. Methods are sufficiently described below.

Method based on the empirically found settings (M1)

As was already said, this method can be used only when the hand in VIS spectrum is also used. In our case all hands were acquired by the same camera which simultaneously stores both, VIS and TH images. Therefore it is possible to empirically find the relation between these two images, i.e. to find the mutual translation, rotation and scale. According to an analysis of big amount of images these three parameters were found and consequently the hands in TH spectrum were segmented in these steps:

  1. Firstly the VIS image was binarized using empirically found threshold (EF), using Otsu’s method or using the K-means with two clusters (KM), while the codebook was created according to CMYK and LAB color models. In case of K-means and Otsu’s method (OM), the images were consequently filtered by median filter using the 3-by-3 neighborhood.
  2. In the second step the VIS binary mask was transformed to the TH binary mask using rotation, translation and scaling. The input parameters of these transformations were found empirically.
  3. Using the TH binary mask, the hand was easily segmented.

The disadvantage of this method is a dependence on the presence of VIS image and also the use of empirically found parameters.

Method based on the image registration (M2)

This method is more universal than M1, because it does not need any empirically found parameters. On the other hand it is not so accurate. Method based on an image registration still considers the presence of VIS image. The procedure of segmentation can be described in these steps:

  1. This step is same as the first step in M1 (i.e. the VIS binary mask is created).
  2. In the second step, the VIS binary mask is registered with hand in TH spectrum. To find the best settings of rotation, translation and scaling the optimization function based on the simplex search method was used. There were used several kinds of measurements that provided the degree of similarity between the registered images: normalized 2D cross-correlation (N2D), joint entropy (JE), Euclidean distance (ED), standardized Euclidean distance (SED), city block metric (CBM), Minkowski distance (MD), Chebychev distance (CD), Mahalanobis distance (MAD), cosine of the included angle between points (COD), correlation (sample correlation between points of image transformed to the vector; CRD), sample Spearman’s rank correlation (SRC), Hamming distance (HD), Jaccard coefficient (JD).
  3. As soon as the parameters of transformation were found, the TH binary mask was created and consequently the hand was segmented.

Using this method it is possible to segment also the hand with cold areas, but the disadvantage of this method is again a dependence on the presence of VIS image and also the speed of segmentation which is increased by the optimization algorithm.

All functions in this toolbox are written in MATLAB. To see how to use them please type “help function_name”. Toolbox contains also an example script demonstrating the use of functions.