Sex estimation from photographs of the ilium and humerus using Deep Learning techniques

Published on January 30, 2023

Written by Marta Panizo

Traditionally, sex estimation is performed by means of numerical/tabular data obtained from manual measurement of the bones of interest. However, the success of a set of Artificial Intelligence techniques, known as Deep Learning (DL) [1], in image analysis problems (mainly through convolutional networks) invites the use of such techniques in the estimation of the biological profile (BP). These techniques allow fully automatic estimation from the images themselves, decreasing times and increasing the objectivity and reproducibility of the identification processes, matching, or even improving the performance of human experts in certain tasks and generating new knowledge.

If we focus on sex estimation, the research done to date by Panacea Cooperative Research, in collaboration with the University of Granada (UGR), has focused on two main problems:

1) Sex estimation in children from photographs of the ilium bone (Figure 1.a) [2].

2) Sex estimation in adults from photographs of the humerus bone (Figure 1.b) [3].

In both problems, a similar strategy was followed (which can be seen in Figure 2).

In the first case, a set of 135 photographs taken from the collection of the Physical Anthropology Laboratory of the UGR was used: 83 boys and 52 girls, with mean ages (± standard deviation) of 240±491 and 292±499 days for male and female infants, respectively. The results showed, on the one hand, the great difficulty of the problem in question, given that even an expert anthropologist did not exceed 61% success rate. This is logical given that the dimorphic characteristics that differentiate females and males do not appear until puberty. On the other hand, the DL techniques obtained a 59% success classification rate, very close to that of the expert. In this specific application, classical DL techniques were able to come very close to the performance of the human expert.

Figure 1: Examples of ilium photographs used in sex estimation in children (a). The left column shows bones from male infants, while the right column shows bones from female infants. Examples of photographs of the humerus used in sex estimation in adults can be seen in b. The left column shows examples of photographs of the humerus used in sex estimation in adult males, while the right column shows bones from adult females.

Figure 2: Approach followed in estimating sex from bone images. A set of labeled examples, belonging to females or males, is available from which to learn. The learning algorithm corresponds to a DL technique that performs relevant feature extraction and binary classification (male vs. female). Finally, once the machine is trained, it can classify new images not seen in the training phase. In this figure, the estimation of sex in adults from photographs of the humerus is used as an example.

In the second case, we used a collection of 401 photographs of the posterior view of the right humerus taken from the collection of the Physical Anthropology Laboratory of the UGR: 188 females and 213 males. The results showed, on the one hand, the lower difficulty of this problem in relation to age estimation in children, given that the accuracy percentages are much higher (both for the anthropologist and for the state-of-the-art techniques in an anthropological level (in this case, geometric morphometric techniques [4]), as well as for the DL techniques). Specifically, the expert achieved a hit rate of 83%, while the geometric morphometric technique obtained 75%, and the proposed DL technique reached 91%. In this particular application, not only has the performance of a physical anthropologist and the best pre-existing techniques been greatly exceeded but also, thanks to the application of certain explainability techniques, it has been possible to identify the most relevant regions of the bone for sex estimation. The areas of the bone containing these dimorphisms have been identified by the model as containing the most relevant information for sex estimation, as proposed in [5]. However, in addition to the already known dimorphisms (trochlea and olecranon fossa), it has also been observed that the convolutional network used considers an area of the bone superior to the distal epiphysis, which had not been evaluated in the field of forensic anthropology. Thus, the neural network is even capable of generating new knowledge and proposing a new anatomical region of possible anthropological interest (see Figure 3).

Figure 3: The higher the intensity of the yellow color, the more weight that area has in the network estimation. The estimates of the deep neural network are based on information from the olecranon fossa (a), the trochlea (b), or both (c). In the image in the lower right corner (d), a new region identified by the network and located over the distal epiphysis (in red, in d) can be seen. Previous scientific literature had not paid attention to this region, which the network has considered relevant for sex estimation.

At Panacea, we are enthusiastic about this technology, which represents the first set of fully automatic sex estimation methods included in Skeleton-ID. This post summarizes the methods and results published in two prestigious scientific journals: the International Journal of Legal Medicine and Neural Computing and Applications [2,3]. Panacea’s R+D team continues to improve the accuracy of these methods and to develop new ones using other anatomical regions and different imaging modalities.

Access to the online application for sex estimation from humerus photographs:

Estimate sex

References

[1] LeCun, Y., Bengio, Y., & Hinton, G. (2015). Deep learning. Nature, 521(7553), 436-444.

[2] Ortega, R. F., Irurita, J., Campo, E. J. E., & Mesejo, P. (2021). Analysis of the performance of machine learning and deep learning methods for sex estimation of infant individuals from the analysis of 2D images of the ilium. International Journal of Legal Medicine, 135(6), 2659-2666.

[3] Venema, J., Peula, D., Irurita, J., & Mesejo, P. (2022). Employing deep learning for sex estimation of adult individuals using 2D images of the humerus, submitted to Neural Computing and Applications (undergoing minor review).

[4] López-Lázaro, S., Pérez-Fernández, A., Alemán, I., & Viciano, J. (2020). Sex estimation of the humerus: A geometric morphometric analysis in an adult sample. Legal Medicine, 47, 101773.[5] Rogers, T. L. (1999). A visual method of determining the sex of skeletal remains using the distal humerus. Journal of Forensic Science, 44(1), 57-60.