Introduction

The consumption of plant-based protein foods is increasing all over the world. The global revenue from plant-based protein market is estimated to grow to $17.4 billion in 2027 from $12.2 billion in 2022.¹ Recently new high-protein plant-based foods such as meat, cheese and yogurt analogues are being developed and marketed all over the world. In United States alone, 619 new plant-based protein foods have been launched in 2022. ² Reformulating the commonly consumed low-protein foods by partially substituting with high protein ingredients is an alternative approach to improve plant-based protein consumption. In this approach, protein isolates or protein concentrates from soybeans, peas or other pulses are mostly used to improve protein content.

New food products have to be evaluated rigorously for sensory qualities to determine their acceptability. In general, trained or untrained panelists are required to evaluate the food products to minimize subjectivity during testing. ³  Selection of optimum number of panelists and identification of inclusion-criteria are important requirements to improve the success of the sensory study. ⁴ Gomez-Corona et al. emphasized the need for additional skills such as memory created  during the training session as it greatly influences the performance of the panelist during actual sensory evaluation. ⁵ Other challenges in selection of panelists include potential allergies for different types of foods, dietary habits, availability for training sessions, concentration during training, concentration during testing, medical conditions (specific anosmia or color vision deficiency), and dental wearing (for oral sensory).^6,7,8,9. Djekic et al. reviewed the characteristics of 179 trained panelists, using data collected from 16 scientific journals. ¹⁰ It was mentioned that the panelists had to undergo up to 10 training sessions for a period of 1 week to 12 weeks. Therefore, an objective method to assess the sensory attributes of food products using images is highly beneficial for the rapid determination of the food quality.

In general, limited literature is available for the comparison of sensory qualities and image attributes. Torrie et al. compared sensory test  scores (18 attributes, 10 panelists) with features collected through electronic nose and image analysis  for bread. ¹¹ They achieved an R² value of 0.69 for the e-nose and 0.56 for the image features.

The objective of this study was to determine the potential of red-green-blue (RGB) color images to evaluate the sensory qualities of whole wheat bread reformulated with pea and soy protein isolates as model food.

Materials

Bread preparation and sensory evaluation

Seven types of bread (3 loaves in each type) were prepared at the Bakery, University Centre, University of Guelph with the help of a Chef: whole wheat flour (WW) (control), 20% whole wheat flour replaced by pea protein isolate (P20), 30% whole wheat flour replaced by pea protein isolate (P30), 40% whole wheat flour replaced by pea protein isolate (P40), 20% whole wheat flour replaced by soy protein isolate (S20), 30% whole wheat flour replaced by soy protein isolate (S30), and 40% whole wheat flour replaced by soy protein isolate (S40). Pure protein isolates were purchased from the Canadian Protein company, ON, Canada (pea protein isolate) and My Protein company, Shepherdsville, KY, USA (soy protein isolate). Other ingredients used for the dough preparation and baking procedure are explained by Shivaani. ¹²

Ten graduate students (untrained) volunteered to evaluate the bread samples using 9-1 hedonic sensory scale (9-like extremely, 8-like very much, 7-like moderately, 6-like slightly, 5-neither like nor dislike, 4-dislike slightly, 3-dislike moderately, 2-dislike very much, 1-dislike extremely) (Shivaani) ¹² . They assessed chewiness, graininess, moistness, taste, desired aroma and overall acceptability of the seven bread samples. All the volunteers were informed about the ingredients and making process of these breads. and their consent to participate in this study was obtained before the sensory evaluation.

Methods

Imaging techniques and image analysis

RGB color images of the bread crusts and slices were acquired using a digital camera (Canon EOS 6D, 1024 X 683 pixels). A stereo microscope (SMZ 745T, Nikon corporation) was used to acquire microscopic images of the bread slices. Python programming was used  for image analysis and feature extraction (OpenCV python framework). RGB color values of the segmented bread crusts in all blends were mapped through 3-Dimensional plots to explain the color distribution.

Edge detection in bread slices

The digital images of the bread slices were used to extract edge detection features. The images were converted into gray scale. Then noise within the segmented image was removed. Finally Canny edge detection operator was applied to map the contour based edges and count the number of edges.

Pores characterization in bread slices

The microscopic images of the bread slices were used to evaluate pores. Watershed method was used to segment bread slices from the background. Pore analysis involved locating microscopic pores within the bread slices. The pixel resolution was reduced to obtained clear difference between foreground (region of interest) and background. Therefore, the RGB image was converted into HSV image (Figs. 1 and 2). Then the saturation channel alone was taken to obtain a grayscale image. The k-means clustering on the grayscale image was used to quantize color pixel based on number of centroids. Then the histogram-based segmentation was applied to obtain the pore numbers and pore area.

Hu-moments similarity assessment

The Hu-moments were analyzed using the microscopic images of the bread slices. After obtaining pores, Hu-moments, a shape descriptor was applied. It created seven types of moments using mean, variance, skew and kurtosis. These moments are invariant to rotation, scale and translation. This 7-Dimensional feature vector was used as a mean for matching shape or distribution similarity in pores. The feature vectors for each sample was used to assess the similarity with respect to the control bread slice (WW) using the nearest neighbor approach . In this k-nearest neighbor technique, clustering of data points that are closer to one another are assigned to the same cluster. During this analysis, the data- points that are closer to one another were similar to one another. ¹³ The Euclidean distance between feature vectors of different samples were compared with respected to the control bread slices. The feature vector of WW bread was used as a reference vector to apply the Euclidean distance with feature vector of all other blends. The lesser the Euclidean distance closer of that bread type in comparison to whole wheat control bread.

Statistical analysis

The linear regression and polynomial regression (fourth order) models were used to predict sensory attributes from number of edges, number of pores, pore area, and Hu-moment similarity. In this study, Microsoft Excel software (version 16.77.1., 2023), Microsoft Corporation, Redmond, Washington, USA was used for all statistical analysis.

Figure 1: Methodology for pore analysis and Hu-moment similarity assessment Click here to view Figure

Figure 2: Pore segmentation from microscopic images of the bread slices Click here to view Figure

Results and Discussion

RGB 3-D color plot for bread crust

The typical RGB color images of the bread crust made with whole wheat flour and protein isolates are shown in Fig. 3a. From the crust images, it is observed that the expansion of the loaf was restricted while  protein isolate was added to wheat flour. The expansion of the WW bread was higher than  other blends, and the top view of the crust was closer to rectangular shape. Whereas the top view of the crust of protein blended loaves was elliptical. The P40, S30 and S40 loaves displayed significant cracks on the surface of the crusts. This might be due to the interaction of protein isolate and wheat flour during proofing of dough, or insufficient water content or both.

Wu and Sun explained color models (RGB, L*a*b*, HSV, and CMY) to describe the color values of the food products. ¹⁴ Among various color models,  RGB space is most common and it is closer to human perception. In RGB color plot, the R-G-B color values of the surface of the object are represented in a 3-dimensional (3-D) plot to visualize the color profile of the samples. Fig. 3b shows the 3-D plots of bread crust in different blends. In general, the differences in 3-D color plot were observed at the tail ends. There was no solid tail at the WW loaf, whereas the P40 and S40 blends demonstrated solid and continuous tail. This might be due to the darkness caused by the surface cracks in the high protein blended loaves.

Edge detection features in bread slices

The process of mapping the sharp discontinuities in an image may  be called as edge detection. These edges can be quantified and used for characterization of food products based on surface texture. In general, the edges have been used to categorize the products into  a few pre-determined groups. For example, Sarkar et al. used edge detection features to classify the gooseberry based on freshness into two groups (good or deteriorated). ¹⁵ They used surface irregularities for edge detection, and achieved more than 90% classification accuracy. To our knowledge, there is no report available in the public domain to develop food quality prediction models using edge features.

RGB images

Three dimensional color plots

Figure 3: RGB images and 3-D color plots of bread crusts in various blends Click here to view Figure

In this study, we developed prediction model for sensory qualities using number of edges. Fig. 4 shows the RGB images and their corresponding edge images of  the bread slices of various blends. The R² value of the linear regression model  was 0.64, 0.39, 0.30, 0.46, 0.34 and 0.42 for chewiness, graininess, moistness, taste, desired aroma, and overall acceptability scores, respectively (Fig. 5). Whereas the R² value was increased for all sensory qualities in polynomial regression model. It was 0.82, 0.60, 0.55, 0.77, 0.67 and 0.50 for chewiness, graininess, moistness, taste, desired aroma, and overall acceptability scores, respectively (Fig. 6). In both models, the correlation between the number of edges and chewiness was the highest among other attributes.

Number of pores in bread slices

The number of pores measured on the microscopic images of the bread slices are 121, 100, 44, 67, 133, 161 and 90, for WW, P20, P30, P40, S20, S30 and S40, respectively. In general, the WW bread had bigger size pores than protein isolates blended breads. The R² values ranged 0.07-0.36 and 0.82-0.99 in linear and polynomial regression models, respectively (Figs. 7 and 8). The pore numbers had the highest correlation (0.99) with taste score.

Naji-Tabasi and Mohebbi measured pore area fraction, pore size distribution, pore shape, and dimension of pore boundaries to evaluate the effect of cress seed gum and xanthan gum in gluten free bread. ¹⁶  It was found that blending hydrocolloids improved the pore area fraction.

Pore area in bread slices

The pore area in the bread slices was 173, 27, 20, 15, 110, 75 and 23 pixel, for WW, P20, P30, P40, S20, S30 and S40, respectively. The WW and S20 blends had more than 100 pixel pore area.

The R² value with pore area was more than 0.80 for chewiness in linear regression model and for chewiness, moistness, taste and overall acceptability in polynomial regression model (Figs. 9 and 10).

Demirkesen et al. used scanning electron microscopic images to characterize the baked rice bread to determine the impact of additional ingredients (chestnut flour and xanthan–guar gum blend–emulsifier) and baking techniques (conventional vs  infrared–microwave combination ovens) on pore area fraction, pore size distribution, and roundness values of pores. ¹⁷ The lowest pore area fraction  was reported for the rice bread without any additives. On the other hand, the highest pore area fraction values were obtained in blended breads (46% replacement with chestnut flour containing xanthan–guar gum) baked in an infrared–microwave combination oven.

Figure 4: RGB images and edge images of bread slices in various blends Click here to view Figure

Figure 5: Linear regression relationship between bread quality and number of edges Click here to view Figure

Figure 6: Polynomial regression relationship between bread quality and number of edges Click here to view Figure

Figure 7: Linear regression relationship between bread quality and pore numbers Click here to view Figure

Figure 8: Polynomial regression relationship between bread quality and pore numbers Click here to view Figure

Figure 9: Linear regression relationship between bread quality and pore area Click here to view Figure

Figure 10: Polynomial regression relationship between bread quality and pore area Click here to view Figure

Hu-moment similarity of bread slices

The Hu-moment feature similarity values were 66.0, 36.1, 24.5, 2.4, 11.0, and 34.0 for P20, P30, P40, S20, S30 and S40, respectively in comparison with WW (WW=1). In general, S20 and S30 scored closer similarity values with respect to WW compared to other protein blends. The R² value with Hu-moment was 0.97, 0.92, 0.96, 0.99, 0.97, and 0,87 for chewiness, graininess, moistness, taste, desired aroma and overall acceptability scores, respectively in polynomial regression models (Fig 12). However, the The R² value with Hu-moment was ranged between 0.05 and 0.31 for sensory qualities in linear regression model (Fig. 11).

The availability of literature on the utilization of similarity assessment in food quality is limited. Gao et al. proposed similarity assessment using dilation based multi-resolutional skeleton (DBMS) technique. ¹³ Zunic et al. explained the potential of circularity measure to define the degree to which a shape of any object differs from a perfect circle. ¹⁸ Zhang et al.  used Hu moment similarity assessment in gesture recognition. ¹⁹ They measured the similarity between the image to be recognized and template image and obtained more than 90% accuracy. Gornale et al.  used Hu invariant moments to classify  knee Osteoarthritis. ²⁰  They used Hu invariant moments on knee X-ray images for early detection of knee osteoarthritis.

Conclusion

In this study, digital color images and microscopic images were used to correlate the sensory attributes of the reformulated whole wheat breads. Hu-moment similarity features yielded the highest correlation coefficient (average R² = 0.95), and have the potential to use for the prediction of sensory qualities of bread slices. Although human-based sensory evaluations are inevitable for food products, image-based characterization will provide supplementary information that can aid in taking objective and accurate decisions during food product development.

Figure 11: Linear regression relationship between bread quality and Hu-moment similarity Click here to view Figure

Figure 12: Polynomial regression relationship between bread quality and Hu-moment similarity Click here to view Figure

Acknowledgement

Ms. Turner, (Extended Essay advisor, Guelph Collegiate Vocational Institute), Chef Vijay Nair, Annie, and Rose (University Centre Bakery, University of Guelph), Rathnapriya and Sindhu (microscopic imaging), Hillary Rooyakkers (digital imaging), and all sensory panelists.

Authors’ Contribution 

Tamilselvan Venkatesan: Image analysis, data analysis, data interpretation, partial manuscript writing Manickavasagan Shivaani: Experiment execution, data analysis and partial manuscript writing

Data Availability Statement

The manuscript incorporates all datasets produced or examined throughout this research study

Ethics Approval Statement

This study does not involve any experiments on humans and animals. For sensory, the volunteers were informed about the ingredients and making process of these breads, and their consent to participate in this study was obtained before the sensory evaluation.

References

1. Markets and Markets. Plant-based protein market. Report no. FB 7358. 2022; Accessed on July 12 2023. Available at: https://www.marketsandmarkets.com/Market-Reports/plant-based-protein-market-14715651.html?gclid=Cj0KCQjwusunBhCYARIsAF BsUP_7i9g434OzpDtGwLDml5uQTvYypFkl_tzzxzMDaPFOn4lu14pVG5waAtw3EALw_wcB
2. Good Food Institute. State of the industry report: Plant-based meat, seafood, eggs, and dairy. 2023; Accessed on April 23, 2023. Available at: https://gfi.org /resource/plant-based -meat-eggs-and- dairy-state-of-the-industry-report/#:~: text=In%202022%2C%20according %20to%20 Euromonitor, U.S.%20retail%20 shelves%20in %202022
3. Meilgaard M.C., Carr B.T., Carr B.T. Sensory evaluation techniques. 2006; New York: CRC.
   CrossRef
4. Talsma P. How much sensory panel data do we need?. Food Quality and Preference. 2018; 67:3-9.
   CrossRef
5. Gomez-Corona C., Pohlenz A., Cayeux I., Valentin D. Panel performance and memory in visually impaired versus sighted panels. Food Quality and Preference. 2020; 80: 103807.
   CrossRef
6. Murray J.M., Delahunty C.M., Baxter I.A. Descriptive sensory analysis: past, present and future. Food research international. 2001; 34(6): 461-471.
   CrossRef
7. Lawless H.T., Heymann H. Sensory evaluation of food: principles and practices. 2010; (Vol. 2). New York: Springer.
   CrossRef
8. Chen J. Food oral processing—A review. Food hydrocolloids. 2009; 23(1): 1-25.
   CrossRef
9. Tomasevic I., Tomovic V., Milovanovic B., Lorenzo J., Đorđević V., Karabasil N., Djekic I. Comparison of a computer vision system vs. traditional colorimeter for color evaluation of meat products with various physical properties. Meat science. 2019; 148: 5-12.
   CrossRef
10. Djekic I., Lorenzo J.M., Munekata P.E., Gagaoua M., Tomasevic I. Review on characteristics of trained sensory panels in food science. Journal of Texture Studies. 2021; 52(4): 501-509.
    CrossRef
11. Torri L., Migliorini P., Masoero G. Sensory test vs. electronic nose and/or image analysis of whole bread produced with old and modern wheat varieties adjuvanted by means of the mycorrhizal factor. Food Research International. 2013; 54(2): 1400-1408.
    CrossRef
12. Shivaani M. Characterization of whole wheat bread reformulated with pea and soy protein isolates. International Journal of Nutrition, Pharmacology, Neurological Diseases. 2020; 10(3):112-119.
13. Gao W., Gao S.M., Liu Y.S., Bai J., Hu B.K. Multiresolutional similarity assessment and retrieval of solid models based on DBMS. Computer-Aided Design. 2006; 38(9): 985-1001.
    CrossRef
14. Wu D., Sun D.W. Colour measurements by computer vision for food quality control–A review. Trends in Food Science & Technology. 2013; 29(1): 5-20.
    CrossRef
15. Sarkar T., Mukherjee A., Chatterjee K., Ermolaev V., Piotrovsky D., Vlasova K., Shariati M.A., Munekata P.E.,  Lorenzo J.M. Edge detection aided geometrical shape analysis of Indian gooseberry (phyllanthus emblica) for freshness classification. Food Analytical Methods. 2022; 15(6): 1490-1507.
    CrossRef
16. Naji-Tabasi S.,  Mohebbi M. Evaluation of cress seed gum and xanthan gum effect on macrostructure properties of gluten-free bread by image processing. Journal of Food Measurement and characterization. 2015; 9: 110-119.
    CrossRef
17. Demirkesen I., Sumnu G.,  Sahin S. Image analysis of gluten-free breads prepared with chestnut and rice flour and baked in different ovens. Food and Bioprocess Technology. 2013; 6: 1749-1758.
    CrossRef
18. Zunic J., Hirota K., Rosin P.L. A Hu moment invariant as a shape circularity measure. Pattern Recognition. 2010; 43(1): 47-57.
    CrossRef
19. Zhang T., Gao X., Li J. The improved hu moment and its application in gesture recognition. International Conference on Computer Vision, Image and Deep Learning (CVIDL). 2020; 577-580.
    CrossRef
20. Gornale S.S., Patravali P.U., Hiremath P.S. Automatic detection and classification of knee osteoarthritis using hu’s invariant moments. Frontiers in Robotics and AI. 2020; 7: 591827.
    CrossRef