Metaheuristic Optimization Algorithms in Mathematical Sciences: A Comprehensive Review of Numerical Methods, Combinatorial Problems, and Emerging Mathematical Applications

Authors

https://doi.org/10.48313/maa.v2i3.50

Abstract

Metaheuristic optimization algorithms have emerged as indispensable tools for solving complex mathematical problems that resist classical analytical and gradient-based methods. This paper presents a comprehensive systematic review of metaheuristic algorithms applied across the mathematical sciences, encompassing continuous function optimization, combinatorial optimization, constrained and multi-objective optimization, numerical methods enhancement, and integer programming. Following the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) framework, a total of 312 primary studies published between 2000 and 2025 were identified from five major databases, of which 187 met the inclusion criteria for detailed analysis. The review provides a rigorous examination of the theoretical foundations underlying metaheuristic convergence, including the No Free Lunch (NFL) Theorem and its mathematical implications. Twelve prominent algorithms Genetic Algorithm (GA), Particle Swarm Optimization (PSO), Differential Evolution (DE), Simulated Annealing (SA), Ant Colony Optimization (ACO), Artificial Bee Colony (ABC), Grey Wolf Optimizer (GWO), Whale Optimization Algorithm (WOA), Sine Cosine Algorithm (SCA), Moth-Flame Optimization (MFO), Harris Hawks Optimization (HHO), and Salp Swarm Algorithm (SSA) are evaluated on standard benchmark functions including Sphere, Rosenbrock, Rastrigin, Ackley, Schwefel, and Griewank in 30 dimensions. Performance comparisons on Traveling Salesman Problem (TSP) benchmark instances,  Congress on Evolutionary Computation (CEC) constrained problems, and multi-objective test suites are presented with statistical significance testing via Friedman and Wilcoxon signed-rank tests. A bibliometric analysis of publication trends from 2000 to 2025 reveals accelerating growth in this field. The review identifies critical open challenges including theoretical convergence guarantees, scalability to ultra-high dimensions, and parameter sensitivity, while outlining future directions such as quantum-inspired metaheuristics, integration with deep reinforcement learning, and the pursuit of a theoretical unification framework.

Keywords:

Metaheuristic algorithms, Mathematical optimization, Combinatorial optimization, Numerical analysis, Function optimization, NP-hard problems, Convergence analysis

References

  1. [1] Dantzig, G. B. (2016). Linear programming and extensions. Princeton University Press. https://www.torrossa.com/en/resources/an/5793009
  2. [2] Yang, X. S. (2020). Nature-inspired optimization algorithms. Academic Press. https://www.researchgate.net/publication/263171713
  3. [3] Mirjalili, S., Song Dong, J., Sadiq, A. S., & Faris, H. (2019). Genetic algorithm: Theory, literature review, and application in image reconstruction. In Nature-inspired optimizers: Theories, literature reviews and applications (PP. 69-85). Springer, Cham. https://doi.org/10.1007/978-3-030-12127-3_5
  4. [4] Nocedal, J., & Wright, S. J. (2006). Numerical optimization. New York, NY: Springer New York. https://doi.org/10.1007/978-0-387-40065-5
  5. [5] Talbi, E. G. (2009). Metaheuristics: From design to implementation. John Wiley & Sons. https://zeus.inf.ucv.cl/~bcrawford/DiplomadoIA_2021/Cap1_Metaheuristics_Talbi.pdf
  6. [6] Glover, F. W., & Kochenberger, G. A. (2003). Handbook of metaheuristics. Springer Science & Business Media. https://scispace.com/pdf/handbook-of-metaheuristics-3ue3f2vgdx.pdf
  7. [7] Glover, F. (1986). Future paths for integer programming and links to artificial intelligence. Computers & operations research, 13(5), 533–549. https://doi.org/10.1016/0305-0548(86)90048-1
  8. [8] Blum, C., & Roli, A. (2003). Metaheuristics in combinatorial optimization: Overview and conceptual comparison. ACM computing surveys (CSUR), 35(3), 268–308. https://doi.org/10.1145/937503.937505
  9. [9] Holland, J. H. (1992). Adaptation in natural and artificial systems: An introductory analysis with applications to biology, control, and artificial intelligence. MIT Press. https://mitpress.mit.edu/9780262581110/adaptation-in-natural-and-artificial-systems/
  10. [10] Goldberg, D. E. (1989). Genetic algorithms in search, optimization, and machine learning. Addison-Wesley. https://www2.fiit.stuba.sk/~kvasnicka/Free books/Goldberg_Genetic_Algorithms_in_Search.pdf
  11. [11] Storn, R., & Price, K. (1997). Differential evolution-a simple and efficient heuristic for global optimization over continuous spaces. Journal of global optimization, 11(4), 341–359. https://doi.org/10.1023/A:1008202821328
  12. [12] Kennedy, J., & Eberhart, R. (1995). Particle swarm optimization. Proceedings of ICNN'95-international conference on neural networks (Vol. 4, pp. 1942-1948). IEEE. https://doi.org/10.1109/ICNN.1995.488968
  13. [13] Dorigo, M., Maniezzo, V., & Colorni, A. (1996). Ant system: Optimization by a colony of cooperating agents. IEEE transactions on systems, man, and cybernetics, part b (CYBERNETICS), 26(1), 29–41. https://doi.org/10.1109/3477.484436
  14. [14] Karaboga, D., & others. (2005). An idea based on honey bee swarm for numerical optimization. https://abc.erciyes.edu.tr/pub/tr06_2005.pdf
  15. [15] Kirkpatrick, S., Gelatt Jr, C. D., & Vecchi, M. P. (1983). Optimization by simulated annealing. Science, 220(4598), 671–680. https://cir.nii.ac.jp/crid/1370016973449261184
  16. [16] Mirjalili, S., Mirjalili, S. M., & Lewis, A. (2014). Grey wolf optimizer. Advances in engineering software, 69, 46–61. https://doi.org/10.1016/j.advengsoft.2013.12.007
  17. [17] Mirjalili, S., & Lewis, A. (2016). The whale optimization algorithm. Advances in engineering software, 95, 51–67. https://doi.org/10.1016/j.advengsoft.2016.01.008
  18. [18] Mirjalili, S. (2016). SCA: A sine cosine algorithm for solving optimization problems. Knowledge-based systems, 96, 120–133. https://doi.org/10.1016/j.knosys.2015.12.022
  19. [19] Mirjalili, S. (2015). Moth-flame optimization algorithm: A novel nature-inspired heuristic paradigm. Knowledge-based systems, 89, 228–249. https://doi.org/10.1016/j.knosys.2015.07.006
  20. [20] Heidari, A. A., Mirjalili, S., Faris, H., Aljarah, I., Mafarja, M., & Chen, H. (2019). Harris Hawks optimization: Algorithm and applications. Future generation computer systems, 97, 849–872. https://doi.org/10.1016/j.future.2019.02.028
  21. [21] Mirjalili, S., Gandomi, A. H., Mirjalili, S. Z., Saremi, S., Faris, H., & Mirjalili, S. M. (2017). Salp swarm algorithm: A bio-inspired optimizer for engineering design problems. Advances in engineering software, 114, 163–191. https://doi.org/10.1016/j.advengsoft.2017.07.002
  22. [22] Jamil, M., & Yang, X. S. (2013). A literature survey of benchmark functions for global optimisation problems. International journal of mathematical modelling and numerical optimisation, 4(2), 150–194. https://doi.org/10.1504/IJMMNO.2013.055204
  23. [23] Derrac, J., Garcia, S., Molina, D., & Herrera, F. (2011). A practical tutorial on the use of nonparametric statistical tests as a methodology for comparing evolutionary and swarm intelligence algorithms. Swarm and evolutionary computation, 1(1), 3–18. https://doi.org/10.1016/j.swevo.2011.02.002
  24. [24] Shaikh, M. S., Raj, S., Zheng, G., Xie, S., Wang, C., Dong, X., & Junejo, N. U. R. (2025). Applications, classifications, and challenges: A comprehensive evaluation of recently developed metaheuristics for search and analysis. Artificial intelligence review, 58(12), 1–110. https://doi.org/10.1007/s10462-025-11377-6
  25. [25] Boyd, S., & Vandenberghe, L. (2004). Convex optimization. Cambridge University Press. https://doi.org/10.1017/CBO9780511804441
  26. [26] Yang, X. S. (2010). Nature-inspired metaheuristic algorithms. Luniver Press. https://www.researchgate.net/publication/235979455
  27. [27] Sörensen, K. (2015). Metaheuristics—the metaphor exposed. International transactions in operational research, 22(1), 3–18. https://doi.org/10.1111/itor.12001
  28. [28] He, J., & Yu, X. (2001). Conditions for the convergence of evolutionary algorithms. Journal of systems architecture, 47(7), 601–612. https://doi.org/10.1016/S1383-7621(01)00018-2
  29. [29] Hajek, B. (1988). Cooling schedules for optimal annealing. Mathematics of operations research, 13(2), 311–329. https://doi.org/10.1287/moor.13.2.311
  30. [30] Rudolph, G. (1994). Convergence analysis of canonical genetic algorithms. IEEE transactions on neural networks, 5(1), 96–101. https://doi.org/10.1109/72.265964
  31. [31] Clerc, M., & Kennedy, J. (2002). The particle swarm-explosion, stability, and convergence in a multidimensional complex space. IEEE transactions on evolutionary computation, 6(1), 58–73. https://doi.org/10.1109/4235.985692
  32. [32] den Bergh, F., & Engelbrecht, A. P. (2006). A study of particle swarm optimization particle trajectories. Information sciences, 176(8), 937–971. https://doi.org/10.1016/j.ins.2005.02.003
  33. [33] Zaharie, D. (2009). Influence of crossover on the behavior of differential evolution algorithms. Applied soft computing, 9(3), 1126–1138. https://doi.org/10.1016/j.asoc.2009.02.012
  34. [34] Gutjahr, W. J. (2002). ACO algorithms with guaranteed convergence to the optimal solution. Information processing letters, 82(3), 145–153. https://www.aco-metaheuristic.org/downloads/ants3.pdf
  35. [35] Črepinšek, M., Liu, S. H., & Mernik, M. (2013). Exploration and exploitation in evolutionary algorithms: A survey. ACM computing surveys (CSUR), 45(3), 1–33. https://doi.org/10.1145/2480741.2480752
  36. [36] Lynn, N., & Suganthan, P. N. (2015). Heterogeneous comprehensive learning particle swarm optimization with enhanced exploration and exploitation. Swarm and evolutionary computation, 24, 11–24. https://doi.org/10.1016/j.swevo.2015.05.002
  37. [37] Shi, Y., & Eberhart, R. (1998). A modified particle swarm optimizer. Evolutionary computation proceedings (Vol. 890, pp. 69–73). IEEE. https://doi.org/10.1109/ICEC.1998.699146
  38. [38] Das, S., & Suganthan, P. N. (2010). Differential evolution: A survey of the state-of-the-art. IEEE transactions on evolutionary computation, 15(1), 4–31. https://doi.org/10.1109/TEVC.2010.2059031
  39. [39] Cuevas, E., Espejo, E. B., & Enriquez, A. C. (2019). Metaheuristics algorithms in power systems. Springer. https://doi.org/10.1007/978-3-030-11593-7
  40. [40] Wolpert, D. H., & Macready, W. G. (2002). No free lunch theorems for optimization. IEEE transactions on evolutionary computation, 1(1), 67–82. https://doi.org/10.1109/4235.585893
  41. [41] Kerschke, P., Hoos, H. H., Neumann, F., & Trautmann, H. (2019). Automated algorithm selection: Survey and perspectives. Evolutionary computation, 27(1), 3–45. https://doi.org/10.1162/evco_a_00242
  42. [42] Igel, C., & Toussaint, M. (2005). A no-free-lunch theorem for non-uniform distributions of target functions. Journal of mathematical modelling and algorithms, 3(4), 313–322. https://doi.org/10.1007/s10852-005-2586-y
  43. [43] Page, M. J., McKenzie, J. E., Bossuyt, P. M., Boutron, I., Hoffmann, T. C., & Mulrow, C. D. (2021). The PRISMA 2020 statement: An updated guideline for reporting systematic reviews. British medical journal, 372. https://doi.org/10.1136/bmj.n71
  44. [44] Yao, X., Liu, Y., & Lin, G. (1999). Evolutionary programming made faster. IEEE transactions on evolutionary computation, 3(2), 82–102. http://dx.doi.org/10.1109/4235.771163
  45. [45] Liang, J. J., Qu, B. Y., Suganthan, P. N., & Hernández Diaz, A. G. (2013). Problem definitions and evaluation criteria for the CEC 2013 special session on real-parameter optimization. Computational intelligence laboratory, zhengzhou university, zhengzhou, china and nanyang technological university, singapore, technical report, 201212(34), 281–295. https://peerj.com/articles/cs-2671/CEC2013.pdf
  46. [46] Applegate, D. L., Bixby, R. E., Chvátal, V., & Cook, W. J. (2011). The traveling salesman problem: A computational study. Princeton University Press. https://doi.org/10.1515/9781400841103
  47. [47] Dorigo, M. (2007). Ant colony optimization. Scholarpedia, 2(3), 1461. http://dx.doi.org/10.4249/scholarpedia.1461
  48. [48] Larranaga, P., Kuijpers, C. M. H., Murga, R. H., Inza, I., & Dizdarevic, S. (1999). Genetic algorithms for the travelling salesman problem: A review of representations and operators. Artificial intelligence review, 13(2), 129–170. https://doi.org/10.1023/A:1006529012972
  49. [49] Shi, X. H., Liang, Y. C., Lee, H. P., Lu, C., & Wang, Q. X. (2007). Particle swarm optimization-based algorithms for TSP and generalized TSP. Information processing letters, 103(5), 169–176. https://doi.org/10.1016/j.ipl.2007.03.010
  50. [50] Mirjalili, S., & Lewis, A. (2013). S-shaped versus V-shaped transfer functions for binary particle swarm optimization. Swarm and evolutionary computation, 9, 1–14. https://doi.org/10.1016/j.swevo.2012.09.002
  51. [51] Malaguti, E., & Toth, P. (2010). A survey on vertex coloring problems. International transactions in operational research, 17(1), 1–34. https://doi.org/10.1111/j.1475-3995.2009.00696.x
  52. [52] Lan, G., DePuy, G. W., & Whitehouse, G. E. (2007). An effective and simple heuristic for the set covering problem. European journal of operational research, 176(3), 1387–1403. https://doi.org/10.1016/j.ejor.2005.09.028
  53. [53] Reinelt, G. (1991). TSPLIB—A traveling salesman problem library. ORSA journal on computing, 3(4), 376–384. http://dx.doi.org/10.1287/ijoc.3.4.376
  54. [54] Deb, K. (2000). An efficient constraint handling method for genetic algorithms. Computer methods in applied mechanics and engineering, 186(2–4), 311–338. https://doi.org/10.1016/S0045-7825(99)00389-8
  55. [55] Coello, C. A. C. (2002). Theoretical and numerical constraint-handling techniques used with evolutionary algorithms: A survey of the state of the art. Computer methods in applied mechanics and engineering, 191(11–12), 1245–1287. https://doi.org/10.1016/S0045-7825(01)00323-1
  56. [56] Liang, J. J., Runarsson, T. P., Mezura Montes, E., Clerc, M., Suganthan, P. N., Coello, C. A. C., & Deb, K. (2006). Problem definitions and evaluation criteria for the CEC 2006 special session on constrained real-parameter optimization. Journal of applied mechanics, 41(8), 8–31. https://www.researchgate.net/publication/216301032
  57. [57] Takahama, T., & Sakai, S. (2006). Constrained optimization by the∊ constrained differential evolution with gradient-based mutation and feasible elites. 2006 IEEE international conference on evolutionary computation (p. 2). IEEE. https://doi.org/10.1109/CEC.2006.1688283
  58. [58] Parsopoulos, K. E., & Vrahatis, M. N. (2002). Particle swarm optimization method for constrained optimization problems. Intelligent technologies–theory and application: New trends in intelligent technologies, 76(1), 214-220. https://www.researchgate.net/publication/2527227
  59. [59] Mezura Montes, E., & Coello, C. A. C. (2011). Constraint-handling in nature-inspired numerical optimization: Past, present and future. Swarm and evolutionary computation, 1(4), 173–194. https://doi.org/10.1016/j.swevo.2011.10.001
  60. [60] Deb, K., Sindhya, K., & Hakanen, J. (2016). Multi-objective optimization. In Decision sciences (pp. 161–200). CRC Press. https://doi.org/10.1201/9781315183176-4
  61. [61] Deb, K., Pratap, A., Agarwal, S., & Meyarivan, T. (2002). A fast and elitist multiobjective genetic algorithm: NSGA-II. IEEE transactions on evolutionary computation, 6(2), 182–197. https://doi.org/10.1109/4235.996017
  62. [62] Coello, C. A. C., Pulido, G. T., & Lechuga, M. S. (2004). Handling multiple objectives with particle swarm optimization. IEEE transactions on evolutionary computation, 8(3), 256–279. https://doi.org/10.1109/TEVC.2004.826067
  63. [63] Zhang, Q., & Li, H. (2007). MOEA/D: A multiobjective evolutionary algorithm based on decomposition. IEEE transactions on evolutionary computation, 11(6), 712–731. https://doi.org/10.1109/TEVC.2007.892759
  64. [64] Zitzler, E., Thiele, L., Laumanns, M., Fonseca, C. M., & Da Fonseca, V. G. (2003). Performance assessment of multiobjective optimizers: An analysis and review. IEEE transactions on evolutionary computation, 7(2), 117–132. https://doi.org/10.1109/TEVC.2003.810758
  65. [65] Grosan, C., & Abraham, A. (2008). A new approach for solving nonlinear equations systems. IEEE transactions on systems, man, and cybernetics-part a: Systems and humans, 38(3), 698–714. https://doi.org/10.1109/TSMCA.2008.918599
  66. [66] Voss, H. U., Timmer, J., & Kurths, J. (2004). Nonlinear dynamical system identification from uncertain and indirect measurements. International journal of bifurcation and chaos, 14(06), 1905–1933. https://doi.org/10.1142/S0218127404010345
  67. [67] Rao, R. V., & Pawar, P. J. (2010). Parameter optimization of a multi-pass milling process using non-traditional optimization algorithms. Applied soft computing, 10(2), 445–456. https://doi.org/10.1016/j.asoc.2009.08.007
  68. [68] Wolsey, L. A. (1998). Integer programming. New York, NY: John Wiley & Sons. https://books.google.com/books/about/Integer_Programming.html?id=x7RvQgAACAAJ
  69. [69] Liao, T., Stützle, T., de Oca, M. A. M., & Dorigo, M. (2014). A unified ant colony optimization algorithm for continuous optimization. European journal of operational research, 234(3), 597–609. https://doi.org/10.1016/j.ejor.2013.10.024
  70. [70] Kennedy, J., & Eberhart, R. C. (1997). A discrete binary version of the particle swarm algorithm. 1997 IEEE international conference on systems, man, and cybernetics. Computational cybernetics and simulation (Vol. 5, pp. 4104–4108). IEEE. https://doi.org/10.1109/ICSMC.1997.637339
  71. [71] Emary, E., Zawbaa, H. M., & Hassanien, A. E. (2016). Binary grey wolf optimization approaches for feature selection. Neurocomputing, 172, 371–381. https://doi.org/10.1016/j.neucom.2015.06.083
  72. [72] Gendreau, M., Potvin, J Y., & others. (2010). Handbook of metaheuristics. Springer. https://doi.org/10.1007/978-1-4419-1665-5
  73. [73] Maniezzo, V., Stützle, T., & Voß, S. (2021). Matheuristics. Springer. https://doi.org/10.1007/978-3-030-70277-9
  74. [74] Garcia, S., Fernández, A., Luengo, J., & Herrera, F. (2009). A study of statistical techniques and performance measures for genetics-based machine learning: Accuracy and interpretability. Soft computing, 13(10), 959–977. https://doi.org/10.1007/s00500-008-0392-y
  75. [75] Wilcoxon, F. (1945). Individual comparisons by ranking methods. Biometrics bulletin, 1(6), 80–83. https://doi.org/10.2307/3001968
  76. [76] Hussain, K., Mohd Salleh, M. N., Cheng, S., & Shi, Y. (2019). Metaheuristic research: A comprehensive survey. Artificial intelligence review, 52(4), 2191–2233. https://doi.org/10.1007/s10462-017-9605-z
  77. [77] Mahdavi, S., Shiri, M. E., & Rahnamayan, S. (2015). Metaheuristics in large-scale global continues optimization: A survey. Information sciences, 295, 407–428. https://doi.org/10.1016/j.ins.2014.10.042
  78. [78] Yang, Z., Tang, K., & Yao, X. (2008). Large scale evolutionary optimization using cooperative coevolution. Information sciences, 178(15), 2985–2999. https://doi.org/10.1016/j.ins.2008.02.017
  79. [79] Jin, Y., & Branke, J. (2005). Evolutionary optimization in uncertain environments-a survey. IEEE transactions on evolutionary computation, 9(3), 303–317. https://doi.org/10.1109/TEVC.2005.846356
  80. [80] Aleti, A., & Moser, I. (2016). A systematic literature review of adaptive parameter control methods for evolutionary algorithms. ACM computing surveys (CSUR), 49(3), 1–35. https://doi.org/10.1145/2996355
  81. [81] Liang, J. J., Qu, B. Y., & Suganthan, P. N. (2013). Problem definitions and evaluation criteria for the CEC 2014 special session and competition on single objective real-parameter numerical optimization. Computational intelligence laboratory, zhengzhou university, zhengzhou china and technical report, nanyang technological university, singapore, 635(2), 2014. http://bee22.com/manual/tf_images/Liang CEC2014.pdf
  82. [82] Han, K. H., & Kim, J. H. (2002). Quantum-inspired evolutionary algorithm for a class of combinatorial optimization. IEEE transactions on evolutionary computation, 6(6), 580–593. https://doi.org/10.1109/TEVC.2002.804320
  83. [83] Zhang, G., Rong, H., Neri, F., & Pérez Jiménez, M. J. (2014). An optimization spiking neural P system for approximately solving combinatorial optimization problems. International journal of neural systems, 24(05), 1440006. https://doi.org/10.1142/S0129065714400061
  84. [84] Hu, Z., Xiong, S., Su, Q., & Zhang, X. (2013). Sufficient conditions for global convergence of differential evolution algorithm. Journal of applied mathematics, 2013(1), 193196. https://doi.org/10.1155/2013/193196
  85. [85] Li, K., Zhang, T., Wang, R., Wang, Y., Han, Y., & Wang, L. (2021). Deep reinforcement learning for combinatorial optimization: Covering salesman problems. IEEE transactions on cybernetics, 52(12), 13142–13155. https://doi.org/10.1109/TCYB.2021.3103811
  86. [86] Hutter, F., Hoos, H. H., & Leyton Brown, K. (2011). Sequential model-based optimization for general algorithm configuration. International conference on learning and intelligent optimization (pp. 507–523). Springer. https://doi.org/10.1007/978-3-642-25566-3_40

Downloads

Published

2025-06-16

Issue

Vol. 2 No. 3 (2025): Metaheuristic Algorithms with Applications

Section

Articles

How to Cite

Abdolmaleki, E. (2025). Metaheuristic Optimization Algorithms in Mathematical Sciences: A Comprehensive Review of Numerical Methods, Combinatorial Problems, and Emerging Mathematical Applications. Metaheuristic Algorithms With Applications, 2(3), 263–284. https://doi.org/10.48313/maa.v2i3.50

Similar Articles

1-10 of 17

You may also start an advanced similarity search for this article.