The accelerating complexity of modern power grids, driven by the large-scale integration of renewable energy sources, the proliferation of Distributed Energy Resources (DER), and the increasing volatility of consumer demand patterns, necessitates intelligent and adaptive energy management solutions that surpass the capabilities of conventional optimization approaches. This comprehensive review systematically examines the role of Machine Learning (ML) techniques encompassing supervised learning, unsupervised learning, Reinforcement Learning (RL), and deep learning in optimizing energy consumption across smart grid infrastructures. We provide an in-depth analysis of ML applications spanning Demand-Side Management (DSM), short-term and Long-Term Load Forecasting (LTLF), renewable energy integration, fault detection and diagnostics, energy storage optimization, and real-time adaptive grid control. This paper reviews and synthesizes findings from over 100 studies published between 2018 and 2025, categorizing ML approaches by their application domain, algorithmic paradigm, and deployment context. We evaluate their performance using standardized metrics including Mean Absolute Percentage Error (MAPE), root mean squared error, cost reduction ratios, and Peak-to-Average Ratio (PAR) improvements. Critical research gaps are identified, including challenges related to training data scarcity and quality, model interpretability in safety-critical infrastructure, scalability from pilot projects to grid-scale deployment, computational constraints for real-time inference, and privacy concerns arising from granular smart meter data. Furthermore, we provide a forward-looking analysis of emerging trends that are poised to reshape the field, including Federated Learning (FL) for privacy-preserving collaborative model training, transfer learning for cross-domain generalization, edge-AI deployment for low-latency on-device intelligence, foundation models for energy time-series, and explainable AI frameworks for regulatory compliance. This review serves as a comprehensive reference for researchers, power systems engineers, and policymakers seeking to understand the current state-of-the-art and chart future research directions at the intersection of ML and smart grid energy optimization.