Deep neural networks (DNNs) have demonstrated significant potential in addressing a wide range of intelligent tasks. However, traditional neural networks deployed on digital platforms face inherent limitations in terms of throughput, computational speed, and energy efficiency, particularly in resource-constrained environments. To address these challenges, a more scalable, faster, and energy-efficient approach is needed for the advancement of deep learning. Optical neural networks (ONNs), which utilize light signals instead of electrical ones as the information carrier for computation, offer such a promising alternative. Among ONNs, free-space diffractive optical neural networks (DONNs) stand out for their high throughput, light-speed computation, and exceptional energy efficiency. They enable all-optical computing at near-light speed by manipulating information-encoded light signals through optical phenomena such as propagation, diffraction, and phase modulation. By leveraging trained passive optical elements, DONNs perform computation without additional energy consumption during all-optical inference.

However, the development and practical deployment of DONNs face several critical challenges. First, the lack of hardware-software co-design algorithms impedes the seamless realization of DONNs, from conceptual design to physical fabrication with analog optical components. Second, the absence of robust emulation frameworks limits system-level applications of DONNs, as designing and exploring DONNs require extensive cross-disciplinary expertise, posing significant technical barriers. Third, current computing engines for DONN emulation and training are computationally intensive, lacking both optimized computing kernels and domain-specific language (DSL) support tailored to ONNs that balances flexibility and maintainability. Fourth, the accessibility of DONN research is limited, necessitating the development of an open-source design infrastructure to facilitate broader community engagement and innovation.

Targeting the improvements and contributions to the development of DONNs, this dissertation presents four key contributions. First, we propose a physics-aware differentiable co-design algorithm designed specifically for DONN systems, enabling the efficient and accurate system training and design automation. Second, we conduct physics-aware optical adversarial investigations, which uncover unique optical security vulnerabilities in ONNs and provide insights into adversarial robustness applicable to other complex-domain systems. Third, we develop an open-source, end-to-end agile design framework, LightRidge, for DONN systems. This framework integrates efficient co-design algorithms, accurate yet high-performance optimized computing kernels, and user-friendly DSL support. It offers a seamless design-to-deployment workflow, bridging the expertise gap for cross-disciplinary research for DONNs. Fourth, we explore DONNs across diverse deep learning applications, including physics-aware multi-task learning, optical-inspired graph learning, and optical-inspired image processing. These applications demonstrate the capability of DONNs for real-world applications and enrich the research landscape of DONNs.