In this thesis, we investigate various aspects of tumor progression through formation, growth, and invasion, by a multidisciplinary approach involving mathematical modeling and experimental validation. We begin this study by modeling the transient formation of tumors by a system of reaction-diffusion partial differential equations (PDEs) that considers adhesion forces, cell proliferation, and pressure-induced growth. The process of tumor formation includes a preliminary contraction phase where adhesion forces densify cell aggregation. This phase proceeds until the cell concentration reaches a threshold, the so-called “relaxed concentration” at equilibrium. Afterwards, cell proliferation raises concentration and produces pressure which breaks the equilibrium. Providing analytical and numerical solutions, the model’s reliability is confirmed through experiments with tumor-cultured human glioblastoma (hGB) cancer cell lines. We expand the model to analyze the instability of radially symmetric growth in response to asymmetric perturbations. By improving the model to incorporate additional variables such as nutrient concentration, consumption rates, and surface tension, we focus on the asymmetric modes of growth, which grow in time and change the spherical configuration of the tumor. We show that a high nutrient source concentration allows for a large tumor size, which increases the number of unstable excited asymmetric modes. However, high rates of nutrient consumption and surface tension can lead to a smaller size of the tumor and a smaller number of growing asymmetric modes. This analysis, indicating the natural instability of the spherical configuration of tumor was confirmed by a comparison between the shapes of in-vitro hGB tumors and the configuration of the first few asymmetric modes predicted by the model.

To further understand the effect of tumor microenvironment (TME) on tumor configuration, we study biomechanical stimulus-induced remodeling of tumors in response to gradients of external biochemical stimuli, considering the tumor as an evolving material. We develop an evolution law for the remodeling-associated deformation which correlates the remodeling to a characteristic tensor of external biochemical stimuli. The asymmetric remodeling and the induced mechanical stresses are analyzed for different types of biochemical distributions. Using a tumor-on-a-chip platform, the degree of remodeling is estimated for the ellipsoidal tumors over time. Additionally, we explore invasion as one of the key hallmarks of tumors by introducing a continuum model that integrates various factors to predict a distinctive shell-type invasion pattern in which cells at the outer layer of the tumor collectively move away from the core and form a shell-type shape. We adopt a non-convex free energy that allows for phase separation to model the motion of the invasive shell.

To develop a more realistic model, we extend our mathematical framework to include heterogeneities within a tumor as they play a crucial role in cancer diagnosis, treatment, and prognosis. We present a hybrid discrete-continuum (HDC) model incorporating experimental measurements and in-vitro tumor-on-a-chip platforms to study tumor growth, invasion, and their dependency on matrix stiffness. The model integrates the continuum field of variables with a discrete approach and incorporates the random walk method for individual cell migration. Moreover, we study the influence of matrix stiffness on tumor growth and invasion using a PEGDA-printed tumor-on-a-chip platform. The presented framework is capable of distinguishing the growth and invasion of non-resistant versus chemo-resistant tumors, as well as the inhibitory effect of a chemotherapeutic drug. We also show that U251 non-resistant tumors grow faster compared to the temozolomide (TMZ)-resistant tumors, whereas the TMZ-resistant tumors have the longest invasion length. We utilize a stochastic approach that is consistent with observed biological behavior and provides a more realistic representation of the invasion process. This hybrid model, validated against an in-vitro co-culturing of non-resistant and TMZ-resistant hGB tumors with healthy neurons all embedded within a hydrogel matrix, shows promise in quantitative predictions on volumetric growth, invasion length, and invasion patterns of tumors.

Our study concludes by highlighting the comprehensive understanding achieved through analytical modeling, experimental validation, and hybrid modeling techniques. The findings lay the groundwork for future investigations into therapeutic interventions, considering the intricate interplay between biological and mechanical factors in the TME.