The influence of the MnS plastic inclusion on the accumulation of internal damage was considered, and the Gurson–Tvergaard–Needleman (GTN) model was calibrated based on the finite element inverse method and image analysis method using ABAQUS and GTN models. The modified GTN damage model was used to simulate the initiation and propagation of cracks in an as-cast 304 stainless steel with MnS inclusions at 900 °C. The simulation results agreed well with the experimental results, indicating that the model can be effectively applied to examine the high-temperature fracture behavior of MnS inclusions. The simulation and high-temperature tensile test results revealed that MnS inclusions increased the number of holes initiation and the probability of hole polymerization, reduced the crack propagation resistance, accelerated the occurrence of material fracture behavior, and were closely related to the stress state at high temperatures. When the stress triaxiality was low, the plastic strain in the metal matrix was high, and the MnS plastic inclusions accelerated the polymerization of the pores, making metal fracture failure more likely. On the other hand, when the stress triaxiality was high, the stress state in the metal matrix was biased to the tensile state, the plastic strain in the metal matrix was low, and the influence of MnS plastic inclusions on the fracture behavior was not evident.