The free trade zones undertake the mission of promoting the optimization and upgrading of the industrial structure. By exploiting the panel data of China’s 169 prefectural-level cities during 2006~2018, this paper combines the Propensity Score Matching and Difference in Differences (PSM-DID) to evaluate the influence posed on the upgrading of the industrial structure by the establishment of free trade zones. According to the research, the free trade zones exert an apparent impact on the promotion and rationalization of the urban industrial structure. There are regional and batch heterogeneity in the upgrading effect of industrial structure in free trade zones. resulting in higher optimizing effect in coastal free trade zones than the in-land ones and more visible rationalizing effect of the former. The rationalization of industrial structure in the first batch of free trade zones has the strongest promotion effect, while the upgrading of industrial structure in the third batch has the strongest promotion effect. The establishment of free trade zones mainly stimulates the upgrading of the industrial structure by facilitating trade and investment, but the other approach to upgrading, enhancing regional innovation, has not been fully utilized.