The construction of an efficient and long-term higher education system is an important prerequisite and guarantee for the development of higher education in a country, which is conducive to laying a good intellectual foundation for the enhancement of national strength. However, in the face of the shortcomings of the construction of China’s higher education system, most of the researches are limited to theoretical researches and many researches are one-sided with self-styled thinking, which lack open and inclusive ideas and practical proof to theory. In order to further analyze the bottleneck of China’s higher education development, on the basis of analyzing the characteristics of Chinese and American higher education system, this paper analyzes the difference between Chinese and American higher education system by the method of comparative research and literature research, and adopts the multi-level fuzzy hierarchy comprehensive evaluation method to deal with the relevant data of Chinese and American higher education system. Then, it discusses and contrasts the macro-control function of the American government and the direct control function of the Chinese government, and then puts forward the aspects that China’s higher education system should improve on the basis of fully considering China’s national conditions. It also proposes that while enhancing the independence of educational evaluation, we should improve the Chinese law, change the government functions, and pay attention to the construction of higher education system theory research to build a pluralistic higher education system. Compared with other researches on the China’s higher education development, this paper draws on the advantages of the American higher education system and puts forward the countermeasures and suggestions which are more suitable for the development of higher education system in China.