Language testing is social science as well as practical activity. It has been acknowledged that the fairness of testing should be maintained throughout the entire process of testing. In Chinese context, the evaluation framework on language testing fairness should clarify responsibilities and rights of all stake-holders such as designers, users and testees. To improve the quality of testing, the fairness evaluation should encompass check both on technical fairness and social justice. For the sake of operational application, it is essential to undertake validation, bias review and sensitivity review. It is also suggested that the result of fairness evaluation should be applied to improve fairness under public supervision to ensure it’s ongoing test quality and social justice.