Automated writing evaluation (henceforth AWE) has become an issue of substantial controversies and debates, as well as a source of written feedback that can’t be ignored. So far there are few researches exploring whether and how AWE impacts the complexity of students’ writings. The current study examined the effectiveness of Juku Pigai Website on the lexical and syntactic complexity of students’ writing. The participants were 45 homogeneous average learners in College English class; they were divided into three groups according to three feedback modes, including two experimental groups —a Pigai feedback only and Pigai with human instruction —and a control group using global scoring. Statistical methods, such as two-way repeated measures ANOVA, were used to measure the linguistic properties of the compositions written in 32 weeks on Pigai Website. The result revealed a significant gain for the complexity in both experimental groups using AWE provided by Pigai Website; the experimental group using Pigai Website assisted by human feedback shows greater changes in syntactic complexity than in lexical complexity.