This study aimed to analyze the quality of English summative test taken by the students of MAN Boarding School Meulaboh I in terms of validity, reliability, difficulty index, discrimination index, and the effectiveness of distractors. Content analysis was employed in this study. Two techniques were carried out to collect the data, namely checklist and document analysis. The data from checklist were analyzed by using a statistical procedure in percentage form and the data from document analysis were analyzed by using Anates Software version 4. The results showed that English summative test in the context of content analysis of the first grade has sufficient validity, the second grade has poor validity, and the third grade has a very poor validity. However, in terms of construct validity the first grade and the third grade have poor construct validity, while the second grade has sufficient construct validity. Meanwhile, in terms of validity coefficient, the first grade and the third grade is 60% and the second grade is 44%. Secondly, English summative test of MAN Boarding School has a consistent degree of reliability. The reliability index of the first grade and the second grade test is 0.70 and the third grade is 0.88. Thirdly, 60% of the test in the first grade test, 48% of which in the second grade, and 8% of which in the third grade test are accepted. It means that the third grade has a very poor difficulty index. Fourthly, a good discrimination index of the test in the first grade is 76%, second grade is 56%, and the third grade is 72%. In other words, more than half of discrimination index is good. Finally, the quality of distractors of English summative test of the third grade is not as good as that of the first grade and second grade. The effectiveness of English summative test in first grade is 53%, the second grade is 67%, and the third grade is 50%.