Create a basic autograder (helloworld)
The autograder is the name for the code package you write which will grade the student submission code and give back a grade, among other types of feedback. The best way to explain how to build an autograder is to show an example – in this case we will be building an autograder which tests a student’s ability to print out “Hello World” to standard output. The example will be in Java code.
Refer here to get a sense of the components involved
- First, you must create the two grading files: autograde.tar and autograde-Makefile
- autograde.tar
- The tarfile contains our “grader code” which needs at least one file with a “main” function to execute.
- See our example TestHelloWorld.java here: (link)
- The essential steps to include are to:
- Read in the student submission file (which likely must match some exact filename or class name)
- Run the student file and run as many tests against it as you wish (more on the intricacies of testing in this way later)
- Calculate a final score and then print it in the needed format (link needed)
- Create the tarfile through whatever means you wish:
tar -cf autograde.tar TestHelloWorld.java
- The tarfile contains our “grader code” which needs at least one file with a “main” function to execute.
- autograde-Makefile
- The makefile is meant to mostly do the following:
- Extract the tarfile package (autograde.tar)
- Compile therelevant files found therein
- Execute the test class (the main entry point into the autograder code)
- It will will look something like the below:
@tar -xf autograde.tar
@javac -cp . *.java
@java -cp . TestHelloWorld
- The makefile is meant to mostly do the following:
- autograde.tar
- You now have an autograde.tar and an autograde-Makefile — these should be uploaded into your assessment via:
- Assessment home page -> Admin Options -> Edit Assessment -> Modules Used -> Autograder (hit the plus sign to create, or pencil sign to edit) -> Click the buttons to upload your files
- You can then return here via: Admin Options -> Autograder Setting
Test your autograder:
- Once your autograder is configured for your assessment, you can test it by making a test submission
- After being graded, you will be sent to the “History” page, and are able to look at the feedback generated by your autograder code
- Keep refreshing the page until the autograding process is complete and Autolab receives the feedback
- Then, click the link which appears in the row for the submission you just made. The link will likely either be a dash or a number grade
- It is possible that you instead see what is referred to as a “traffic light filter,” but this would be have to have been configured earlier – see Advanced Options (need link)
- Note: also visible here will be links to view additional feedback types: Hints, Reports, Alerts. For more on these, see Feedback (link needed)
- You should then be taken to a page titled “Feedback for – <your name/email>”
- This page shows the “raw” feedback generated by your autograder code. You can check here to see if you get the expected results.
- Some common issues usually involve one or more of the following:
- Failure to make or build the package
- Not linking proper libraries/jars, etc
- Differing in binaries on the autograding system (i.e. earlier version of java, resulting in compilation failure)
- For more on these types of issues – check the Debugging page (link needed)