Skip to Main Content

AutoLab Documentation

This is your site's tagline and can be changed

Create a basic autograder (helloworld)

The autograder is the name for the code package you write which will grade the student submission code and give back a grade, among other types of feedback. The best way to explain how to build an autograder is to show an example – in this case we will be building an autograder which tests a student’s ability to print out “Hello World” to standard output. The example will be in Java code.

Refer here to get a sense of the components involved

  • First, you must create the two grading files: autograde.tar and autograde-Makefile
    • autograde.tar
      • The tarfile contains our “grader code” which needs at least one file with a “main” function to execute.
        • See our example TestHelloWorld.java here: (link)
        • The essential steps to include are to:
          • Read in the student submission file (which likely must match some exact filename or class name)
          • Run the student file and run as many tests against it as you wish (more on the intricacies of testing in this way later)
          • Calculate a final score and then print it in the needed format (link needed)
        • Create the tarfile through whatever means you wish: 
          • tar -cf autograde.tar TestHelloWorld.java
    • autograde-Makefile
      • The makefile is meant to mostly do the following:
        • Extract the tarfile package (autograde.tar)
        • Compile therelevant files found therein
        • Execute the test class (the main entry point into the autograder code)
      • It will will look something like the below:
        • @tar -xf autograde.tar 
          @javac -cp . *.java
          @java -cp . TestHelloWorld
  • You now have an autograde.tar and an autograde-Makefile — these should be uploaded into your assessment via:
    • Assessment home page -> Admin Options -> Edit Assessment -> Modules Used -> Autograder (hit the plus sign to create, or pencil sign to edit) -> Click the buttons to upload your files
    • You can then return here via: Admin Options -> Autograder Setting

Test your autograder:

  • Once your autograder is configured for your assessment, you can test it by making a test submission
  • After being graded, you will be sent to the “History” page, and are able to look at the feedback generated by your autograder code
  • Keep refreshing the page until the autograding process is complete and Autolab receives the feedback
  • Then, click the link which appears in the row for the submission you just made. The link will likely either be a dash or a number grade
    • It is possible that you instead see what is referred to as a “traffic light filter,” but this would be have to have been configured earlier – see Advanced Options (need link)
    • Note: also visible here will be links to view additional feedback types: Hints, Reports, Alerts. For more on these, see Feedback (link needed)
  • You should then be taken to a page titled “Feedback for – <your name/email>”
    • This page shows the “raw” feedback generated by your autograder code. You can check here to see if you get the expected results.
    • Some common issues usually involve one or more of the following:
      • Failure to make or build the package
      • Not linking proper libraries/jars, etc
      • Differing in binaries on the autograding system (i.e. earlier version of java, resulting in compilation failure)
    • For more on these types of issues – check the Debugging page (link needed)