Models

Grader consists of several classes that represent different apsects of the grader. Namely:

• Grader - The whole shebang. This object has members that allow for all kinds of high-level assignment manipulation: creation, grading, etc. The grader can be configured with a YAML file (which is represented with a GraderConfig)

• Assignment - A single assignment

• GradeSheet - A grade sheet for a single assignment. Grade sheets describe how an assignment should be graded, which includes:

  • Grading scripts
  • A Dockerfile for building the execution environment
  • Assignment-specific configuration (via AssignmentConfig) for running and evaluating submitted assignments.

Grader

class grader.models.Grader(path)

A Grader.

This object can be used to perform all of the actions that a Grader can perform.

__init__(path)

Instantiate a Grader.

Parameters:

path (str) – The path to the grader on disk. The path should contain the configuration file for the grader.

Raises:

• GraderError – if the provided path doesn’t exist
• GraderConfigError – if the grader config file cannot be read

classmethod new(path, course_name, course_id)

Creates a new Grader instance and sets up its operating environment. This includes creating associated files on disk.

Parameters:

• path (str) – The path to the directory that will contain the Grader
• course_name (str) – The name of the course. It must comply with GraderConfig.SCHEMA
• course_id (str) – The unique ID for the course. It must comply with GraderConfig.SCHEMA

Raises:

GraderConfigError – if course_name or course_id don’t comply with the configuration schema

assignment_dir

The path to the Grader’s assignment directory.

assignments

All assignments associated with this grader

student_ids

All student IDs from the roster

create_assignment(name, repo=None)

Creates a new assignment directory on disk as well as an associated gradesheet and subdirectories.

Parameters:

• name (str) – The name of the assignment. Must comply with AssignmentConfig.SCHEMA
• repo (str) – An optional URL to a gradesheet repository. Read more about it in Assignment.new()

Returns:

None

get_assignment(name)

Retrieves an assignment.

Parameters:name (str) – The name of the assignment to get Raises:AssignmentNotFoundError – if no such assignment exists Returns:The Assignment Return type:Assignment

delete_assignment(name)

Deletes an assignment from the Grader’s assignments directory.

Parameters:name (str) – The name of the assignment.

Assignment

class grader.models.Assignment(grader, assignment_name)

An Assignment with several neato attributes:

• A place (directory) for storing student submissions for the assignment
• A place (directory) for storing the grade reports for graded submissions
• A place (git repository) for storing grading scripts and configuration files

This class makes dealing with all those files and folders just a little easier. Creating new assignments, grading them, etc.

__init__(grader, assignment_name)

Instantiate a new Assignment

Parameters:

• grader (Grader) – The grader with which this assignment is associated
• assignment_name (str) – The name of the assignment

Raises:

• AssignmentError – if the assignment path (Assignment.SUB_DIR/assignment_name) doesn’t exist, if the submission path doesn’t exist with in the assignment path, if the results path doesn’t exist within the assignment path, or if the directory for the gradesheet repository doesn’t exist
• GradeSheetError – if there was an error constructing the Assignment’s GradeSheet

SUB_DIR = 'assignments'

Name of the subdirectory for assignments

classmethod new(grader, assignment_name, gradesheet_repo=None)

Creates a new Assignment for a Grader. This includes...

• Creating a directory to hold assignments, if necessary.
• Creating a directory within the assignments directory to hold the new assignment.
• Creating subdirectories for submissions and grading results within the assignment directory
• Creating a gradesheet repository by...
  • Cloning from a URL, if provided
  • Initializing a repository with default files

Note

If you provide a repository URL, the Assignment will attempt to clone the repo. You will need pull access to it.

Parameters:

• grader (Grader) – The Grader this assignment belongs to
• assignment_name (str) – The name of this assignment. Must comply with AssignmentConfig.SCHEMA
• gradesheet_repo (str) – The URL of a git repository to clone for the gradesheet. If set to None, a repository with default values will be created.

Returns:

The newly created Assignment

Return type:

Assignment

Raises:

• GradeSheetError – if there was an error creating the GradeSheet
• ConfigValidationError – if there was an error creating the GradeSheet’s assignment-specific config file

image_tag

Unique tag for an assignment’s docker image

image_id

Unique ID for an assignment’s docker image

submissions_dir

File path to the assignment’s submissions directory

submissions

All submissions for this assignment

submissions_by_user

All submissions for this assignment grouped by user

results_dir

File path to the assignment’s results directory

gradesheet_dir

File path to the assignment’s gradesheet repository

build_image()

Build’s an assignment’s docker image using the Dockerfile from its GradeSheet.

The docker image will be tagged, so that this assignment’s image is unique from the rest of the assignment images on a given machine.

Returns:None

delete_image()

Deletes an assignment’s docker image based on its tag.

GradeSheet

class grader.models.GradeSheet(assignment)

A gradesheet for an assignment. Gradesheets are git repositories. They have the following attributes...

A configuration file

Assignment-specific configuration information (refer to AssignmentConfig)

A Dockerfile

Used to build a docker image. That image will be used to create containers for running student submissions

Grading scripts

The assignment-specific configuration details how to run grading scripts. The gradesheet repository must have the scripts in order to run them.

__init__(assignment)

Instantiates a GradeSheet.

Parameters:

assignment (Assignment) – The assignment to which this gradesheet belongs.

Raises:

• AssignmentConfigError – if the assignment-specific config file in the gradesheet cannot be loaded
• GradeSheetError – if the Dockerfile can’t be found

SUB_DIR = 'gradesheet'

The name to use when creating a new gradesheet directory on disk.

classmethod from_repo(gradesheet_path, repo_url)

Clone a gradesheet from a remote git repository.

Parameters:

• gradesheet_path (str) – The path to the directory into which the gradesheet repository will be cloned.
• repo_url (str) – A URL pointing to a gradesheet repository to clone.

Raises:

GradeSheetError – if there was a problem cloning the repo

classmethod new(gradesheet_path, assignment_name)

Initializes a new gradesheet repository with default files

Parameters:

• gradesheet_path (str) – The path to the directory into which the gradesheet repository will be cloned.
• assignment_name (str) – The name of the assignment.

dockerfile_path

The path to this gradesheet’s Dockerfile

Submission

class grader.models.Submission(assignment, tar_name)

An assignment submission.

A Submission essentially represents a .tar.gz file on disk that contains a student assignment submission.

__init__(assignment, tar_name)

Instantiates a new Submission.

Parameters:

• assignment (str) – The Assignment object to which this assignment belongs.
• tar_name (str) – The name of the submission’s .tar.gz file (including extension)

SUBMISSION_ID_RE = re.compile('^(?P<student_id>\\w+)--(?P<uuid>[0-9a-f]{8}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{12})$')

Full submission IDs must match this pattern

classmethod split_full_id(full_id)

Splits a full Submission ID into the student’s ID, and the submission’s UUID

Parameters:full_id (str) – A full submission ID. See SUBMISSSION_ID_RE Returns:The student’s id, and the submission’s UUID Return type:(str, str)

classmethod get_full_id(basename, sid=None, sid_pattern='(?P<id>.*)')

Generates a full Submission ID based on the student’s ID

Parameters:

• basename (str) – The basename of a file to use for generating the submisison id. This string should contain the student’s ID in some way.
• sid (str) – If provided, this will be used as the student’s ID.
• sid_pattern (str) – If provided, this will be used to pull the student’s ID out of basename. The match group named “id” (i.e., match.group('id')) will be used as the student’s id.

Raises:

SubmissionError – If there was an issue with matching basename against sid_pattern

Returns:

The full submission id (for filenames)

Return type:

str

classmethod import_blackboard_zip(assignment, zip_path, sid_pattern='(?P<id>.*)')

Imports submissions from a ZIP downloaded from Blackboard

Parameters:

• assignment (str) – Assignment associated with this submission
• zip_path (str) – Path to the Blackboard ZIP to import
• sid_pattern (str) – An optional pattern to use to convert the name of a submission to a submission id

Returns:

A list of Submissions

classmethod import_multiple(assignment, source, sid_pattern='(?P<id>.*)')

Imports multiple submissions from a folder of .tar.gz‘s or folders of submission material.

By default, the names of the .tar.gz files (minus extensions) will be used as the ID for each submission. For example, if a folder contains hsimpson.tar.gz, a new Submission with id hsimpson will be created.

Parameters:

• assignment (str) – Assignment associated with this submission
• source (str) – Path to the folder of ``.tar.gz``s/folders to import
• sid_pattern (str) – An optional pattern to use to convert the name of a submission to a submission id

Returns:

A list of Submissions

classmethod import_single(assignment, source, sid=None, sid_pattern='(?P<id>.*)')

Imports a single submission.

The submission may be:

• A .tar.gz file.
• A folder containing files to submit. The folder will be compressed as a .tar.gz then imported.

By default the name of the submission (minus extensions) will be used as its ID. For example, if a folder is named hsimpson/, the new Submission with id hsimpson will be created.

Parameters:

• assignment (str) – Assignment associated with this submission
• source (str) – Path to the submission item
• sid (str) – The submission id to use. If None, the basename of source will be used.
• sid_pattern (str) – An optional pattern to use to convert the name of a submission to a submission id

Returns:

A list containing a single item: the new Submission

classmethod import_repo(assignment, repo_url, sid=None, sid_pattern='(?P<id>.*)')

Imports a submission from a git repository

Parameters:

• assignment (str) – Assignment associated with this submission
• repo_url (str) – URL to a clone-able git repository.
• sid (str) – The submission ID for the new submission. If None, the name of the repository (sans .git) will be used.
• sid_pattern (str) – An optional pattern to use to convert the name of a submission to a submission id

Returns:

the new submission

Return type:

Submission

classmethod get_importers()

Returns a dict that maps nicknames for importers to classmethods.

Return type:dict

classmethod get_importer(submission_type)

Returns an importer function for a given submission type. Refer to

the definition of Submission.get_importers() for a list of acceptable values for submission_type.

Parameters:submission_type (str) – The type of submission to import. Returns:a function that will import a submission into Grader Raises:SubmissionError – if no importer can be found

import_time

The time stamp on the Submission (as a datetime object)

file_mtimes

A dict mapping file names (from the submission archive) a datetime object indicating the last time that file was modified

latest_mtime

The time stamp of the most recently modified file in the submission archive (as a datetime object)

sha1sum

The SHA1 sum of the submission archive

unpacked_files

A context manager for working with unpacked submissions. Returns the name of the directory where all files are unpacked. When the context is exited, the directory and its contents are deleted.

unpacked_repo

A context manager for working with unpacked submission repositories. Returns a git.Repo object corresponding to the submission’s repository.

If the submission is not a git repository, then “None” is returned.

latest_commit

The time stamp of the default branch of the submission, or None if the submission isn’t a git repository.

container_labels

The unique label to use for this submission’s docker container

Configuration

class grader.models.config.Config(path)

A base class for configuration objects. Provides two vaguely-useful methods to subclasses.

Requirements:

• The class must have a class-wide CONFIG_FILE_NAME variable, which corresponds to the name of the configuration file (e.g., "config.yml")
• The class must have a class-wide SCHEMA variable, which is a valid JSONSchema that can be used to validate the fields in the Config.
• Instances of the class must have an attribute path, which is the file path to the directory containing the corresponding configuration file.
• Instances of the class must have an attribute data, which is a dict. It should contain data loaded from the corresponding configuration file.

__init__(path)

Instantiates a Config by reading a YAML file

Parameters:path (str) – The path to the directory containing the YAML-formatted configuration file.

__getitem__(name)

Returns the configuration item for key name

Parameters:name (str) – The key to fetch

classmethod new(path, config)

Creates a new GraderConfig object and corresponding config file on disk.

Parameters:

• path (str) – The path to the directory in which the config file will be saved
• config (dict) – A dict of default configurations to store in the new configuration file.

Raises:

if config doesn’t meet the requirements of the Config’s SCHEMA

file_path

The full file path to the corresponding configuration file.

Returns:The path to the configuration file Return type:str

get(name, default=None)

Returns the configuration item for key name, defaulting to default

Parameters:

• name (str) – The key to fetch
• default – The item to return if the key is not found

class grader.models.GraderConfig(path)

A class for creating, loading, and accessing grader-wide, YAML-formatted configuration files

CONFIG_FILE_NAME = 'grader.yml'

The name of the configuration file on disk

SCHEMA = {'type': 'object', 'additionalProperties': False, 'properties': {'course-name': {'type': 'string', 'pattern': '^[\\w-]+$'}, 'roster': {'type': 'array', 'items': {'type': 'object', 'additionalProperties': False, 'properties': {'name': {'type': 'string'}, 'id': {'type': 'string', 'pattern': '^\\w+$'}}, 'required': ['name', 'id']}}, 'course-id': {'type': 'string', 'pattern': '^[\\w-]+$'}}, '$schema': 'http://json-schema.org/schema#', 'required': ['course-id', 'course-name']}

The schema for a Grader-wide configuration file

class grader.models.AssignmentConfig(path)

A class for creating, loading, and accessing assignment-specific, YAML-formatted configuration files

CONFIG_FILE_NAME = 'assignment.yml'

The name of the configuration file on disk

SCHEMA = {'type': 'object', 'additionalProperties': False, 'properties': {'image-build-options': {'type': 'object'}, 'assignment-name': {'type': 'string', 'pattern': '^[\\w-]+$'}}, '$schema': 'http://json-schema.org/schema#', 'required': ['assignment-name']}

The schema for a Assignment-wide configuration file

Mixins

class grader.models.mixins.DockerClientMixin

A mixin class that gives subclasses access to a docker Client object.

docker_cli

A docker Client object. Always returns the same one.