From 40ac4359abad202570c4936f6466db2ed98dae23 Mon Sep 17 00:00:00 2001 From: Justin Hsu Date: Mon, 24 Aug 2020 18:19:45 -0500 Subject: [PATCH] More on reviewing. --- website/docs/assignments/reviews.md | 53 +++++++++++++++++++++++------ website/docs/index.md | 20 ++++++++--- 2 files changed, 57 insertions(+), 16 deletions(-) diff --git a/website/docs/assignments/reviews.md b/website/docs/assignments/reviews.md index 3843eba..3281e46 100644 --- a/website/docs/assignments/reviews.md +++ b/website/docs/assignments/reviews.md @@ -3,22 +3,28 @@ !!! attention * Paper reviews are for students in time zones who **cannot** attend live lectures. - * Students who are able to attend live lectures should complete a paper - presentation and presentation summary instead. + * Students who are able to attend live lectures are required to complete a + paper presentation and presentation summary instead. -Students who cannot attend live lectures will complete **one paper review per -week** (10 in all). We will be using HotCRP---standard conference management -software---to manage review uploads and discussions. Reviews must be uploaded -**before the paper is presented in class**. +Starting from the first week of paper presentations (**September 14**), students +who cannot attend live lectures will complete **two paper reviews per week**, +**16** in all. We will be using HotCRP---standard conference management +software---to manage reviews. Reviews must be uploaded **before the paper is +presented in class**. + +The HotCRP instance for this course is available here: + +- ## What makes a good review? A good review accomplishes several things: - It **summarizes** the main contributions of the paper. -- It highlights **strengths and weaknesses** of the paper. +- It highlights **strengths and weaknesses** of the paper. Note that these + points do not need to be purely technical. - It **evaluates** the paper, explaining why the reviewer thinks the paper is - strong or weak. + strong or weak, interesting or not interesting. - It gives authors **suggestions to improve** the paper. ## FAQ @@ -26,12 +32,37 @@ A good review accomplishes several things: - **Can I switch from doing paper presentation/summary to paper reviews or vice versa?** No: if you are doing paper reviews, you must let me know on the first week of -class. +class so that I can add you as a reviewer to HotCRP. + +- **How long should reviews be?** + +You should aim for around 400 words, total. We will not be counting words, but +if your review is three sentences long we will probably not be too happy. - **Are late reviews accepted?** No: reviews must be uploaded before the paper is presented in class. -- **Can I submit more than one review a week?** +- **Can I submit more than two reviews a week?** -No: should submit exactly one review per week. +No: should submit exactly two reviews per week. + +- **I was not sure how to evaluate the paper: what should I do?** + +You should say so, and explain the strengths and weaknesses of the paper. + +- **It takes me too much time to read through two papers. What should I do?** + +Given the short reviewing schedule, you will not have time to read through every +single word in every single paper. Instead, you should skim over parts that are +not so crucial. More concretely, you should do the first **two passes** of the +three-pass system described +[here](http://ccr.sigcomm.org/online/files/p83-keshavA.pdf)---the third pass is +**not** required. + +- **I found a review of the paper online. Can I look at it for inspiration?** + +Definitely not: doing so is an academic honesty violation. Anyways, there is +absolutely right or wrong conclusion when reviewing a paper---the idea is to +give **your opinion** of the paper based on **your understanding**, and then +argue why your opinion is correct. diff --git a/website/docs/index.md b/website/docs/index.md index 63c3b50..be18682 100644 --- a/website/docs/index.md +++ b/website/docs/index.md @@ -52,16 +52,16 @@ Grades will be posted on Canvas. - Milestone 2: **10%** - Final project: **50%** -Everything except the final project will be graded on a simple **3-point** -scale: below expectations, meets expectations, or exceeds expectations. The +Everything except the final project will be graded on a simple scale: no +submission, below expectations, meets expectations, or exceeds expectations. The final project will be graded on a **10-point** scale. ## Academic Honesty **Writing is a central part of this course.** All students are expected to follow academic honesty standards. In brief: all the text that you submit must -be **in your own words**, and you are not allowed to copy anything (from a -paper, or from someone else) without full attribution. +be **in your own words**, and you are not allowed to copy anything---from a +paper, or from someone else---without full attribution. If you are completing paper reviews, you should not search for reviews that may be online---this is expressly **against the course policies**. You should @@ -69,7 +69,7 @@ complete the review as if you were seeing the paper for the first time. ## Piazza -We will be using Piazza to discuss papers, ask questions, and find group +We will be using **Piazza** to discuss papers, ask questions, and find group members: - @@ -77,6 +77,16 @@ members: You can also contact me directly. To ensure that your email goes to the right place, please start the subject with **CS763**. +## HotCRP + +If you are not able to attend live lectures and you are completing paper +reviews, we will be using **HotCRP** to organize paper reviews. HotCRP is the +software used to manage most conferences in computer science. Through this site, +you will be able to submit reviews. We have set up a mock HotCRP for this +course: + +- + ## Course Staff - **Instructor**: [Justin Hsu](https://justinh.su)