More on reviewing.

website/docs/assignments/reviews.md

@@ -3,22 +3,28 @@
 !!! attention
     * Paper reviews are for students in time zones who **cannot** attend live
       lectures.
-    * Students who are able to attend live lectures should complete a paper
+    * Students who are able to attend live lectures are required to complete a
-      presentation and presentation summary instead.
+      paper presentation and presentation summary instead.
 
-Students who cannot attend live lectures will complete **one paper review per
+Starting from the first week of paper presentations (**September 14**), students
-week** (10 in all). We will be using HotCRP---standard conference management
+who cannot attend live lectures will complete **two paper reviews per week**,
-software---to manage review uploads and discussions. Reviews must be uploaded
+**16** in all. We will be using HotCRP---standard conference management
-**before the paper is presented in class**.
+software---to manage reviews. Reviews must be uploaded **before the paper is
+presented in class**.
+
+The HotCRP instance for this course is available here:
+
+- <https://wisc-cs763-20.hotcrp.com/>
 
 ## What makes a good review?
 
 A good review accomplishes several things:
 
 - It **summarizes** the main contributions of the paper.
-- It highlights **strengths and weaknesses** of the paper.
+- It highlights **strengths and weaknesses** of the paper. Note that these
+  points do not need to be purely technical.
 - It **evaluates** the paper, explaining why the reviewer thinks the paper is
-  strong or weak.
+  strong or weak, interesting or not interesting.
 - It gives authors **suggestions to improve** the paper.
 
 ## FAQ
@@ -26,12 +32,37 @@ A good review accomplishes several things:
 - **Can I switch from doing paper presentation/summary to paper reviews or vice versa?**
 
 No: if you are doing paper reviews, you must let me know on the first week of
-class.
+class so that I can add you as a reviewer to HotCRP.
+
+- **How long should reviews be?**
+
+You should aim for around 400 words, total. We will not be counting words, but
+if your review is three sentences long we will probably not be too happy.
 
 - **Are late reviews accepted?**
 
 No: reviews must be uploaded before the paper is presented in class.
 
-- **Can I submit more than one review a week?**
+- **Can I submit more than two reviews a week?**
 
-No: should submit exactly one review per week.
+No: should submit exactly two reviews per week.
+
+- **I was not sure how to evaluate the paper: what should I do?**
+
+You should say so, and explain the strengths and weaknesses of the paper.
+
+- **It takes me too much time to read through two papers. What should I do?**
+
+Given the short reviewing schedule, you will not have time to read through every
+single word in every single paper. Instead, you should skim over parts that are
+not so crucial. More concretely, you should do the first **two passes** of the
+three-pass system described
+[here](http://ccr.sigcomm.org/online/files/p83-keshavA.pdf)---the third pass is
+**not** required.
+
+- **I found a review of the paper online. Can I look at it for inspiration?**
+
+Definitely not: doing so is an academic honesty violation. Anyways, there is
+absolutely right or wrong conclusion when reviewing a paper---the idea is to
+give **your opinion** of the paper based on **your understanding**, and then
+argue why your opinion is correct.

website/docs/index.md

@@ -52,16 +52,16 @@ Grades will be posted on Canvas.
     - Milestone 2: **10%**
     - Final project: **50%**
 
-Everything except the final project will be graded on a simple **3-point**
+Everything except the final project will be graded on a simple scale: no
-scale: below expectations, meets expectations, or exceeds expectations. The
+submission, below expectations, meets expectations, or exceeds expectations. The
 final project will be graded on a **10-point** scale.
 
 ## Academic Honesty
 
 **Writing is a central part of this course.** All students are expected to
 follow academic honesty standards. In brief: all the text that you submit must
-be **in your own words**, and you are not allowed to copy anything (from a
+be **in your own words**, and you are not allowed to copy anything---from a
-paper, or from someone else) without full attribution.
+paper, or from someone else---without full attribution.
 
 If you are completing paper reviews, you should not search for reviews that may
 be online---this is expressly **against the course policies**. You should
@@ -69,7 +69,7 @@ complete the review as if you were seeing the paper for the first time.
 
 ## Piazza
 
-We will be using Piazza to discuss papers, ask questions, and find group
+We will be using **Piazza** to discuss papers, ask questions, and find group
 members:
 
 - <https://piazza.com/class/ke3clkclul16hq>
@@ -77,6 +77,16 @@ members:
 You can also contact me directly. To ensure that your email goes to the right
 place, please start the subject with **CS763**.
 
+## HotCRP
+
+If you are not able to attend live lectures and you are completing paper
+reviews, we will be using **HotCRP** to organize paper reviews. HotCRP is the
+software used to manage most conferences in computer science. Through this site,
+you will be able to submit reviews. We have set up a mock HotCRP for this
+course:
+
+- <https://wisc-cs763-20.hotcrp.com/>
+
 ## Course Staff
 
 - **Instructor**: [Justin Hsu](https://justinh.su)