Technical Reports
Components of the Report
For each TR, you are expected to submit the following on Moodle by the deadline:
A pdf of your report, following the structure of this provided outline (.tex, .pdf).
Any code files (.R, .stan, etc.) you used in completing your analyses.
A survey response, explaining which components of the TR you personally contributed to. While you are not expected to each contribute to every component of the TR, you will be expected to be able to answer questions about every component of the TR during the oral assessment (including code), so you must be intimately familiar with your entire submitted report. Both group members must be involved in the methodological development (a.k.a. statistical modeling choices) process.
Peer Review
Peer review is an important part of not only the academic work-flow, but also relevant in industry! Work is not completed in a vacuum. Feedback cycles are central to improving and adapting statistical methodology, written work, and presentations. Additionally, having a close-up look at others’ work can often give us a better sense of things we’d like to improve about our own work.
For each TR, we will have an in-class day for peer review one week prior to the oral assessment. You will conduct peer reviews in your TR pairs, and these will not be done anonymously. While you do not need to have your entire TR completed by the time of peer review, your group must have the following completed and ready for peer review (at a minimum):
A completed analysis plan, including any relevant model statements and justification for modeling decisions
Some code for review. This could be either a pseudo-code outline of a Gibbs sampler, a sample of a .stan file you plan on adapting to your given analysis, and/or exploratory data anlyses.
The more you are able to provide to your peer reviewers, the better feedback you will be able to receive and then update accordingly!
Templates
We’ll follow these templates for peer-review in class (Note: this template is based on peer reviews that I actually conduct for journal articles!). Note that whether or not you “pass” peer-review is not dependent on your reviewer’s recommendation, but is instead based on whether you demonstrated effort in your own referee report and in your submission for peer-review.
Author Comments template (anonymously, for your peers!)
Editor Comments template (not anonymously, for the instructor!)
A heavily redacted and edited example of a referee report for a journal article (Note: the links are only accessible students currently enrolled in STAT 454)
Oral Assessment
One week after peer-review, you and your partner will schedule a 15-minute meeting with Taylor in her office (or on zoom, if needed). For most pairs, this will be during our usual class time. In the oral assessment, you and your partner will work together to support each other and communicate the results of your technical report, as well as justify decisions that you made in terms of statistical modeling choices. Taylor will have read your TR in advance of the meeting.
For the first 3-4 minutes of the assessment, you and your partner will give a big-picture summary of your technical report. You should spend a roughly equal amount of time speaking (no one person should dominate the conversation), and should cover the entire report. Do not get bogged down in details; tell me the motivation for the report, decisions you made, and what your results were.
The remainder of the assessment will consist of Taylor asking you and your partner clarifying questions about specific details of the report. These may have to do with methodological choices made, clarification on writing, or details on the code provided alongside the report. Some questions will be directed at you individually, and others will be asked of you as a team. You should expect to get at least one question on a component of the report that you did not primarily contribute to; for example, if your partner did most of the coding, you will be asked a question about explaining a specific line of code; if your partner wrote the results section, you may be asked to relay the results in different words.
Following the assessment, Taylor will update your feedback spreadsheets with a list of the questions asked in addition to feedback.
Why oral assessments?
First, why technical reports. Written communication is a valuable skill that I want you to practice throughout this course, and since much of the content we will cover in Bayes is related to computation, it’s difficult to assess this in a traditional exam format. I don’t want there to be any time pressure related to coding; hence technical reports. The benefit of doing this in pairs is to have the support of another person when completing the assignment. In both industry and academia, work is consistently done as a team rather than individually, and one goal of this assessment format is to help prepare you for that.
So… why oral assessments in addition to technical reports? Two main reasons. (1) I think it’s valuable for you to receive real-time feedback on your work, and in this setting I can provide that in such a way where we can have a conversation about it, rather than writing some comments on a document and expecting you to read it on your own time. (2) I can prompt you if you need help, and get a better overall understanding of how well you know your own work. If I ask a question and you don’t immediately know the answer, it may just be because I phrased it in a way that’s unfamiliar to you, or it may because you actually don’t know the answer. This is something I hope to be able to distinguish between. In an oral assessment setting, we can both adapt to each other as the conversation continues, and I’ll (hopefully) get a good sense for what material you know well and what you need more practice with.
How to prepare
With your partner and individually, review your technical report and code. Make sure you know why you made certain modeling decisions, what your visualizations / tables display, and the limitations of your analysis. Make sure the code makes sense to you, including specific functions used (particularly if you used AI to generate any code… you will be asked about specifics)
Openly think and discuss how you’ll both make sure your opening discussion of the report is equitable in terms of time and contributions within the pair of you
Openly think and discuss how you want to be supported by your partner if you get flustered
Rubric
The technical report rubric can be found in a tab on your feedback/grade spreadsheets. You will be assessed on five components:
Technical communication
Clarity and structure
Methodological Appropriateness
Effective Visualization(s) and Summary Tables
Oral Examination
Each of the 5 components will be qualitatively marked as “High Pass”, “Pass”, “Low Pass”, or “Insufficient / Not attempted”. If you do not attend the oral examination or contribute to a TR, you will not pass the course. Note that while your technical reports are completed in pairs, your score may be different than your partner’s based on the oral examination and/or contribution survey responses.
Technical Report 1
Details posted two-weeks prior to deadline.
Technical Report 2
Details posted two-weeks prior to deadline.