Programming Puzzle: Exam-Cheating Codes

This is an archived version of this module from Spring 2022.
Click here for the current iteration.

This continuous-assessment exercise consists of a small applied project with algorithmic and programming components, including a real-time leaderboard of the competition.

Will you be able to beat your classmates, or even your demonstrator?

As for the Bamboo Trimming Problem, this assignment requires more thinking than coding, and the focus is as much on the process of finding solutions as on the actual outcome.

Exam Cheating Codes

Final exams are fast approaching. While you are perfectly well prepared and are eagerly awaiting examination day, your significant other is much less relaxed about an upcoming exam. Actually, desperate is more the appropriate term. You decide to help.

The most frightening test, COMP666 👿, is a multiple-choice exam with 20 questions, and each question requires a single Yes/No answer. You are confident that you can perfectly answer any possible question on behalf of your better half, but of course, the main challenge is how to communicate the answers to your loved one after you left the exam. (To your great relief, the instructor chose not to randomize questions or shuffle answers…)

Clearly, you are not able to enter the exam hall once you have left, but you arrange with your significant other the following code: Honking once (on your moped parked in the side street next to the building) will mean Yes, quickly honking twice means No. Unfortunately, as you figured out from painful experimentation, you have only time to send 10 Yes/No signals before campus police politely removes you (and your irritating source of noise) from university premises.

Without knowing the possible answers in advance, what is the best mark you can achieve for your sweetheart in expectation?

Formalization

Clearly, the correct answers to the 20 Yes/No questions of the exam can be given as an array of 20 bits $B[0..20)$, where $B[i]=1$ (for $i=0,\ldots,19$) means that the correct answer to the $i$th question is Yes, whereas $B[i] = 0$ means the answer is No.

Likewise, your honking “message” can be encoded as an array of 10 bits $H[0..10)$.

The task now consists in finding an encoding function $e:\lbrace 0,1\rbrace^{20} \to \lbrace0,1\rbrace^{10}$ and a decoding function $d:\lbrace0,1\rbrace^{10} \to \lbrace0,1\rbrace^{20}$ for you respectively your significant other to memorize. Your goal is to get the best expected mark for your sweetie when all possible exam answers are equally likely, i.e., to find $d$ and $e$ so that \[ \sum_{B \in \lbrace 0,1\rbrace^{20}} \frac 1{2^{20}}\, d_H(d(e(B)),B)\] is as small as possible. Here, $d_H$ denotes the Hamming distance. (The sum is taken over all possible bit arrays of length 20.)

Inputs

Your protocol has to work for any possible set of correct answers $B$, and it will be tested on all of them.

Design as good a protocol as you can find!

Code template

We prepared a Python implementation of the exam sitting that you will use to evaluate your protocol:

• Python sources

Obey the comments! Once you have downloaded the code, replace the marked sections in examCheatingCode.py with

1. code for your encoding function and
2. code for your decoding function.

To run the simulation, extract the zip archive to a folder and run python3 sitExam.py there. The output is the average mark (average number of correctly answered questions) that your loved one could achieve with the help of your protocol.

Deliverables

For how and where to submit, please check Canvas.

This is an individual project; each student has to submit his or her own solution comprising the following:

1. The examCheatingCode.py source file.
2. A report of at most 2 pages containing 3 sections:
   1. a clear statement of your code’s achieved average exam mark,
   2. a description of how your code works,
   3. and a lower bound argument, showing that a certain quality is not possible.

Please upload the files in a single zip archive.

Marking Scheme

The overall mark will consist of a weighted average.

• 60% for the report, out of which
  • 40% are for the lower bound and
  • 20% are for the description of how your solution works;
• 40% for the quality of the achieved solution.
  The baseline is the solution that Ben has found; in principle you can get more than 100% for this subtask if you manage to beat his solutions!

Collaboration

This programming puzzle is mainly an individual project, and you have to submit you own solution. In particular, the description of your solution must be a single-author document.

Collaboration in small groups (not more than five students) on a conceptual level (discussing ideas, not sharing entire solutions) is accepted, but it must be declared in the description document, including proper mention of others’ contributions.

Leaderboard

We run a (voluntary, anonymous) leaderboard of the current best solutions. When you have a code tried in the simulation, use the below form to share your achievement with the rest of the class!

Highscores

The plots below show all answers over time. Recall that higher is better.

New submissions are immediately added at the right end, but might take a few seconds and refreshing before they show up.