Skip to content

Latest commit

 

History

History
248 lines (193 loc) · 13.9 KB

assignments.md

File metadata and controls

248 lines (193 loc) · 13.9 KB

General assignment information

Getting started

To edit/execute a Homework/lecture notebook:

  1. Open the page for the Homework/Lecture on this site.
  2. Click the launch button (🚀) at the top. {% if id == "nyu" -%}
    • {{coding_env_name}} may take a few minutes to start up. {%- endif %}
  3. You should now see the notebook in {{coding_env_name}}. {% if id == "columbia" -%}
  4. Make sure you're using your Columbia Google account. How to switch.
  5. Click Copy to Drive. {% endif -%}

That is now your own copy; make edits in there directly.

Tips

{% if id == "nyu" -%}

  • All lecture slides and homework templates can be found under python-public-policy/. The contents of this directory will be automatically updated from the GitHub repository, but should keep any changes you make.
  • Access {{coding_env_name}} via the links on this site rather than bookmarking {{coding_env_name}}, as that will pull down the latest changes. {% endif -%}
  • Read the instructions carefully. Like word problems from math class, they are very specific in what they are asking for.
  • Spot check your results. If you are transforming data from a previous Step, compare the results, do a handful of the calculations manually, etc. to ensure that the results are correct.
  • You are always welcome to add cells. You probably don't want more than a few lines of code in each. This makes the spot checking of intermediate results easier.
  • Don't repeat yourself (DRY). If you find yourself copying and pasting code within a notebook, there's probably a better way to do it.
  • Avoid hard-coding values. Don't rely on things like row numbers or column order being stable, in case the dataset were to be updated.
  • Ask for help.
    • The assignments are meant to be challenging, not impossible.
    • Try and work through problems on your own to start. If you are stuck for more than a half hour, step away. If you still can't figure it out, ask for help.

Storing data

{% if id == "columbia" -%} To keep data between sessions, you'll probably want to store it in Google Drive.

  1. Upload the file(s) somewhere in Drive.
  2. In the Colab sidebar, click the Files icon.
  3. Click the Mount Drive icon.
    • Think of this as attaching your Drive to your Colab instance, as if you were plugging in a USB flash drive.
  4. Navigate to the file.
  5. Next to the filename, click the three dots.
  6. Click Copy path.
    • The value should be something like /content/drive/My Drive/....
  7. Use this path with read_csv(). {% else -%}
  8. Open the {{coding_env_name}} file browser.
  9. Navigate to the folder your notebook is in.
  10. Upload the data.
  11. From Python, use read_csv("./<filename>.csv").

Note that that file path should be to relative to the notebook within {{coding_env_name}} — ./ means "in the same directory". {{coding_env_name}} cannot access the file on your local machine; in other words, the path shouldn't start with C:\\ or anything like that. More info about file paths

Limits

{{coding_env_name}} has a disk storage limit of 1GB (a.k.a. 1,024 MB or 1,048,576 KB) across all your files, and a memory limit of 3GB. {%- endif %}

Reducing data size

You can make data smaller before uploading by filtering it through:

{% if id == "nyu" -%}

Generative AI

See also: course generative AI policy.

We will be using the AI magic.

Documentation

Note that we only have the gpt-4.0-32k model enabled.

Usage

  1. One-time setup: Visit projects.rit.nyu.edu and accept the Terms of Use.

  2. Open {{coding_env_name}}.

  3. Load the models by running the following in a Code cell:

    %load_ext rit_jupyter_ai_magics

  4. In another cell, run:

    %%ai gpt-4.0-32k
how are you?

  5. Replace how are you? with your prompt. You can repeat that cell as many times as you need. {%- endif %}

Submission

  1. Ensure all the outputs are visible and the notebook is cleaned up.
    • This is a good time to run the notebook end-to-end with Restart and run all{% if id == "nyu" %} (⏩){% endif %}.
    • See general scoring criteria. {% if id == "columbia" -%}
  2. Confirm that the notebook is shared with the instructor and {{assistant_name}} with Commenter permissions. If it isn't, share the parent folder and re-confirm.
  3. Copy the URL of your notebook.
    • The URL should be of the format https://colab.research.google.com/drive/<long identifier>. If it's https://colab.research.google.com/github/..., click Copy to Drive.
  4. Paste your notebook URL in the {{lms_name}} Assignment.

Engaging with comments left in {{coding_env_name}} is more than welcome. After the late submission deadline for a given Homework passes, the solution will be shared. {% else -%}

  1. Export the notebook as a PDF. From the Jupyter interface, go to:
    1. File
    2. Save and Export Notebook As…
    3. PDF
    4. You may need to allow popups
  2. Glance through the PDF to ensure everything is showing up as you intend.
    • In particular, check your visualizations.
    • What you see is what the instructors will see.
    • Troubleshooting tips
  3. If one of the Homeworks: Upload the PDF to the {{lms_name}} Assignment.
  4. If the Final Project:
    1. In {{lms_name}}, go to Content, then Final Project. You should see the TurnItIn/PeerMark dashboard.
    2. Follow these instructions to upload the PDF.

When you're ready to have it formally re-graded, please resubmit through the same Assignment in {{lms_name}}. {%- endif %}

Note: In-class exercises will not be graded.

Common issues

{% if id == "columbia" -%}

  • Mounting Google Drive is slow or fails: See the Google Colab help page.

  • Can't load a file from Drive with requests.get(): Use open().

  • Error: Command '['git', 'diff', '..origin/nyu', '--name-status']' returned non-zero exit status 128 when trying to launch {{coding_env_name}}: Something got corrupted in your copy of the files that come through nbgitpuller. Easiest thing is to move the existing ones and restart with a fresh copy.

    1. Launch {{coding_env_name}}.
    2. There should be a python-public-policy folder. Rename it, something like python-public-policy-old.
    • If you've edited any of the template notebooks, you can find them in that -old folder.
    1. Repeat the getting started steps. {% endif -%}

  • When using choropleth_map(), nothing appears on the map: Make sure:

    • Your locations corresponds to the DataFrame column name and featureidkey is set to properties.<property name> matching the GeoJSON
    • The column and the GeoJSON properties have values that match

  • SettingWithCopyWarning: How to fix

  • input() stuck: Jupyter can be a bit buggy when dealing with interactive input. If it seems to get stuck or you aren't seeing a prompt when you'd expect one, try clicking the {{coding_env_kernel_name.title()}} menu then Restart Kernel.

  • The values are out of order along the axis of a Plotly chart: Make sure that:

  • AttributeError: partially initialized module 'orjson' has no attribute 'OPT_NON_STR_KEYS': Add the following cell before your other Plotly code, then Restart and run all{% if id == "nyu" %} (⏩){% endif %}.

    import plotly.io

plotly.io.json.config.default_engine = 'json'

{% if id == "nyu" -%}

Disk full

If you get an error of Disk is full / No space left on device / Out of diskspace: You've used all the available disk space. If you do fill it up, your server may not be able to start again (spawn failed). You'll need to delete one or more large files that you don't need anymore:

  1. If you server is started already (you're seeing notebooks), click Control Panel -> Stop My Server.
  2. Go to start your server again.
  3. Select Troubleshooting Only - Clear Disk.
  4. Look at the File size Jupyter shows in the file browser.
  5. Delete one or more large files.
  6. If you're still using those datasets, make them smaller.

Error loading notebook

This error can happen if you tried to output a lot of data in tables/charts. Steps to resolve:

  1. Open the {{coding_env_name}}) file browser

  2. Click New, then Terminal

  3. Run the following, changing the path at the end to match whatever notebook needs to be repaired:

    jupyter nbconvert --to notebook --clear-output ~/python-public-policy/hw_<NUMBER>.ipynb

If you're confused by these instrucions, download the notebook file and email to the instructor. {%- endif %}

{{coding_env_kernel_name}} and memory issues

The {{coding_env_kernel_name}} is the place where Python is installed and the code is actually executing, in the cloud somewhere.

{% if id == "nyu" -%}

  • Make sure Python [conda env:python-public-policy] is selected as the {{coding_env_kernel_name}}.
    • Shows in the top right of the notebook interface
    • To change:
      1. Open the {{coding_env_kernel_name.title()}} menu
      2. Click Change {{coding_env_kernel_name}}
      3. Click Python [conda env:python-public-policy] {%- endif %}
  • If your {{coding_env_kernel_name}} is repeatedly crashing, you're probably running out of memory.
    • Make sure you aren't loading data sets you don't need.
    • If loading a new dataset, make it smaller {% if id == "nyu" %}- Close {{coding_env_kernel_name}}s you aren't using from the Running page.{% endif %}

{% if id == "nyu" -%}

PDF export

Jupyter notebook export to PDF is fragile, especially with interactive charts through Plotly.

If you get a 500 error, it could be happening for a handful of reasons. Scroll to the bottom and read the message.

  • If it mentions Undefined control sequence and \pandocbounded, it's an issue with the exporter. Jump to the Alternatives.
  • If it mentions a package being missing, ask on Ed.
  • Otherwise, it may be crashing due to not being handle the complexity of the render. Try reducing your output (in the Jupyter sense) to smaller subsets. This can include:
    • Not displaying so many rows/values
    • Reducing the number of points that are plotted

Alternatives

If you are unable to get the direct-to-PDF export working, try:

  • Going through HTML:
    1. Export the notebook as HTML.
    2. Open the HTML file in your browser.
    3. File->Print…
    4. Save as PDF
    5. If any of the visualizations are cut off:
      1. Cancel the Print to PDF.
      2. Viewing the HTML file, make the browser window narrower.
        • This forces the visualizations to re-draw.
      3. Try from File->Print… again.
  • An online converter, such as Ploomer {%- endif %}