Skip to content

fix: fetching S3 objects and tutor forum in DigitalOcean

Keith Grootboom requested to merge keith/checklist-fixes into main

Description

Part 1 - Fixing instructor tasks

Fixes showing completed tasks in the instructor Dashboard for Digital Ocean deployed instances.

When trying to list tasks (eg. a Grading report) in the instructor dashboard, there will be an error like:

Traceback
2022-09-22 06:58:47,386 ERROR 7 [django.request] [user 5] [ip 10.10.10.2] log.py:224 - Internal Server Error: /courses/course-v1:OpenCraft+September+September/instructor/api/list_report_downloads
Traceback (most recent call last):
  File "/openedx/venv/lib/python3.8/site-packages/django/core/handlers/exception.py", line 47, in inner
    response = get_response(request)
  File "/openedx/venv/lib/python3.8/site-packages/django/core/handlers/base.py", line 181, in _get_response
    response = wrapped_callback(request, *callback_args, **callback_kwargs)
  File "/opt/pyenv/versions/3.8.12/lib/python3.8/contextlib.py", line 75, in inner
    return func(*args, **kwds)
  File "/openedx/venv/lib/python3.8/site-packages/django/views/decorators/http.py", line 40, in inner
    return func(request, *args, **kwargs)
  File "/openedx/venv/lib/python3.8/site-packages/django/utils/decorators.py", line 130, in _wrapped_view
    response = view_func(request, *args, **kwargs)
  File "/openedx/edx-platform/./lms/djangoapps/instructor/views/api.py", line 2463, in list_report_downloads
    return _list_report_downloads(request=request, course_id=course_id)
  File "/openedx/venv/lib/python3.8/site-packages/django/views/decorators/cache.py", line 31, in _cache_controlled
    response = viewfunc(request, *args, **kw)
  File "/openedx/edx-platform/./lms/djangoapps/instructor/views/api.py", line 223, in wrapped
    return func(*args, **kwargs)
  File "/openedx/edx-platform/./lms/djangoapps/instructor/views/api.py", line 2481, in _list_report_downloads
    for name, url in report_store.links_for(course_id) if report_name is None or name == report_name
  File "/openedx/edx-platform/./lms/djangoapps/instructor_task/models.py", line 327, in links_for
    _, filenames = self.storage.listdir(course_dir)
  File "/openedx/venv/lib/python3.8/site-packages/storages/backends/s3boto3.py", line 540, in listdir
    for page in pages:
  File "/openedx/venv/lib/python3.8/site-packages/botocore/paginate.py", line 255, in __iter__
    response = self._make_request(current_kwargs)
  File "/openedx/venv/lib/python3.8/site-packages/botocore/paginate.py", line 332, in _make_request
    return self._method(**current_kwargs)
  File "/openedx/venv/lib/python3.8/site-packages/botocore/client.py", line 317, in _api_call
    return self._make_api_call(operation_name, kwargs)
  File "/openedx/venv/lib/python3.8/site-packages/botocore/client.py", line 615, in _make_api_call
    raise error_class(parsed_response, operation_name)
botocore.errorfactory.NoSuchKey: An error occurred (NoSuchKey) when calling the ListObjects operation: Unknown

This error occurs because of a misconfigured endpoint URL for Digital Ocean. When doing queries you only need the region and not the BucketName in the URL.

Part 2 - Fixing the forum uploads

Added to this PR are changes to make the tutor-forum plugin work.

Specifically, added are S3 policy changes that match the ones in Ocim so that uploads done via the discussion board are public.

Supporting information

https://docs.digitalocean.com/products/spaces/reference/s3-sdk-examples/#configure-a-client

Testing instructions

Part1

Steps to test the changes:

  1. There is a test instance on https://studio.keith-relic.kgcluster.opencraft.hosting
  2. Go to the instructor dashboard
  3. Create a grading report
  4. If the fix is deployed your report will show up.

Part2

  1. Go to the discussion boards for any course
  2. Create a new post and verify that adding an image works.

Dependencies

Other information

Note that changes to the AWS provider are specifically not included here as they'll be done in a future PR.

Checklist

If any of the items below is not applicable, do not remove them, but put a check in it.

  • All providers include the new feature/change
  • All affected providers can provision new clusters
  • Unit tests are added/updated
  • Documentation is added/updated
  • The TOOLS_CONTAINER_IMAGE_VERSION in ci_vars.yml is updated
  • The grove-template repository is updated
Edited by Keith Grootboom

Merge request reports