3  Upload and process images

3.1 Introduction

This chapter describes the workflow we use when coming back to the lab with SD cards from a recent game camera deployment. The script we use for processing these files is upload_and_process_images.py found in the PythonScripts directory of this repository. The script includes some subcommands that can do only select pieces of the workflow. Typically, however, you would use the subcommand completely_process_images_from_sd_card if you are trying to complete the whole workflow in one pass.

3.2 The script and its subcommands

The script upload_and_process_images.py has the following possible subcommands:

Table 3.1: upload_and_process_images.py subcommands

3.3 Normal Workflow

Under normal circumstances, when we have an SD card that we are ready to work with, we follow these steps:

  1. Create an empty directory in the appropriate Raw folder to receive the raw images. This folder will be named CXXX_SDXXX where XXX is the 3-digit camera number and SD card number, respectively, including leading zeroes.

  2. Create an empty directory in the appropriate Processed folder to receive the processed images. This folder will be named CXXX_SDXXX where XXX is the 3-digit camera number and SD card number, respectively, including leading zeroes. You can simply copy the empty folder from the Raw directory and paste it in the Processed directory since both are empty.

  3. Open a terminal window. CD into the PythonScripts folder of this repository.

  4. For us to keep track of the files we are pushing up to s3, we begin this process by creating and switching to a new branch in the project directory. Generally, we use a branch name such as DD-MON-YYYY-AWS-upload to make clear what we are doing. For example:

git checkout -b 14-Aug-2025-AWS-upload

  1. We use the upload_and_process_images.py script under normal conditions, coupled with the completely_process_images_from_sd_card subcommand. The arguments you will need to provide are shown in Table 3.2
Table 3.2: Arguments required by completely_process_images_from_sd_card subcommand
argument explanation
--memory-card-path provide (drag to terminal) the path to the SD card directory of images
--path-to-raw-images provide(drag to terminal) the path to the empty directory you created in Step 1 above.
--path-to-processed-images provide(drag to terminal) the path to the empty directory you created in Step 2 above.
--camera-number provide the camera number. If you fail to give the 3-digit number, the script will add it.
--sd-card-number provide the sd card number. If you fail to give the 3-digit number, the script will add it.
--path-to-uploaded-file provide the path to a file called files_uploaded_to_s3.txt, which is located in the misc_files directory one level up from the PythonScripts directory in this repository.
--bucket-name provide the name for the s3 “bucket” on AWS to which the images will be uploaded. Generally, this argument should be north-country-wild-images
  1. Run the code in the terminal by stringing together the following lines code, all as one continuous command in the terminal, but replace each case of FILEPATH, CAMERA-NUMBER, SD-CARD-NUMBER or BUCKET-NAME with either the requested file path or the correct argument.

python3 upload_and_process_images.py completely_process_images_from_sd_card --memory-card-path PATH-TO-MEMORY-CARD —path-to-raw-images FILEPATH —path-to-processed-images FILEPATH —camera-number CAMERA-NUMBER —sd-card-number SD-CARD-NUMBER —path-to-uploaded-file FILEPATH —bucket-name BUCKET-NAME

  1. As the script processes:

    • it will begin by copying files from the SD card to the appropriate folder in Raw. There will be a line-by-line output in the terminal for each file that is copied.
    • Next, it will ask you if you want to upload the files to s3. If you choose y, it will begin uploading the files, and will again give line-by-line feedback as it works and will let you know that each upload was successful. It is copying the files to the appropriate folder in Processed at the same time.
    • Next it will ask Do you want to overwrite uploaded files file? (y/n) If not, writes to uploaded_files.txt:
    • If you choose y, the file you provided as an argument, /North-Country-Wild/misc_files/files_uploaded_to_s3.txt will be modified to add all of the files that were just uploaded.
    • If you choose n, the file /North-Country-Wild/PythonScripts/uploaded_files.txt will be modified so that you still have a list of the files that you have just processed, even if they are not uploaded to s3.
    • Once the files are uploaded to s3 and copied to the correct Processed folder, the script next begins to resize and change the copyright information for all of the files in the appropriate Processed subfolder.
    • And that’s it!

  2. Once the script has processed, you need to create a pull request from the branch you are working on in the terminal:

    • use git add . to stage any files that were changed (which should just be /North-Country-Wild/misc_files/files_uploaded_to_s3.txt if everything went smoothly)
    • use git commit -m "commit message here" to commit your changes
    • Assuming that the branch you just created does not exist on github, then to push the changes to a new branch on git hub for making a pull request, use git push --set-upstream origin name-of-branch-you-are-working-from
    • if you read along in the terminal output, you will see a link to github where you can make the pull request.
    • If you are finished uploading for that session, go ahead and make the pull request on github. Once it is complete, you can merge your branch into main locally and then delete the branch.
    • If you have more SD cards to process in this session, continue to process them, using the script in the way we have described, and when you are completely finished, then follow the instructions here in step 8.

3.4 Atypical Workflows

More coming soon!