When trying to develop and debug in container environment, in lieu of local machine

Contact info

Yueteng | yh958@cornell.edu

Note

Multiple versions of docs are kept in this page in the following order:

• Newer versions towards the top
• Older  versions towards the end

...

Doc Version 1.1

Purposes

Solved these problems:

1. Two containers (one for VS Code, the other one for running) appear in Docker Desktop
   1. → Would like to merge them into one
   2. → Even better, remove the container after finishing using (clean for the eye)
2. After making changes to the code, the changes are not auto-reloaded. Must re-run docker-compose up
   1. → Enable auto-reloading
3. Env variables not all enabled
   1. → Re-enable all previously used development env variables, with auto-reloading enabled (via volume)
   2. → Create a separate prod env variables, with auto-reloading disabled (via volume)
4. VS Code extension Python does not (always) work
   1. → Fix the "couldn't load python tool" error

Step-by-step guide

Step 0 Preparation; Overview

docker volume rm $(docker volume ls -f dangling=true -q)

...

• Python image issues:
  • The current Python image used in this project is based on alpine Linux.
  • alpine Linux does not include many necessary libs for development, explaining why Python extension fails and additional lib installation (RUN apk *) in Dockerfile
  • We're going to switch to the standard Python image to avoid all the problems.
• VS Code and container
  • Previously, we let VS Code create a container and develop inside (remember the container name of random scientist/mathematician), hence the extra container.
  • Now, we're going to build/create/run the container by ourselves, and let VS Code attach to the container. Only ONE both for running and VS Code
  • Additionally, upon finished (developing and running), we'll shut down the container, and it will be removed from Docker Desktop > Containers Apps, making it look clean.
• Others (if you're interested, optional)
  • Dockerfile: The base image FROM is changed to python:3.8.6, a full-feature image. You will also see many lines commented out, unnecessary for the moment.
  • docker-compose.yml : This is used for development (also the default one). You can see all env variables for  development.
  • docker-compose-prod.yml: This will be for production. Not finished yet.
  • docker-compose.debug.yml.old: Just some backup. Ignore it.

Step 1 Run the container

# Path for online repo
backend-online/dev-backend
 
# Path for offline repo
backend-offline/dev-dashboard

...

Note: If there is an error message says "cannot import name 'Markup' from 'jinja2', you can try to fix it through the means provided by ImportError: cannot import name 'Markup' from 'jinja2'. (Basically type python3 in terminal, then type  'from jinja2.utils import markupsafe', then  'markupsafe.Markup()'. The error will likely to be fixed.

If you are using Mac and get an error message when running the container for the first time, saying "Ports are not available: listen tcp 0.0.0.0:5000: bind: address already in use", it is likely that your Mac AirPlay server is running on this port (even when you try to kill -9 the process, it restarts automatically). To solve this problem, deactivate the AirPlay Receiver by:

Go to System Preferences › Sharing, and unchecking AirPlay Receiver to release port 5000

Image Added

You should see a new container running.

...

• Remote - Containers
• Python

Step 2 Attach VS Code to the container; Necessary extensions

On VS Code, click the Remote button at bottom left > Attach to running container > Click the container name in the menu.

...

It might show loading/downloading analysis at the bottom status bar, after which it should work properly.

Step 3 Run and debug; Auto-reload

Everything under www, if changed:

...

Try the above to see auto-reloading is working.

Step 4 Close connexion with container; Shut down container

On VS Code, click the Remote button at bottom left > Close remote connection.

...

In terminal, run the command above. It will shutdown and remove container (as you can see it disappears from Docker Desktop > Containers / Apps)

...

Doc Version 1.0

Purpose

1. Environment consistency: Avoiding inconsistency due to different dependency/lib on local machine v in container
2. Centralised dependency/lib management: Only one member needs to update the config (requirements.txt) in docker, and all other members will have access to updated dependency/libs automatically after git pull
3. Convenience: Avoiding install/keep the same dependency/lib twice, i.e. on local machine and in container

Step-by-step guide

Note: This guide is based on VS Code. I've also tried IntelliJ IDEA (including PyCharm), but unfortunately it doesn't work on my laptop. If interested, see the link below under "Read more"

Step 1 Install extension and launch docker

On VS Code, make sure you've installed these two extensions:

...

Launch docker (you don't need to start the project container)

Step 2 Open container

Click the remote container button > Open folder in container > Open your working directory with Dockerfile inside the repo. For instance, for backend-offline subteam, choose backend-offline/dev-dashboard/dashboard-backend

...

When completed, you'll notice VS Code has deployed an image (name starting with vsc-) in your docker, and has inserted/started an container (name including some random scientist/mathematician).

Step 3 Run and debug in container environment

Click the button to the right of the container button, to choose your interpreter/SDK.

...

If you run it, it should show in terminal the same output as above. Note the 2nd line Linux shows it's been run in container (not local machine of Mac OS/Windows)

Note

Everything you've coded is done in the same git repo.

If you choose to stop the container after finish coding, it's alright. Next time VS Code will automatically start it when you open the working directory as in steps above

Read more

Guide from VS code: https://code.visualstudio.com/docs/remote/create-dev-container

...