Skip to content

Development getting started

Jump to bottom
Erik Chen edited this page Jun 28, 2023 · 39 revisions

Overview

Planscape is structured as a classic REST-based web application. The major components are

  • A Postgres database, with PostGIS extensions, for storing geometry, data layers, and user input in the form of plans, projects, and scenarios;
  • A Django REST backend with GeoDjango extensions; and
  • An Angular frontend with Leaflet extensions for drawing maps.

Two helpful tutorials for learning about this stack:

Coding standards

We try to maintain the following development standards:

  • Use the Python 3.0 type system to catch errors early in the Django backend. The instructions below show how to see type errors in VSCode.
  • Write and maintain unit tests to go along with the code, with tests run automatically in GitHub.
  • Perform code reviews of GitHub pull requests using the github.com interface.

GitHub Account

While not specifically required, it is recommended that you use an account that will persist (e.g. a personal account not related to any employer).

You will need to ask a member of the Planscape team to be added to the group in order to make changes to the repository.

Development environment

The instructions below have been developed using MacOS. It should be straightforward to adapt them to other environments (Linux, Windows).

During installation of packages you may be asked to modify directory permissions (e.g. sudo chmod ...).

Install Python and GDAL

First you must install Python and GDAL. On MacOS,

  • Install homebrew from a terminal window

    /bin/bash -c "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/HEAD/install.sh)"
echo 'eval "$(/opt/homebrew/bin/brew shellenv)"' >> /Users/$USER/.profile

  • Install Python 3.10 (or later):

    brew reinstall python@3.10

Please read the install output carefully since most of the installs are dependent on python 3.10+. You can check to see if it installed properly in /usr/local/bin/.

  • Install GDAL (necessary for processing raster GIS data): 
    brew install gdal

Source code structure

The directory structure will look like

env/                        # Python environment
  bin/                      # Python env binaries
  include/                  # Python env includes
  lib/                      # Python env libraries
  share/                    # Python env share
  pyvenv.cfg                # Python env configuration
  .git/                     # Local Git repository
  Planscape/                # Project root in GitHub
    CONTRIBUTING.md
    LICENSE.md
    README.md
    analysis/               # Experimental analyses
    doc/                    # Documentation files
    mypy.ini                # Configuration for Python type system
    src/
      .gitignore            # Git configuration for files to ignore
      deployment/           # Scripts for deployment to AWS, etc.
      docker-compose.yaml   # Docker file
      interface/            # Angular frontend
      planscape/            # Django backend

Set up Python

In a terminal window,

  • Make a virtual environment env in a fresh directory:

    python3 -m venv env

  • Check that the file env/pyvenv.cfg has the following or similar contents:

    home = /opt/homebrew/opt/python@3.10/bin
include-system-site-packages = false
version = 3.10.6

  • Use the following command to start using Python in the environment:

    source env/bin/activate

    TIP: Create an alias to this command, e.g.,

    alias startdev="source env/bin/activate"

Set up Service Software

Before you can do something with the software, you'll want to install Postgres and R.

Install PostgreSQL (at least version 14)

brew install postgresql

Install R

You'll want R, but you also need to install a bunch of prerequisite libraries through brew:

brew install libgit2
brew install harfbuzz
brew install fribidi
brew install udunits
brew install r

After you have installed R, run r and install more libraries. There are plenty of R mirrors around the world; choose one close to you.

% r
> install.packages("dplyr")
> install.packages("textshaping")
> install.packages("stringi")
> install.packages("ggnewscale")
> install.packages("udunits2")
> install.packages("sf")
> install.packages("ragg")
> install.packages("pkgdown")
> install.packages("devtools")

Warning: devtools is enormous. Get a cup of coffee when you do this. You may encounter other missing libraries, and so be ready to install those and re-install devtools. After installing devtools, go back to r and install the forsys libraries:

% r
> devtools::install_github("forsys-sp/forsysr")
> devtools::install_github("forsys-sp/patchmax")

Download the source code

The Planscape source code is stored in GitHub, which provides support for code reviews, continuous integration, deployment, etc. The OurPlanscape/Planscape GitHub Repository contains the code, and the Planscape develoment GitHub project contains the issues and bugs.

In order to check in/review code or create issues, you will need a GitHub account that is a member of the OurPlanscape organization. If you don't already have a GitHub account, please create one and ask own of the owners of the GitHub repository to add you to the organization.

Our repository requires that you connect via SSH rather than https. You can download code via https but without SSH you won't be able to merge changes in.

Install SSH Key (possibly optional)

Use the email address you used for your github account:

ssh-keygen -t ed25519 -C "your_email@your_domain.com"

You can ignore any fancy ASCII artwork that shows up in your terminal.

Follow the Github instructions on how to apply the public key you just generated into your github account.

For more info, Github's full docs on installing SSH keys can be found here.

Get the source code

To download the source code, run the following commands in your terminal window:

cd env
git init
git remote add origin git@github.com:OurPlanscape/Planscape.git
git clone ssh://git@github.com/OurPlanscape/Planscape.git

If you intend on making modifications, see instructions on how to make your own branch.

Install More Planscape-specific Libraries

Install pip

If you do not have pip installed, this will fail:

pip --version

In that case, you will need to install pip. For macOS, you will have to "bootstrap" pip; instructions can be found here.

Download the Django libraries:

cd Planscape/src/planscape/
python -m pip install --upgrade pip
pip install -r requirements.txt

Install Frontend Libraries and Services

npm is a tool to manage javascript libraries (including more Angular code).

brew install npm

Note that if you are asked for PATH_TO_ENV, typically that's your home directory path, e.g. /home/$USER.

Finally, download the necessary Angular libraries (which includes ng). You may be required to install Node JS. Depending on your operating system you can find the exact steps here

cd ../interface
npm install -g @angular/cli

Install VSCode (optional)

VSCode is very helpful for developing both frontend and backend code. It provides syntax highlighting, code completion, and type checking.

Some additional setup steps:

  1. Create a file

    env/.env

    with the contents

    PYTHONPATH=Planscape/src/planscape

  2. Create a settings file

    env/.vscode/settings.json

    with the following contents (replacing PATH_TO_ENV):

    {
 "python.linting.mypyEnabled": true,
 "python.envFile": "${workspaceFolder}/.env",
 "python.linting.enabled": true,
 "mypy.dmypyExecutable": "PATH_TO_ENV/env/bin/dmypy",
}

  3. In VS Code, install the Python and Pylance extensions (⇧⌘X).

Set environment variables

Backend

Django requires some secrets that we can't include in public source control, since anyone could use it to break the encryption on our DB. To set these variables, create a Python config file:

src/planscape/planscape/.env

with the contents

# SECURITY WARNING: keep the secret key used in production secret!
# This file should remain untracked by version control.
SECRET_KEY = ...
PLANSCAPE_DATABASE_HOST = ...
PLANSCAPE_DATABASE_PASSWORD = ...

Ask one of the current Planscape developers for these secrets.

Frontend

The angular code reads two kinds of files that can contain settings that are specific to the environment or development box.

  1. features.json The features.json file (located in src/interface/src/app/features) contains feature flags, which can be used to show or hide certain features in the UI. You must add a features.json file for angular to behave correctly. You can find more details in the features.json readme.
  2. environment[.prod].ts The environment.ts file (or environment.prod.ts file), found in src/interface/src/environments/, contains server-side settings for the frontend (such as a Google Analytics ID). You can edit your files such as:

features.json sample

Add this to src/interface/src/app/features if you don't have it yet.

{
  "login": true,
  "show_socal": false,
  "testFalseFeature": false,
  "testTrueFeature": true,
  "unlaunched_layers": false,
  "top_percent_slider": true,
}

environment.ts example

export const environment = {
  production: false,
  backend_endpoint: 'http://localhost:8000/planscape-backend',
  google_analytics_id: '' // Replace with actual ID.
};

Test run some tests

From src/planscape, run:

python manage.py test -p "*test*.py"

This should end in "OK"

Test run the application

In one terminal window or tab, start the Django backend from the src/planscape directory:

python manage.py runserver

The output should look like

Watching for file changes with StatReloader
Performing system checks...
System check identified no issues (0 silenced).
September 30, 2022 - 14:28:21
Django version 4.1.1, using settings 'planscape.settings'
Starting development server at http://127.0.0.1:8000/
Quit the server with CONTROL-C.

In another shell, start the Angular frontend from the src/interface directory:

npm install
ng serve --open

Note that npm install may reveal some package vulnerabilities. It is recommended that you fix at least those packages that won't break:

npm audit fix

The output should look something like

✔Browser application bundle generation complete.
Initial Chunk Files   | Names         | Raw Size
vendor.js             | vendor        |   2.84 MB |
polyfills.js          | polyfills     | 318.01 kB |
styles.css, styles.js | styles        | 304.02 kB |
main.js               | main          |  14.94 kB |
runtime.js            | runtime       |   6.52 kB |

                      | Initial Total |   3.47 MB

Build at: 2022-09-30T14:29:12.454Z - Hash: 4158b2a1e75c035d - Time: 3248 ms

** Angular Live Development Server is listening on localhost:4200, open your browser on http://localhost:4200/ **

✔ Compiled successfully.

Navigate to localhost:4200 in your browser. You should see the Planscape application.

Clone this wiki locally