Getting Started¶
PREPOBS-BGC provides a set of modules to process and map biogeochemical data.
Requirements¶
In order to execute the scripts of this project, It is necessary to have conda installed to be able to create and use the virtual environements needed.
How to install conda ?
Having GNU Make installed can also simplify the project's setup.
How to install GNU Make ?
Building the virtual environment¶
make all
conda env create --file environment.yml --prefix ./.venv
conda activate ./.venv
poetry install
Configuration files¶
Each scripts has an associated configuration file to set up all necessary parameters. By default, these configuration don't exists but can be created from a 'default configuration' existing in config/default. If these copies don't exist, the following command will create the files:
make copy-default-config
Manually copy/paste all scripts from config/default into config. Or use the following command:
for name in config/default/*.toml; do cp config/default/$(basename ${name}) config/_$(basename ${name}) ; done
Before running any script, one has to verify that all parameters indicated in the configuration are relevant. For example, one has to fill in all PATH parameters in the providers.toml configuration file to indicate where the providers data can be find.
Running the Scripts¶
To run a script from the folder scripts and named script_1.py, one can use the following command:
make run-script-1 # (1)!
- make will install the correct environment and run the script
Any .py file located in the folder ./scripts/ can be run with a make rule starting by run- and ending with the name of the file (without the .py extension) and where all underscores ('_') ar replaced by hyphens ('-').
These rules are created dynamically so if a new script is added, there is no modification to apply to the Makefile to use the corresponding rule
Virtual environment must have been built, see here
conda activate ./.venv # (1)!
- Activate virtual environment
poetry install --without dev,docs # (1)!
- Install all required libraries
python scripts/script_1.py # (1)!
- Run script.
All the details about the scripts, their execution, their input parameters and their output can be found in the Scripts section of this documentation.
Make rules cheatsheet¶
Installation¶
Main
make all
Create the environment, install main libraries and copy the configuration files (if needed).
make copy-default-config
Copy default configuration files to the config folder if default files have been modified or if the configuration file doesn't exist.
make pre-commit
Install git pre-commit hooks to ensure that the code meets editing standards before committing to github.
Development
make install-dev
Install the environment as make all does with additional development libraries and installs git hooks to ensure that the code meets editing standards before committing to github.
make hooks
Install git hooks to ensure that the code meets editing standards before committing to github.
Cleaning¶
make clean
'Clean' the repository environment: remove virtual environment folder and git hooks.
make clean-dirs
'Clean' the outputs: remove both bgc_fig and bgc_data directories if existing.
Documentation¶
make view-docs
Create the environment, install documentation-related libraries and build the documentation website locally. The documentation is then accessible from a browser at the localhost:8000 adress. See MkDocs documentation on mkdocs serve for more informations.
make build-docs
Create the environment, install documentation-related libraries and build the documentation website into the 'site' folder. See MkDocs documentation on mkdocs build for more informations.
make deploy-docs
Create the environment, install documentation-related libraries and deploy documentation to a github branch. See MkDocs documentation on mkdocs deploy for more informations.
Running scripts¶
make run-any-script
Create the environment, install scripts-running-related libraries and runs the scripts/any_script.py python script. 'any-script' can be replaced by anything in order to run a script. For example, calling make run-another-script will run the scripts/another_script.py python script (if it exists). To make this rule work, the following syntax must be respected:
- script must be a python script
- script must be in the folder
scripts - underscores ('_') must be replaced by hyphens ('-') in the script name
- extension ('.py') must be removed from the script's name
- rule must start with the `run-` prefix