NB: If copying and pasting commands from this wiki, do not include the '$' symbol!
NB: If copying and pasting commands from this wiki, do not include the '$' symbol!
## Install Python 3.7.5
As most Linux distributions already come with Python installed there are a couple of options available:
**The recommended Python version is v3.7.5.**
1. Install Python 3.7.5 separate from your system Python (recommended)
1. Use your system Python (provided it is compatible)
There are several options to get Python and it may already be installed on your computer.
## Install Python 3.7.5 separate from your system Python
If you are running a GNU/Linux operating system then you probably already have Python 3 installed. To find out, open a terminal session or command-line window and type `python3 -V`. If you have Python you will be told which version. You can also try `python -V` (without the 3) but this is usually an older 2.x version.
Regardless of your system Python, you can compile and install Python 3.7.5 so that it does not interfere with any existing Pythons. By compiling Python you will also start with 'blank slate' to fill with the required CAML packages and so avoid package dependency conflicts.
These scripts have been tested on Python v3.7.5 although they are known to work on v3.5 and v3.6 as well.
To begin, proceed to [compiling Python 3.7.5 from source](Setting Up Linux Compile Python)...
**I have Python 3.5, 3.6, or 3.7 on my computer already**
## Use your system Python
If you have Python installed already and it is 3.6 or 3.7 then you can skip to the section on [setting up a virtual environment](#setting-up-a-virtual-environment). If you have these but would still like to install the recommended version of v3.7.5 alongside your existing Pythons, then skip down to [download and compile Python](#download-and-compile-python).
**I have Python 3 but it's not the right version**
Python v3.5, v3.6 and v3.7 should all work with CAML although it is recommended to use v3.7.5 as this has been tested.
If you have Python3 but it's the wrong version then follow the instructions in the next section to [download and compile Python](#download-and-compile-python) without affecting any of your system Python installations.
**I don't have Python 3 at all**
Check your Python version with:
[Follow the instructions at Python.org to install yourself a new system Python](https://docs.python.org/3.6/using/unix.html#on-linux). If you do not want Python installed system-wide then read the instructions for [downloading and compiling Python](#download-and-compile-python).
Alternatively, you can also [download Anaconda](https://www.anaconda.com/download/), a Python data science platform, which comes preinstalled with many packages and integrated development environments (IDE, where you can write and run code) options.
If you opt for Anaconda be sure to follow their instructions on how to install the required packages. The list of packages (and required versions for each package) are given in the 'requirements' text files. There are two such files depending on whether you intend to make use of GPU-accelerated processing (requirements_gpu...) or not (requirements_no-gpu...). If you are unsure, go with the no-gpu file. You will need to use Anaconda's package manager to install the packages (and of the appropriate version) as indicated in the requirements file.
However, for the most flexibility it is recommended here to set up Python in a virtual environment with one of the following methods.
## Download and Compile Python
**If you already have Python 3.7.5 installed then you can skip to the next section ['Setting up a virtual environment'](#setting-up-a-virtual-environment).**
**If you don't have Python on your system at all you can also ['install it to your system'](https://docs.python.org/3.6/using/unix.html#on-linux).**
Here we will download the Python source and compile it to a specific location (instead of installing it as our system Python). This will preserve any existing Python versions that your operating system may be reliant on to function correctly.
First, install some required dependencies using *apt* and create a destination folder (called *python375*, in the home folder) for the compiler to use:
Now download the Python source code to your *Downloads* folder using *wget*, and extract it using *tar*.
[Go to the Python 3.7.5 Release page](https://www.python.org/downloads/release/python-375/). At the bottom of the page, check that the URL for the Gzipped source tarball matches what is below. If not then copy the new location when downloading.
Hop into the source folder and configure the build. The important part here is the `--prefix` flag which will send the Python build into the folder we made earlier. Also, if you *want* to install it as a system-wide Python then don't include the `--prefix` tag. Be aware that this can cause problems if you have an existing Python installation!
The `--enable-optimizations` flag will result in a better performing Python but it will take slightly longer to build.
Now we can begin the build process. It can take a little while:
```bash
$ make
$ make install
```
When it has finished, go back to your home folder and set up a new virtual environment. We'll use the existing system Python3 to do this but it will copy our freshly built Python 3.7.5 into the environment.
As you have just set up your virtual environment you can skip the next section and go to [Install Required Packages](#install-required-packages).
## Setting up a virtual environment
A virtual environment is an installation of Python which is separate from the system’s own installation. Using the virtual environment you can set it up in a specific way (such us to run the CAML scripts) without accidentally installing a package (or a version of a package) which is not compatible with your operating system. Similarly if you break the virtual Python environment you won’t affect your system’s Python.
Follow the instructions at https://packaging.python.org/guides/installing-using-pip-and-virtualenv/ to install the virtualenv package and set up a new environment. Assuming you have a recent GNU/Linux distribution with Python 3.6 already installed you can follow the next series of commands.
First, launch the Terminal application and check that you have pip installed:
```bash
$ pip3 -V
```
If you do not have pip installed, install it with:
```bash
$ sudo apt install python3-pip
```
Next, upgrade pip and install virtualenv:
```bash
$ python3 -m pip install--user--upgrade pip
$ python3 -m pip install--user virtualenv
```
Then, from your home directory, create a Python environment which we will name ‘envCAML here’:
```bash
$ cd ~
$ python3 -m virtualenv envCAML
```
Switch to the ‘envCAML’ environment:
```bash
$ source ~/envCAML/bin/activate
(envCAML) user@computer:~$
```
The command prompt will be changed to indicate the environment, e.g. 'envCAML', that it is working under (tip: to exit the environment just use the command ‘deactivate’). Now we can modify the environment by installing some Python packages and our system’s Python configuration will not be changed.
## Install required packages
From your *envCAML* environment prompt, supply pip with a requirements file, depending on whether you will use GPU accelerated processing (*_gpu*) or not (*_no-gpu*). The list of packages is the same in each except for Tensorflow which is either the regular tensorflow package or the CUDA-enabled tensorflow-gpu package. If you are not sure then install the packagse without GPU support.
### Installation without GPU processing
To install without GPU acceleration (processing on the CPU only) run:
If you decide to use the GPU then you will also need to ensure that you have correctly installed and configured an Nvidia CUDA-capable graphics card, appropriate drivers (v384.81), CUDA Toolkit v10.0, CuDNN Libraries v7.4, and the recommended compilers and dependencies. There is [a bare-metal guide here](Setting Up CUDA from scratch), which may or may not work.
If you wish to install the packages with the CUDA-enabled tensorflow, then run:
After you have set up your development environment and installed the necessary Python packages you will then need to install the graphviz software, which requires software to be installed from the Ubuntu repositories:
```bash
(envCAML)$ sudo apt-get install graphviz
```
## Launch Spyder IDE (Integrated Development Environment)
All the required packages and software should now be installed so you can launch Spyder and start running some code! First check that you are in the virtual environment. Your terminal’s command prompt should indicate this, e.g. if we are in our virtual environment ‘envCAML’ we should see:
```bash
```bash
(envCAML) user@computer:~$
$ python3 -V
```
```
If you are in the right environment, launch Spyder with:
If it is compatible, proceed to [setting up a virtual environment](Setting Up Linux Virtual Env)...
\ No newline at end of file
```bash
(envCAML)$ spyder3 & disown
```
NB: If you launch Spyder outside of the environment then it will use your system’s Python configuration, which may not be set up correctly for CAML. Launching with `& disown` directs Spyder to launch as a new process without ties to the Terminal window, so you can close the Terminal window if you want.
In future, to launch Spyder from ‘envCAML’ environment, open a new Terminal window and enter the following commands:
```bash
(envCAML)$ source ~/envCAML/bin/activate
(envCAML)$ spyder3 & disown
```
If anything goes wrong with your envCAML environment you can just create a new one with a different name and use that instead. You can remove a virtual environment by deleting the environment’s main folder, e.g. with a command like ‘rm -r ~/envCAML’.
Once you have Spyder up and running you might want to look at [changing some settings](Optional Spyder Settings) to make it easier to use.
#### Leaving the virtual environment
To leave the virtual environment you can either just close the terminal window or use the command `deactivate`. Note the change in the terminal prompt; the name of the virtual environment is no longer displayed:
```bash
(envCAML)$ deactivate
$
```
### Perform a test run
In Spyder's 'File explorer' tab, change to the folder with the downloaded CAML scripts.
Open `0.0_SimuCell.py` and run it by clicking the large green play-button or pressing F5.
This will simulate a small amount of test data as part of 'Stage 0'. You can then move to the Stage 1 (data preparation) and Stage 4 (evaluate using a model) scripts. For more information, read the ['Using an Existing Model' Tutorial](Tutorial Existing Model).
For example, using these conditions in the data simulation script:
will generate five datasets: four sets with clustering as 40 nm clusters, 10 points per cluster, and 25%, 50%, 75%, or 100% of the points clustered. A fifth non-clustered set is produced automatically; all sets have 50 points per μm² overall.
Note: if this is your first time running any code in Spyder , a dialog box appear asking where you wish to execute the code; you can safely accept the default options here (the code will execute in the IPython ‘Console 1/A’ at the lower right corner of the workspace) and click OK.
