Skip to content

python

Jump to bottom
Stratis Ioannidis edited this page Mar 22, 2019 · 25 revisions

Running Python

Including the Python Module

To load the python module for python 2.7.15, type

module load python/2.7.15

You can find all the available modules by typing:

module avail

For example, beyond 2.7.15, other available python variants are:

python/3.6.6
python/3.7.0

You can also add the appropriate module load command in your bashrc file.

Launching the Interpreter & Running a Script

You can launch the python interpreter by typing python. Press Ctrl-D to exit the interpreter. You can run a python script by typing, for example:

python main.py

Make sure you run your scripts on compute nodes only, either in interactive or batch mode.

Package Management

If you need a package/module that is not already installed by default, you can use pip install with the --user option to install it locally (i.e., for your account only). For example

pip install cvxopt --user

installs package cvxopt to your python distribution.

Alternatively, to install this across the cluster, you need to issue a software request with the ITS ticketing system.

Editing Files Remotely Using PyCharm

There are several ways of transferring files to and from the discovery cluster, described in detail here. Specifically for python scripts, the Python IDE PyCharm includes built-in SFTP support and allows automatic syncing.

To use SFTP, you need the Professional version of PyCharm, which is free for students and teachers. Apply for an education account here, then download and install PyCharm.

Code in PyCharm is organized into Projects. PyCharm will ask you to create one when opened for the first time. Once you have created a project, you can connect to Discovery over SFTP:

  • In PyCharm, go to Tools > Deployment > Configuration, and click the green plus sign to add a server. Name it Discovery and for Type select SFTP. Click OK.

  • In the configuration screen that appears, in the Connection tab, for "SFTP host" enter xfer.discovery.neu.edu. Enter your myNEU username and password, and check the box marked "Save password" if you don't want to enter it every time you restart PyCharm. Click OK.

  • Now click on Tools > Deployment > Browse Remote Host and you should see the Discovery file structure. From here you can create and edit files remotely.

PyCharm also provides the ability to sync a project with a folder on Discovery.

  • Go to Tools > Deployment > Configuration, then click the Mappings tab.
  • For "Deployment path on server 'Discovery'", enter the folder in which you want to put your project on Discovery, e.g. /scratch/myNEUid/myProject, where myNEUid is your myNEU username and myProject is anything you want.

You can now upload and download your project files to and from Discovery, compare your local and remote files, or automatically upload your local files to Discovery upon saving changes, all from the Tools > Deployment menu.

In Tools > Deployment > Configuration, you can change "Root path" if you'd like a specific folder opened by default in the remote host browser panel. For example, you could set "Root path" to /scratch/myNEUid or /home/myNEUid. Note that the "Deployment path on server 'Discovery'" in the Mappings tab is relative to the "Root path" in the Connection tab.

Also note that you must still run your code from a compute node via SSH and not directly from PyCharm.

Additional Resources

Back to main page.

NEU

Main Topics

Special Topics

Clone this wiki locally