Python - Revision history

Dmitri: /* Python and parallel programming */

2024-10-25T18:55:16Z

Python and parallel programming

← Older revision		Revision as of 18:55, 25 October 2024
Line 77:		Line 77:
	: Cython is a compiled language that is typically used to generate CPython extension modules.		: Cython is a compiled language that is typically used to generate CPython extension modules.

	= Python and parallel programming =		== Python and parallel programming ==

	There are a lot of nuances to implementing parallelism in python depending on your expertise and what you are trying to achieve.		There are a lot of nuances to implementing parallelism in python depending on your expertise and what you are trying to achieve.

Dmitri: /* Using Miniconda */

2024-09-03T20:08:54Z

Using Miniconda

← Older revision		Revision as of 20:08, 3 September 2024
Line 99:		Line 99:
	= Installing your own Python =		= Installing your own Python =

	== Using ~~Miniconda~~ ==		== Using Conda ==

	You will need a working local '''Conda''' install in your home directory first. If you do not have it yet, please follow [[Conda_on_ARC#Installing_Conda \| these instructions]] to have it installed.		You will need a working local '''Conda''' install in your home directory first. If you do not have it yet, please follow [[Conda_on_ARC#Installing_Conda \| these instructions]] to have it installed.

Dmitri: /* Using Miniconda */

2024-03-05T23:08:53Z

Using Miniconda

← Older revision		Revision as of 23:08, 5 March 2024
Line 131:		Line 131:
	Try your new <code>python</code> and <code>pip</code>:		Try your new <code>python</code> and <code>pip</code>:
	<pre>		<pre>
	(yourenvname) ~~[drozmano@arc pytorch]~~$ python --version		(yourenvname) $ python --version
	Python 3.11.8		Python 3.11.8

	(yourenvname) ~~[drozmano@arc pytorch]~~$ pip --version		(yourenvname) $ pip --version
	pip 23.3.1 from /home/username/software/miniconda3/envs/pytorch/lib/python3.11/site-packages/pip (python 3.11)		pip 23.3.1 from /home/username/software/miniconda3/envs/pytorch/lib/python3.11/site-packages/pip (python 3.11)
	</pre>		</pre>

Dmitri: /* Using Miniconda */

2024-03-05T23:08:26Z

Using Miniconda

← Older revision		Revision as of 23:08, 5 March 2024
Line 133:		Line 133:
	(yourenvname) [drozmano@arc pytorch]$ python --version		(yourenvname) [drozmano@arc pytorch]$ python --version
	Python 3.11.8		Python 3.11.8

	(yourenvname) [drozmano@arc pytorch]$ pip --version		(yourenvname) [drozmano@arc pytorch]$ pip --version
	pip 23.3.1 from /home/~~drozmano~~/~~my_software~~/miniconda3/envs/pytorch/lib/python3.11/site-packages/pip (python 3.11)		pip 23.3.1 from /home/username/software/miniconda3/envs/pytorch/lib/python3.11/site-packages/pip (python 3.11)
	</pre>		</pre>

Dmitri: /* Using Miniconda */

2024-03-05T23:07:34Z

Using Miniconda

← Older revision		Revision as of 23:07, 5 March 2024
Line 142:		Line 142:
	At this point you should be able to use your own python with the modules you added to it.		At this point you should be able to use your own python with the modules you added to it.

	You can deactivate your python environment and conda with the commands		You can '''deactivate your python environment''' and '''conda''' with the commands
	<pre>		<pre>
	(yourenvname) $ conda deactivate		(yourenvname) $ conda deactivate

Dmitri: /* Using Miniconda */

2024-03-05T22:58:22Z

Using Miniconda

← Older revision		Revision as of 22:58, 5 March 2024
Line 110:		Line 110:
	(base) $ which python		(base) $ which python
	~/software/miniconda3/bin/python		~/software/miniconda3/bin/python

			(base) $ python --version
			Python 3.9.18
	</pre>		</pre>
	This is a '''Conda system python''', we are not going to use it. We are going to install '''our working python''' into an environment.		This is '''Conda's system python''', we are not going to use it. We are going to install '''our own working python''' into an environment.

	Create a virtual environment for your project, and request <code>python v3.11.8</code> and <code>pip</code> to be installed into it:		Create a virtual environment for your project, and request <code>python v3.11.8</code> and <code>pip</code> to be installed into it:

Dmitri: /* Using Miniconda */

2024-03-05T22:56:49Z

Using Miniconda

← Older revision		Revision as of 22:56, 5 March 2024
Line 144:		Line 144:
	(base) $ conda deactivate		(base) $ conda deactivate
	$		$
	</~~code~~>		</pre>

	Your can create additional '''conda environments''' with different versions of python and combinations of software, if you need them.		Your can create additional '''conda environments''' with different versions of python and combinations of software, if you need them.

Dmitri: /* Using Miniconda */

2024-03-05T22:56:09Z

Using Miniconda

← Older revision		Revision as of 22:56, 5 March 2024
Line 108:		Line 108:
	Ensure it is using the python installed in your home directory		Ensure it is using the python installed in your home directory
	<pre>		<pre>
	$ which python		(base) $ which python
	~/software/miniconda3/bin/python		~/software/miniconda3/bin/python
	</pre>		</pre>
Line 114:		Line 114:

	Create a virtual environment for your project, and request <code>python v3.11.8</code> and <code>pip</code> to be installed into it:		Create a virtual environment for your project, and request <code>python v3.11.8</code> and <code>pip</code> to be installed into it:
	$ conda create -n <yourenvname> python=3.11.8 pip		<pre>
			(base)$ conda create -n <yourenvname> python=3.11.8 pip
			....
			....
			</pre>

	Activate the virtual environment		Activate the virtual environment
	$ source activate <yourenvname>		<pre>
			(base) $ source activate <yourenvname>
			(yourenvname) $
			</pre>

	Try your new <code>python</code> and <code>pip</code>:		Try your new <code>python</code> and <code>pip</code>:
Line 128:		Line 135:

	Install additional Python packages to the virtual environment		Install additional Python packages to the virtual environment
	$ conda install [package]		(yourenvname) $ conda install [package]

	At this point you should be able to use your own python with the modules you added to it.		At this point you should be able to use your own python with the modules you added to it.

			You can deactivate your python environment and conda with the commands
			<pre>
			(yourenvname) $ conda deactivate
			(base) $ conda deactivate
			$
			</code>

			Your can create additional '''conda environments''' with different versions of python and combinations of software, if you need them.

	== From Sources ==		== From Sources ==

Dmitri: /* Using Miniconda */

2024-03-05T22:51:48Z

Using Miniconda

← Older revision		Revision as of 22:51, 5 March 2024
Line 101:		Line 101:
	== Using Miniconda ==		== Using Miniconda ==

	You ~~can install~~ a local ~~copy of miniconda~~ in your home directory ~~on our clusters~~.		You will need a working local '''Conda''' install in your home directory first. If you do not have it yet, please follow [[Conda_on_ARC#Installing_Conda \| these instructions]] to have it installed.
	~~It will give~~ you ~~flexibility~~ to ~~install packages needed for the workflow~~. ~~Here are the steps to follow:~~

	Once ~~connected to the login node, in~~ your ~~SSH session~~, ~~make sure you are in your home directory:~~		Once you have your own '''Conda''', activate it with
	~~$ cd~~		$ source ~/software/init-conda

	~~Create a "software" subdirectory for all custom software you are going to have:~~
	~~$ mkdir software; cd software~~

	~~Download the software the latest Miniconda distribution file:~~
	~~$wget https://repo.anaconda.com/miniconda/Miniconda3-latest-Linux-x86_64.sh~~

	~~Install the downloaded <code>.sh</code> file:~~
	$ ~~bash Miniconda3-latest-Linux-x86_64.sh~~

	~~Follow the instructions (choosing <code>~~~/software/~~miniconda3</code> as the directory to create),~~
	~~agree to the license, decline the offer to initialize.~~


	~~Every time you launch a new terminal and want to use this version of python, set the path as follows~~
	~~$ export PATH=/home/<username>/software/miniconda3/bin:$PATH~~

	Ensure it is using the python installed in your home directory		Ensure it is using the python installed in your home directory
Line 128:		Line 111:
	~/software/miniconda3/bin/python		~/software/miniconda3/bin/python
	</pre>		</pre>
	This is a '''Conda system python'''.		This is a '''Conda system python''', we are not going to use it. We are going to install '''our working python''' into an environment.

	Create a virtual environment for your project		Create a virtual environment for your project, and request <code>python v3.11.8</code> and <code>pip</code> to be installed into it:
	~~$ conda create -n~~ <~~yourenvname~~>		$ conda create -n <yourenvname> python=3.11.8 pip

	~~Install additional Python packages~~ to ~~the virtual environment~~
	$ conda ~~install~~ -n <yourenvname> ~~[package]~~

	Activate the virtual environment		Activate the virtual environment
	$ source activate <yourenvname>		$ source activate <yourenvname>

			Try your new <code>python</code> and <code>pip</code>:
			<pre>
			(yourenvname) [drozmano@arc pytorch]$ python --version
			Python 3.11.8
			(yourenvname) [drozmano@arc pytorch]$ pip --version
			pip 23.3.1 from /home/drozmano/my_software/miniconda3/envs/pytorch/lib/python3.11/site-packages/pip (python 3.11)
			</pre>

			Install additional Python packages to the virtual environment
			$ conda install [package]

	At this point you should be able to use your own python with the modules you added to it.		At this point you should be able to use your own python with the modules you added to it.

Dmitri: /* Using Miniconda */

2024-03-05T22:38:15Z

Using Miniconda

← Older revision		Revision as of 22:38, 5 March 2024
Line 128:		Line 128:
	~/software/miniconda3/bin/python		~/software/miniconda3/bin/python
	</pre>		</pre>
			This is a '''Conda system python'''.

	Create a virtual environment for your project		Create a virtual environment for your project