virtualenv

virtualenv is a third-party tool to create isolated Python environments. Since Python 3.3, a subset of it has been integrated into the standard library under the venv module. However, the venv module does not offer all features of this library. virtualenv can be installed via pip (and pip3 for python3), see more details about istallation here.

1. Create virtual environment

mkdir adir && cd adir
virtualenv venv

2. Activate virtual environment

source venv/bin/activate
and now we'll be in a new python environment: see for example how the prompt changed or type which python to check the new python path. Now one can list the python packages installed by default in the virtual environment (pip list), and we can start adding more (pip install python_package).

3. Export (local) packages and version numbers (e.g. to be used in another project)

pip freeze --local > requirements.txt
The --local flag takes only the local dependencies in the virtual environment.

4. Exit virtual environment

deactivate

5. Specify a python version for the virtual environment

$ python --version
Python 2.7.16

$ virtualenv -p /usr/local/bin/python3  py3_venv
$ source py3_venv/bin/activate
(py3_venv) $ python --version
Python 3.8.2

$ deactivate

$ virtualenv -p /usr/local/bin/python2  py2_venv
$ source py2_venv/bin/activate
(py2_venv) $ python --version
Python 2.7.16

Python

Python 2.7.14

tar xvzf  Python-2.7.14.tgz
cd  Python-2.7.14
./configure --prefix=/afs/cern.ch/work/s/sevilla/software
make 
make install

How to know from where a module is loaded

[sevilla@sermac16]:~$ python2
Python 2.7.14 (default, Nov 17 2017, 10:28:24) 
[GCC 4.2.1 Compatible Apple LLVM 8.0.0 (clang-800.0.38)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import gi
>>> gi.__file__
'/usr/local/lib/python2.7/site-packages/gi/__init__.pyc'
>>> 

How does Python find its packages

Links:

Python imports work by searching the directories listed in sys.path. How sys.path gets populated ? Well, assuming the PYTHONPATH environment variable is not set, sys.path will consist of the current working directory plus any manipulations made to it by the site module which controls installation-dependent default paths. The site module is automatically imported when you start Python, you can read more about how it manipulates your sys.path in the Python docs (it's a bit involved).

To copy & paste:

import sys
print '\n'.join(sys.path)

So, if we have several python versions installed in our system:

[sevilla@Sergios-MBP]:~$ which python
/usr/bin/python

[sevilla@Sergios-MBP]:~$ python
Python 2.7.10 (default, Feb  7 2017, 00:08:15) 
[GCC 4.2.1 Compatible Apple LLVM 8.0.0 (clang-800.0.34)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import sys
>>> print '\n'.join(sys.path)

/Library/Python/2.7/site-packages/Pygments-2.1.3-py2.7.egg
/Users/sevilla/cernbox/scripts
/Users/sevilla
/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python27.zip
/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7
/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/plat-darwin
/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/plat-mac
/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/plat-mac/lib-scriptpackages
/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/lib-tk
/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/lib-old
/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/lib-dynload
/Library/Python/2.7/site-packages
/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python
/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/PyObjC
>>> 

[sevilla@Sergios-MBP]:~$ which python2
/usr/local/bin/python2

[sevilla@Sergios-MBP]:~$ ls -ltr /usr/local/bin/python2
lrwxr-xr-x  1 sevilla  admin  35 Nov 17 10:29 /usr/local/bin/python2 -> ../Cellar/python/2.7.14/bin/python2

[sevilla@Sergios-MBP]:~$ python2
Python 2.7.14 (default, Nov 17 2017, 10:28:24) 
[GCC 4.2.1 Compatible Apple LLVM 8.0.0 (clang-800.0.38)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import sys
>>> print '\n'.join(sys.path)

/Users/sevilla/cernbox/scripts
/Users/sevilla
/usr/local/Cellar/python/2.7.14/Frameworks/Python.framework/Versions/2.7/lib/python27.zip
/usr/local/Cellar/python/2.7.14/Frameworks/Python.framework/Versions/2.7/lib/python2.7
/usr/local/Cellar/python/2.7.14/Frameworks/Python.framework/Versions/2.7/lib/python2.7/plat-darwin
/usr/local/Cellar/python/2.7.14/Frameworks/Python.framework/Versions/2.7/lib/python2.7/plat-mac
/usr/local/Cellar/python/2.7.14/Frameworks/Python.framework/Versions/2.7/lib/python2.7/plat-mac/lib-scriptpackages
/usr/local/Cellar/python/2.7.14/Frameworks/Python.framework/Versions/2.7/lib/python2.7/lib-tk
/usr/local/Cellar/python/2.7.14/Frameworks/Python.framework/Versions/2.7/lib/python2.7/lib-old
/usr/local/Cellar/python/2.7.14/Frameworks/Python.framework/Versions/2.7/lib/python2.7/lib-dynload
/usr/local/lib/python2.7/site-packages
/usr/local/lib/python2.7/site-packages/gtk-2.0
/usr/local/lib/python2.7/site-packages/gtk-2.0
>>>

[sevilla@Sergios-MBP]:~$ which python3
/usr/local/bin/python3

[sevilla@Sergios-MBP]:~$ ls -ltr /usr/local/bin/python3
lrwxr-xr-x  1 sevilla  admin  35 Nov 17 10:32 /usr/local/bin/python3 -> ../Cellar/python3/3.6.3/bin/python3

[sevilla@Sergios-MBP]:~$ python3
Python 3.6.3 (default, Nov 17 2017, 10:31:19) 
[GCC 4.2.1 Compatible Apple LLVM 8.0.0 (clang-800.0.38)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import sys
>>> print ('\n'.join(sys.path))

/Users/sevilla/cernbox/scripts
/Users/sevilla
/usr/local/Cellar/python3/3.6.3/Frameworks/Python.framework/Versions/3.6/lib/python36.zip
/usr/local/Cellar/python3/3.6.3/Frameworks/Python.framework/Versions/3.6/lib/python3.6
/usr/local/Cellar/python3/3.6.3/Frameworks/Python.framework/Versions/3.6/lib/python3.6/lib-dynload
/usr/local/lib/python3.6/site-packages
>>>
ALERT! Note the needed slight change in the print format for python 3...

GUIs

Remi

Basically trying to get installed a Python GUI framework in lxplus.

PyGtk

Install dependencies in order.

ALERT! Due to the impossibility of compiling the minimum pygobject version required (>= pygobject-2.21.3) with Python2.6, we'll compile against Python2.7 instead.

libffi 3.2.1 (this is the latest version released, Nov. 2014)

tar xvzf libffi-3.2.1.tar.gz
cd libffi-3.2.1
./configure --prefix=/afs/cern.ch/work/s/sevilla/software
make 
make install

glib 2.29.92

export PKG_CONFIG_PATH=/afs/cern.ch/work/s/sevilla/software/lib/pkgconfig:${PKG_CONFIG_PATH}
tar xvf glib-2.29.92.tar.xz
cd glib-2.29-92
./configure --prefix=/afs/cern.ch/work/s/sevilla/software
make 
make install

gobject-introspection-1.29.17

export PKG_CONFIG_PATH=/afs/cern.ch/work/s/sevilla/software/lib/pkgconfig:${PKG_CONFIG_PATH}
tar xvf gobject-introspection-1.29.17.tar.xz
cd gobject-introspection-1.29.17
./configure --prefix=/afs/cern.ch/work/s/sevilla/software
make
make install

pygobject-2.21.3

export PYTHON=/afs/cern.ch/work/s/sevilla/software/bin/python2.7
export PKG_CONFIG_PATH=/afs/cern.ch/work/s/sevilla/software/lib/pkgconfig:${PKG_CONFIG_PATH}
export LD_LIBRARY_PATH=/afs/cern.ch/work/s/sevilla/software/lib:${LD_LIBRARY_PATH}
tar xvf pygobject-2.90.4.tar.xz
cd  pygobject-2.90.4
./configure --prefix=/afs/cern.ch/work/s/sevilla/software
make
make install

pygtk-2.24.0

export PYTHON=/afs/cern.ch/work/s/sevilla/software/bin/python2.7
export PKG_CONFIG_PATH=/afs/cern.ch/work/s/sevilla/software/lib/pkgconfig:${PKG_CONFIG_PATH}
export LD_LIBRARY_PATH=/afs/cern.ch/work/s/sevilla/software/lib:${LD_LIBRARY_PATH}
tar xvf pygtk-2.24.0.tar.gz
cd pygtk-2.24.0
./configure --prefix=/afs/cern.ch/work/s/sevilla/software
make
make install

cairo-1.14.10

tar xvf cairo-1.14.10.tar.xz
cd cairo-1.14.10
./configure --prefix=/afs/cern.ch/work/s/sevilla/software
make
make install

pycairo-1-8-6

tar xvzf pycairo-1.8.6.tar.gz
cd pycairo-1.8.6
./configure --prefix=/afs/cern.ch/work/s/sevilla/software
make
make install

PyQt

PyQt, developed by Riverbank Computing Limited, is one of the two most popular Python bindings for the Qt cross-platform GUI/XML/SQL C++ framework (another binding is PySide). Qt itself is developed as part of the Qt Project. PyQt provides bindings for Qt 4 and Qt 5, and it is distributed under a choice of licences: GPL version 2, GPL version 3, or a commercial license.

PyQt is available in two editions:

  • PyQt4 which will build against Qt 4.x and 5.x and
  • PyQt5 which will only build against Qt 5.x.
Both editions can be built for Python 2 and 3. PyQt contains over 620 classes that cover graphical user interfaces, XML handling, network communication, SQL databases, Web browsing and other technologies available in Qt. The latest iteration of PyQt is v5.4. It fully supports Qt 5.4 which adds support for QtBluetooth, QtPositioning, QtMacExtras, QtWinExtras and QtX11Extras.

Qt (from source)

1.- Download Qt from https://www.qt.io/download-qt-for-application-development -> "Get your open source package" -> source code

ALERT! The latest Qt (5.9.2) needs at least gcc-4.7 and python 2.7 (lxplus6 defaults are gcc-4.4.7 and python 2.6.6).

2.- Install:

tar xvf qt-everywhere-opensource-src-4.8.6.tar
cd qt-everywhere-opensource-src-4.8.6
./configure --prefix=/afs/cern.ch/work/s/sevilla/software/qt-4.8.6
make -j4
make install

PyQt4

PyQt4 requires Qt (see above), and SIP

SIP v4:

Installation:

tar xvd sip-4.19.5.tar
cd sip-4.19.5
export SOFT=/afs/cern.ch/work/s/sevilla/software
python  configure.py --bindir=${SOFT}/bin --destdir=${SOFT}/lib64/python2.6/site-packages --incdir=${SOFT}/include/python2.6
make -j4
make install
After successful compilation, we'll have the following directories created:
/afs/cern.ch/work/s/sevilla/software/bin 
/afs/cern.ch/work/s/sevilla/software/include/python2.6
/afs/cern.ch/work/s/sevilla/software/lib64/python2.6/site-packages

PyQt4

Installation:

tar xvd PyQt4_gpl_x11-4.12.1.tar
cd PyQt4_gpl_x11-4.12.1
export SOFT=/afs/cern.ch/work/s/sevilla/software
python configure-ng.py --bindir=${SOFT}/bin --destdir=${SOFT}/lib64/python2.6/site-packages --no-stubs --no-tools --no-sip-files --qmake=${SOFT}/qt-4.8.6/bin/qmake --sip=${SOFT}/bin/sip --sip-incdir=${SOFT}/include/python2.6
make
and after several tries I gave up due to compilation errors frown

PyForms

Site: http://pyforms.readthedocs.io/en/v2.0/

% git clone https://github.com/UmSenhorQualquer/pyforms.git
% scl enable python33 bash
% mkdir -p /afs/cern.ch/work/s/sevilla/software/pyforms/lib/python3.3/site-packages
% export PYTONPATH=/afs/cern.ch/work/s/sevilla/software/pyforms/lib/python3.3/site-packages:${PYTHONPATH}
% cd pyforms
% python setup.py install --prefix=/afs/cern.ch/work/s/sevilla/software/pyforms
but when I try afterwards to run one of the examples provided with the code:
% cd pyforms/tutorials/1.SimpleExamples/SimpleExample1
% python SimpleExample1.py 
Traceback (most recent call last):
  File "SimpleExample1.py", line 13, in <module>
    from __init__ import *
  File "/afs/cern.ch/work/s/sevilla/pyforms/tutorials/1.SimpleExamples/SimpleExample1/__init__.py", line 13, in <module>
    from pyforms 			import BaseWidget
  File "/afs/cern.ch/work/s/sevilla/software/pyforms/lib/python3.3/site-packages/PyForms-2.1.0-py3.3.egg/pyforms/__init__.py", line 4, in <module>
    import loggingbootstrap
ImportError: No module named 'loggingbootstrap'
which can be installed with pip install git+https://bitbucket.org/fchampalimaud/logging-bootstrap.git., but this just starts to be cumbersome... ABANDONED

Jupyter

Jupyter Notebook is a web-based environment that enables interactive computing in notebook documents. It allows you to create and share documents that contain live code, equations, visualizations, and explanatory text. The Jupyter Notebook system is extensively used in domains such as data cleaning and transformation, numerical simulation, statistical modeling, machine learning, and much more.

  • Installation: sudo pip install jupyter

IPython

Python in OS

Although Python comes pre-installed on Mac OS X, due to Apple's release cycle it's often one or even two years old... The overwhelming recommendations around are to upgrade Python by installing a newer version. There are several options in the market:

  • MacPython: This is an installer of the latest Python framework build, along with some Mac-specific apps to help use it, like IDLE.app. This is the default recommendation of the MacPython community. See the Python download page for the Mac for instructions on how to download the installer and run it.

  • ActivePython: An up-to-date, commercial and free (as in beer) distribution of Python. ActivePython installers for a large number of platforms (including Mac OS X 10.3, 10.4 and greater, PowerPC and Intel) are available. Note that ActivePython does not include readline support, but it can be added.

  • Fink: a project that packages up many open-source projects in a Debian-like system of installers with explicit dependencies. A version of Python can be installed with Fink, but it's not recommended by some people in the MacPython community, mainly because too many prerequisite packages have to be installed for Python, and some of them seem to conflict with system packages.

  • MacPorts: another project bundling open-source software for Mac OS X and Darwin is called MacPorts. Again, Python can be installed with this software, but it isn't recommended by some people in the MacPython community, mainly because it's complicated and involves installing prerequisite packages. On the other hand, MacPorts probably has the by far largest amount of ready-to-run python modules available, all installable via a simple command. The project was previously known as DarwinPorts

  • Homebrew: free and open-source software package management system that simplifies the installation of software on Apple's macOS operating system. Homebrew is written in the Ruby programming language and targets the version of Ruby that comes installed with the macOS operating system.

ALERT! The Apple-supplied Python files are in /System/Library/Frameworks/Python.framework, and all python links within /usr/bin point to that location. One should never modify or delete these, as they are Apple-controlled and are used by Apple- or third-party software. Remember that if I choose to install a newer Python version, I will have two different but functional Python installations on my computer, so it will be important that the paths and usages are consistent with what I want to do, i.e, we must update our PATH accordingly to the newer files (for example to /opt/local/bin for the python binaries installed from MacPorts).

The python installations through MacPython and MacPorts seem the most promising. Though I got installed via homebrew !

Python distribution location binary links
OSX builtin Python /System/Library/Frameworks/Python.framework/Versions /usr/bin/pythonX.Y
python.org (MacPython) /Library/Frameworks/Python.framework/Versions /usr/local/bin/pythonX.Y
MacPorts /opt/local/Library/Frameworks/Python.framework/Versions /opt/local/bin/pythonX.Y
homebrew /usr/local/Cellar/python /usr/local/bin/pythonX.Y

Thumbs-up Homebrew

See also MacTipsSgs#Homebrew.

Homebrew is written in the Ruby programming language and targets the version of Ruby that comes installed with the macOS operating system. It is by default installed into /usr/local (with links to /usr/local/Cellar) and consists of a git repository, allowing the user to update Homebrew by pulling an updated repository from GitHub. The package manager builds software from source using "formulae", Ruby scripts constructed with Homebrew's domain specific language (DSL) for managing dependencies, downloading source files, and configuring and compiling software. Binary packages called "bottles" provide pre-built formulae with default options.

MacPython

  • select a given python release from http://www.python.org/download/releases/
  • download the corresponding package for OS
  • What we get after installation is:
    • a MacPython folder in the Applications folder. In here you find IDLE, the development environment that is a standard part of official Python distributions; PythonLauncher, which handles double-clicking Python scripts from the Finder; and the “Build Applet” tool, which allows you to package Python scripts as standalone applications on your system.
    • the installation of executable and libraries is performed in: /Library/Frameworks/Python.framework/Versions/ and /usr/local/bin (with symlinks pointing to the latter folder)
    • to uninstall MacPython, simply remove above three things.

https://trac.macports.org/wiki

  • install MacPorts: https://trac.macports.org/wiki/InstallingMacPorts
  • Add the option +universal at the end of /opt/local/etc/macports/variants.conf
  • sudo port -v selfupdate
  • sudo port install python27
  • sudo port install python31
  • The python binaries are installed in /opt/local/bin, pointing to /opt/local/Library/Frameworks/Python.framework/Versions/2.7/.
  • port_select has been replaced by port select:
    • to see the possible python runs: sudo port select --list python
    • to set a default python version: sudo port select --set python python27

Additional packages installed: numpy, cairo, py27-cairo, py27-opengl, py27-opengl-accelerate Other packages that are not needed to be installed: xorg-libX11, freetype, xpm

To uninstall MacPorts

  • sudo port -fp uninstall --follow-dependents installed
  • sudo rm -rf /opt/local /Library/Tcl/macports1.0/

Useful commands:

  • search: sudo port search numpy
  • show installed: sudo port installed | grep  py27

OpenGL packages:

  • download and extract each of them independently: OpenGLContext, PyVRML97, PyDispatcher
  • go to each directory and sudo python setup.py install
  • the stuff goes into /opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/

-- SergioGonzalez - 2016-08-29

Edit | Attach | Watch | Print version | History: r19 < r18 < r17 < r16 < r15 | Backlinks | Raw View | WYSIWYG | More topic actions
Topic revision: r19 - 2020-09-24 - SergioGonzalez
 
    • Cern Search Icon Cern Search
    • TWiki Search Icon TWiki Search
    • Google Search Icon Google Search

    Main All webs login

This site is powered by the TWiki collaboration platform Powered by PerlCopyright &© 2008-2021 by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
or Ideas, requests, problems regarding TWiki? use Discourse or Send feedback