Psi Lambda LLC | Parallel computing made practical.

Kappa for Python

What is Kappa?

Kappa is an advanced software framework that makes it easy to bring the best performance to processing data by using relatively inexpensive highly parallel CPU and GPU hardware. Kappa works with your current programming languages and data sources. Kappa is ready to improve your TCO by implementing the production work loads of clusters of hundreds of servers on just a few servers.

Psi Lambda LLC is discontinuing support for Python

Due to the lack of multi-processor thread support in Python, Psi Lambda LLC is discontinuing support for Python. While usage of Kappa is stable with Python when Python is embedded within Kappa such that Kappa properly creates the interpreter, the common distributions of the Python interpreters do not properly initialize thread support and are subsequently unstable with Kappa’s multi-processor threaded execution. The 1.5.0 KappaCUDA Python package and the source code will remain available but no further versions or support will be offered. This will be reviewed and changed when:

  • Python adds multi-processor thread support.
  • The Python interpreters that are distributed enable full thread and multi-processor support

The source code for the KappaCUDA Python package and keyword are available under MIT License. Interested parties may take over maintenance and support.

Kappa for Python

The combination of the KappaCUDA Python module and the keyword Python gives Python as close to complete parity (other than performance) with C++ as GPU hardware (currently) allows. The KappaCUDA module allows access to CUDA via the Kappa Library while the keyword Python allows running Python subroutines on the host execution threads that have a CUDA GPU context (with access to the CUDA context, Variables, etc.).

This Python integration functionality gives CUDA programmers access to the very extensive Python libraries available on http://pypi.python.org/pypi.

Installation and usage

As of version 1.2, these files are available in the Kappa installation in either the kappa-langauges RPM or DEB file or the Languages section of the Windows installer. Please use one of those installations for versions newer than version 1.2. The files on this page are no longer the most current.They are installed under either /usr/share/kappa/extras or C:\Program Files\Kappa\share\kappa\extras. The Python keyword is in the keyword/Python subfolder and the package is in the lanuages/python subfolder.

Install the KappaCUDA module, after the Kappa Library and Kappa Library development files are installed, in the usual way for installing Python packages:

tar zxvf KappaCUDA-1.2.0.tar.gz
cd KappaCUDA-1.2.0
python3 setup.py install

Install the keywordpython by running the following commands (see the INSTALL file in the keyword/Python directory):

cd baseshare/kappa/extras/keyword/Python
cmake .
make
make install

(NOTE: there are FindPython3*.cmake files in the keyword/Python folder for finding the Python3 interpreter and libraries–see the INSTALL file or CMake documentation for proper usage of these files.)
Edit the keyword_python.conf configuration file and replace <Path to keywordpython> with the appropriate path–usually /usr/local/lib or /usr/local/lib64. This keyword_python.conf file also has a ‘[/kappa/python]‘ section that is only an example that corresponds to the python_example.py and keyword_python.py files that are distributed as part of keywordpython. In that section, the script_file parameter specifies the name of a Python script to be used by the keyword Python. Parameters of the form: <name>_prepare and <name>_execute translate a function name given in a Kappa command to corresponding Python subroutines. For example, if the Kappa command:

!Python -> name(1,2,3,A) [ A= ];

is given, then the ‘name’ has ‘_prepare’ and ‘_execute’ appended to it and are used to lookup Python subroutines names in the ‘[/kappa/python]‘ section of the configuration file. Example Python subroutines foo_prepare and foo_execute are present in the keyword_python.py Python scripts.

Once the keyword_python.conf file is configured correctly, copy it to the either the ‘/etc/kappa.d’ or the ‘~/.kappa.d’ directory.

If everything for the KappaCUDA module and keyword Python shared library is configured and installed properly, then running:

python3 python_example.py

(NOTE: there are stability problems at this time with this configuration: running python as the main program.)
or

ikappa -c python.k

should produce the following output:

Execute arguments: 4

Execute arguments: 1 2 3 /kappa/context/main/context/A

Variable dimensions: 2

Variable bounds: 2

Variable element size: 4

A firstrow 0 1 2 3 4 5 6 7 89 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47
A secondrow 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95

KappaCUDA and KeywordPython License terms

The KappaCUDA and KeywordPython files are licensed under the following MIT license:

Copyright (c)

Permission is hereby granted, free of charge, to any person
obtaining a copy of this software and associated documentation
files (the "Software"), to deal in the Software without
restriction, including without limitation the rights to use,
copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the
Software is furnished to do so, subject to the following
conditions:

The above copyright notice and this permission notice shall be
included in all copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES
OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT
HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY,
WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR
OTHER DEALINGS IN THE SOFTWARE.

Trivial modifications of KappaCUDA, KeywordPython, or KappaCUDA.i derived works do not qualify as software applications for the Kappa Library Developer License.

Articles