Java Python Integration

I have a Java app that needs to integrate with a 3rd party library. The library is written in Python, and I don't have any say over that. I'm trying to figure out the best way to integrate with it. I'm trying out JEPP (Java Embedded Python) - has anyone used that before? My other thought is to use JNI to communicate with the C bindings for Python.

Any thoughts on the best way to do this would be appreciated. Thanks.

Why not use Jython? The only downside I can immediately think of is if your library uses CPython native extensions.

EDIT: If you can use Jython now but think you may have problems with a later version of the library, I suggest you try to isolate the library from your app (e.g. some sort of adapter interface). Go with the simplest thing that works for the moment, then consider JNI/CPython/etc if and when you ever need to. There's little to be gained by going the (painful) JNI route unless you really have to.

Frankly most ways to somehow run Python directly from within JVM don't work. They are either not-quite-compatible (new release of your third party library can use python 2.6 features and will not work with Jython 2.5) or hacky (it will break with cryptic JVM stacktrace not really leading to solution).

My preferred way to integrate the two would use RPC. XML RPC is not a bad choice here, if you have moderate amounts of data. It is pretty well supported — Python has it in its standard library. Java libraries are also easy to find. Now depending on your setup either Java or Python part would be a server accepting connection from other language.

A less popular but worth considering alternative way to do RPCs is Google protobuffers, which have 2/3 of support for nice rpc. You just need to provide your transport layer. Not that much work and the convenience of writing is reasonable.

Another option is to write a C wrapper around that pieces of Python functionality that you need to expose to Java and use it via JVM native plugins. You can ease the pain by going with SWIG SWIG.

Essentially in your case it works like that:

1. Create a SWIG interface for all method calls from Java to C++.
2. Create C/C++ code that will receive your calls and internally call python interpreter with right params.
3. Convert response you get from python and send it via swig back to your Java code.

This solution is fairly complex, a bit of an overkill in most cases. Still it is worth doing if you (for some reason) cannot afford RPCs. RPC still would be my preferred choice, though.

Many years later, just to add an option which is more popular these days...

If you need CPython functionality, py4j is a good option. py4j has seen seen frequent updates in 2016 2017 2018 2019 2020 and has gained some popularity, because it is used e.g. by Apache Spark to achieve CPython interoperability.

The best solutions, is to use Python programs throw REST API. You define your services and call them. You perhaps need to learn some new modules. But you will be more flexible for futures changes.

Here a small list of use full modules for this purpose: Python modules

• Flask
• Flask-SQLAlchemy
• Flask-Restful
• SQlite3
• Jsonify

Java modules (for calling rest api) Jersey or Apache CXF

You will need a small Learning curve, but later you will get more productivity and modularity and even elasticity...

> My other thought is to use JNI to communicate with the C bindings for Python.

I like very much JNA:

> JNA provides Java programs easy access to native shared libraries (DLLs on Windows) without writing anything but Java code—no JNI or native code is required. This functionality is comparable to Windows' Platform/Invoke and Python's ctypes. Access is dynamic at runtime without code generation.

My 0.02$ :)

You could use a messaging service like ActiveMQ. It has both Python and Java support. This way, you can leave the complicated JNI or C bindings as they are and deal solely with what I consider a simple interface. Moreover, when the library gets updated, you don't need to change much, if anything.

Have you considered running Jython on the Java VM?

I've investigated a similar setup with JNI. Maybe this will help if haven't seen it yet:

http://wiki.cacr.caltech.edu/danse/index.php/Communication_between_Java_and_Python

http://jpe.sourceforge.net/

These are some of the tools which make it easier to bridge the gap between Python and Java:

1.Jython Python implemented in Java

2.JPype Allows Python to run java commands

3.Jepp Java embedded Python

4.JCC a C++ code generator for calling Java from C++/Python

5.Javabridge a package for running and interacting with the JVM from CPython

6.py4j Allows Python to run java commands.

7.voc Part of BeeWare suite. Converts python code to Java bytecode.

8.p2j Converts Python code to Java. No longer developed.

If you can get your Python code to work in Jython, then you should be able to use that to call it from Java:

• http://jython.sourceforge.net/cgi-bin/faqw.py?req=show&file=faq06.001.htp

I also think that run command line in the Java wouldn't by bad practice (stackoverflow question here). Potentially share the data through some database.

I like the way to connect two apps via the bash pipe, but I have not practice in this, so I'm wondering how difficult is to write logic to handle this on both python/java sides.

Or other productive way could be to use the Remote Procedure Call (RPC) which supports procedural programming. Using RPC you can invokes methods in shared environments. As an example you can call a function in a remote machine from the local computer using RPC. We can define RPC as a communication type in distributed systems. (mentioned above by Marcin)

Or, very naive way would to be to communicate by the common file. But for sake of simplicity and speed, my vote is to use the shared database x rest API x socket communication.

Also I like the XML RPC as Marcin wrote.

> I would like to recommend to avoid any complication to run Python under JVM or C++ binding. Better to use today trends which are obviously web technologies.

As a shared database the MongoDB may be good solution or even better the Redis as in memory database.