Passing data from a Java program to a Python program and getting results back

Question

What is the preferred way to pass data (list of strings) from a Java program to a Python script. The Python script does some data processing and then I need to return the results in my Java program.

Is there a framework that allows you to do this easily?

EDIT: More specific requirements.

My Java program is a scheduler (executed every X minutes and Y seconds) that connects to an external service and receives RAW data and sends it to python.

I can rewrite everything in Python, but it will take a lot of time. I looked to see if there is a way to reuse what I already have.

I want to use an existing Python script with minimal change. My python script uses a bunch of external libraries (e.g. numpy) The data transferred from Java to Python is in Json format, and the data returned by Python is also Json.

Using sockets is the parameters, but then I have to start the server processes.

Answer 1

I hacked it together a couple of months ago when I ran into a similar problem. I avoided Jython because I need separate processes. Java code is a server because it listens for requests, but it does not reconnect on error. The concept is that classes are extended threads that have a socket member, so send and receive commands can block object streams and not affect host streams.

Python Code:

import StringIO import re import select import socket import sys import threading class IPC(threading.Thread): def __init__(self, line_filter = None): threading.Thread.__init__(self) self.daemon = True self.lock = threading.Lock() self.event = threading.Event() self.event.clear() self.sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM) self.recv_buffer_size = 8192 self.buffer = StringIO.StringIO() if(line_filter == None): self.line_filter = lambda x: x else: self.line_filter = line_filter def run(self): self.sock.connect(("localhost", 32000)) data = True while data: try: data = self.sock.recv(self.recv_buffer_size) except socket.error, e: print e self.sock.close() break self.lock.acquire() self.buffer.write(data) self.lock.release() self.event.set() def readlines(self): self.lock.acquire() self.buffer.seek(0) raw_lines = self.buffer.readlines() self.buffer.truncate(0) self.lock.release() lines = map(self.line_filter, raw_lines) return lines proc_control = IPC() while True: proc_control.event.wait() data = proc_control.readlines() if(data): # Do Stuff proc_control.event.clear() 

Java Code:

SocketIPC.java:

 package project; import java.net.Socket; import java.net.ServerSocket; import java.io.BufferedReader; import java.io.BufferedWriter; import java.io.PrintWriter; import java.io.OutputStreamWriter; import java.io.InputStreamReader; import java.util.ArrayList; import java.util.List; public class SocketIPC { public PrintWriter out; public BufferedReader in; Socket socket = null; ServerSocket serverSocket = null; ConnectionListener connlisten = null; DataListener datalisten = null; Thread connlisten_thread = null; Thread datalisten_thread = null; CommandObject ipc_event_cmd = null; // Server thread accepts incoming client connections class ConnectionListener extends Thread { private int port; ConnectionListener(int port) { this.port = port; } @Override public void run() { try { serverSocket = new ServerSocket(port); socket = serverSocket.accept(); out = new PrintWriter(new BufferedWriter(new OutputStreamWriter(socket.getOutputStream())), true); in = new BufferedReader(new InputStreamReader(socket.getInputStream())); datalisten = new DataListener(); datalisten_thread = new Thread(datalisten); datalisten_thread.start(); } catch (Exception e) { System.err.println("SocketIPC creation error: " + e.getMessage()); } } } // Server thread accepts incoming client connections class DataListener extends Thread { String data_str = null; DataListener() { } @Override public void run() { try { while(true) { data_str = recv(); ipc_event_cmd.buffer.add(data_str); ipc_event_cmd.execute(); } } catch (Exception e) { System.err.println("SocketIPC reading error: " + e.getMessage()); } } public String read() { String ret_string = null; if(!ipc_event_cmd.buffer.isEmpty()) { ret_string = ipc_event_cmd.buffer.remove(0); } return ret_string; } } public SocketIPC(int port) { ipc_event_cmd = new CommandObject(); connlisten = new ConnectionListener(port); connlisten_thread = new Thread(connlisten); connlisten_thread.start(); } public void send(String msg) { if (out != null) { out.println(msg); } } public void flush() { if (out != null) { out.flush(); } } public void close() { if (out != null) { out.flush(); out.close(); try { in.close(); socket.close(); serverSocket.close(); } catch (Exception e) { System.err.println("SocketIPC closing error: " + e.getMessage()); } } } public String recv() throws Exception { if (in != null) { return in.readLine(); } else { return ""; } } public void set_cmd(CommandObject event_cmd) { if (event_cmd != null) { this.ipc_event_cmd = event_cmd; } } } 

CommandObject.java:

 package project; import java.util.List; import java.util.ArrayList; public class CommandObject { List<String> buffer; public CommandObject() { this.buffer = new ArrayList<String>(); } public void execute() { } } 

DoStuff.java:

 package project; import java.util.List; import java.util.ArrayList; import java.util.Map; import java.util.HashMap; import java.util.Random; public class DoStuff extends CommandObject { public DoStuff () { } @Override public void execute() { String tmp_string = null; while (!buffer.isEmpty()) { tmp_string = buffer.remove(0); // Do Stuff } } } 

Answer 2

Sounds like a job to Jython ! Jython is the Python built-in runtime written in Java. Until you need to run the Python script in another process (for example, if you want to kill it, you can use a lot of memory, etc.), This is the best way to date.

Answer 3

If you are trying to work together with Java and python, make your life simple with Jython .

Jython, successor to JPython, is a Python implementation of a programming language written in Java. Jython programs can import and use any Java class. With the exception of some standard modules, Jython programs use Java classes instead of Python modules. Jython includes almost all modules in the standard Python distribution language, only some of the implemented modules in C are missing.

Assuming you have java lib in your python path. Here is a snippet of code to give you an idea of ​​how easy it is to use java classes:

 ''' Import JavaUtilities class from a java package ''' from com.test.javalib import JavaUtilities ''' Call a java method ''' response = JavaUtilities.doSomething(); 

Answer 4

Please see Jython , which is best suited for communication between java and Python.