Apache Spark MLlib: how to import a model from PMML

Question

I have a PMML file that encodes a logistic regression model that is NOT exported from MLlib.

How to import a model from PMML using MLlib in Java for estimation / forecasting?

(I know that MLlib can export to PMML , but I need to import from PMML)

Answer 1

To import, you need to perform PMML export operations in the reverse order:

• Retrieve interception ratios and functions from the PMML RegressionModel/RegressionTable element.
• Create a Spark ML LogisticRegressionModel object using these values.

This is my second time posting this answer. I wonder why the first answer was deleted (without discussion / explanation)?

Answer 2

Have you considered using a PMML loader like jpmml-spark ? You may have compatibility issues depending on where you built the model and which pmml exporter you used. I believe that sklearn2pmml is based on the jpmml library, so you should have good interoperability if you use them in combination.

Answer 3

You can use PMML4S-Spark to import PMML as a SparkML converter, and then make predictions / estimates in Scala, for example:

 import org.pmml4s.spark.ScoreModel val model = ScoreModel.fromFile("the/pmml/model/path") val scoreDf = model.transform(df) 

If you use PySpark, you can use PyPMML-Spark , for example:

 from pypmml_spark import ScoreModel model = ScoreModel.fromFile('the/pmml/model/path') score_df = model.transform(df)