All geek questions in one place

Java - MySQL to Hive Import, where MySQL runs on Windows and Hive runs on Cent OS (Horton Sandbox)

Before any answer and comments. I tried several options that I found on Stackoverflow, but ended up crashing. Below are the links -

I tried it in Horton Sandbox through the command line and succeed.

sqoop import --connect jdbc: mysql: //192.168.56.101: 3316 / database_name --username = user --password = pwd --table table_name --hive-import -m 1 - --schema default

Where 192.168.56.101 for Windows and 192.168.56.102 for Horton Sandbox 2.6.

Now I want to do the same with Java, where this Java code runs somewhere else, but not in the horton sandbox.

  • How to decompose the parameters of HIVE_HOME and other Sqoop, since they work in Sandbox.
  • the options that I have to go through. It should be passed as SqoopOptions or Sqoop.runTools String Array Arguments. Both refused.
  • I am also confused. For now, import the library (com.cloudera.sqoop and org.apache.sqoop) and get this

The execution of the method (com.cloudera.sqoop.SqoopOptions) in the ImportTool type is not applicable to arguments (org.apache.sqoop.SqoopOptions) with these two lines (the option parameter is added between these two lines)

 SqoopOptions options = new SqoopOptions();
 int ret = new ImportTool().run(options);

if I select the Cloudera method, it will become obsolete, but if I select the apace run method, it will not accept the options argument

It struck me for weeks. Please, help.

+3
1

, ssh. Horton Sandbox ssh. sqoop ssh- . , ( , java), .

import net.neoremind.sshxcute.core.SSHExec;
import net.neoremind.sshxcute.core.ConnBean;
import net.neoremind.sshxcute.task.CustomTask;
import net.neoremind.sshxcute.task.impl.ExecCommand;

public class TestSSH {

public static void main(String args[]) throws Exception{

    // Initialize a ConnBean object, parameter list is ip, username, password

    ConnBean cb = new ConnBean("192.168.56.102", "root","hadoop");

    // Put the ConnBean instance as parameter for SSHExec static method getInstance(ConnBean) to retrieve a singleton SSHExec instance
    SSHExec ssh = SSHExec.getInstance(cb);          
    // Connect to server
    ssh.connect();
    CustomTask sampleTask1 = new ExecCommand("echo $SSH_CLIENT"); // Print Your Client IP By which you connected to ssh server on Horton Sandbox
    System.out.println(ssh.exec(sampleTask1));
    CustomTask sampleTask2 = new ExecCommand("sqoop import --connect jdbc:mysql://192.168.56.101:3316/mysql_db_name --username=mysql_user --password=mysql_pwd --table mysql_table_name --hive-import -m 1 -- --schema default");
    ssh.exec(sampleTask2);
    ssh.disconnect();   
}
}
+3

Source: https://habr.com/ru/post/1545563/

More articles:


All Articles