All geek questions in one place

Hadoop IOException Login Error

I am new to Hadoop. However, I was able to successfully configure hadoop 2.7.3 with Java 7 in cluster mode on my servers. Everything works perfectly.

But then when I try to switch to Java 8 and run dfs, an error occurs:

Exception in thread "main" java.io.IOException: failure to login
    at org.apache.hadoop.security.UserGroupInformation.loginUserFromSubject(UserGroupInformation.java:824)
    at org.apache.hadoop.security.UserGroupInformation.getLoginUser(UserGroupInformation.java:761)
    at org.apache.hadoop.security.UserGroupInformation.getCurrentUser(UserGroupInformation.java:634)
    at org.apache.hadoop.hdfs.tools.GetConf.run(GetConf.java:315)
    at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
    at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:84)
    at org.apache.hadoop.hdfs.tools.GetConf.main(GetConf.java:332)
Caused by: javax.security.auth.login.LoginException: java.lang.NullPointerException: invalid null input: name
    at com.sun.security.auth.UnixPrincipal.<init>(UnixPrincipal.java:71)
    at com.sun.security.auth.module.UnixLoginModule.login(UnixLoginModule.java:133)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:498)
    at javax.security.auth.login.LoginContext.invoke(LoginContext.java:755)
    at javax.security.auth.login.LoginContext.access$000(LoginContext.java:195)
    at javax.security.auth.login.LoginContext$4.run(LoginContext.java:682)
    at javax.security.auth.login.LoginContext$4.run(LoginContext.java:680)
    at java.security.AccessController.doPrivileged(Native Method)
    at javax.security.auth.login.LoginContext.invokePriv(LoginContext.java:680)
    at javax.security.auth.login.LoginContext.login(LoginContext.java:587)
    at org.apache.hadoop.security.UserGroupInformation.loginUserFromSubject(UserGroupInformation.java:799)
    at org.apache.hadoop.security.UserGroupInformation.getLoginUser(UserGroupInformation.java:761)
    at org.apache.hadoop.security.UserGroupInformation.getCurrentUser(UserGroupInformation.java:634)
    at org.apache.hadoop.hdfs.tools.GetConf.run(GetConf.java:315)
    at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
    at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:84)
    at org.apache.hadoop.hdfs.tools.GetConf.main(GetConf.java:332)

    at javax.security.auth.login.LoginContext.invoke(LoginContext.java:856)
    at javax.security.auth.login.LoginContext.access$000(LoginContext.java:195)
    at javax.security.auth.login.LoginContext$4.run(LoginContext.java:682)
    at javax.security.auth.login.LoginContext$4.run(LoginContext.java:680)
    at java.security.AccessController.doPrivileged(Native Method)
    at javax.security.auth.login.LoginContext.invokePriv(LoginContext.java:680)
    at javax.security.auth.login.LoginContext.login(LoginContext.java:587)
    at org.apache.hadoop.security.UserGroupInformation.loginUserFromSubject(UserGroupInformation.java:799)
    ... 6 more

It seemed like hadoop could not login using my workers username. I try to find it from other sources and follow their instructions, for example, add an environment variable HADOOP_USER_NAME, delete and reinstall the entire datanode and namenode, etc. However, none of them work for me in this case.

, Java 7. Hadoop , . , Java, Hadoop . , ssh ​​ .

.bashrc fyi:

export JAVA_HOME="/scratch/dsat_server/jdk1.8.0_121"
export HADOOP_PREFIX="/scratch/dsat_server/hadoop-2.7.3"
export HADOOP_COMMON_LIB_NATIVE_DIR="$HADOOP_PREFIX/lib/native"
export HADOOP_CONF_DIR="$HADOOP_PREFIX/conf"
export HADOOP_OPTS="-Djava.net.preferIPv4Stack=truei-Djava.library.path=$HADOOP_PREFIX/lib"
+11
4

hbase Java 8. -, com.sun.security.auth.module.UnixLoginModule, unix, , NullPointerException. .

hadoop , :

UserGroupInformation.setLoginUser(UserGroupInformation.createRemoteUser("hduser"));

, . :

  • IBM JDK
  • ($ whoami). - " XXXX", /etc/passwd
+12

IDE.

Exception in thread "main" java.io.IOException: failure to login
        at org.apache.hadoop.security.UserGroupInformation.loginUserFromSubject(UserGroupInformation.java:822)
        at org.apache.hadoop.security.UserGroupInformation.getLoginUser(UserGroupInformation.java:774)
        at org.apache.hadoop.security.UserGroupInformation.getCurrentUser(UserGroupInformation.java:647)
        at org.apache.spark.util.Utils$$anonfun$getCurrentUserName$1.apply(Utils.scala:2464)
        at org.apache.spark.util.Utils$$anonfun$getCurrentUserName$1.apply(Utils.scala:2464)
        at scala.Option.getOrElse(Option.scala:121)
        at org.apache.spark.util.Utils$.getCurrentUserName(Utils.scala:2464)
        at org.apache.spark.SparkContext.<init>(SparkContext.scala:292)
        at org.apache.spark.SparkContext$.getOrCreate(SparkContext.scala:2486)
        at org.apache.spark.sql.SparkSession$Builder$$anonfun$7.apply(SparkSession.scala:930)
        at org.apache.spark.sql.SparkSession$Builder$$anonfun$7.apply(SparkSession.scala:921)
        at scala.Option.getOrElse(Option.scala:121)
        at org.apache.spark.sql.SparkSession$Builder.getOrCreate(SparkSession.scala:921)
        at SimpleApp$.main(SimpleApp.scala:17)
        at SimpleApp.main(SimpleApp.scala)
Caused by: javax.security.auth.login.LoginException:
java.lang.NullPointerException: invalid null input: name

.

UserGroupInformation.setLoginUser(UserGroupInformation.createRemoteUser("vyxx"))
+1

, jdk currupt. jdk jdk JAVA_HOME hadoop-env.sh, .

0

, , , jenkins. , jenkins Debian/Ubuntu:

# Add Jenkins user
groupadd --gid 1000 jenkins
useradd --uid 1000 --gid jenkins --shell /bin/bash --home-dir /var/jenkins_home jenkins
mkdir /var/jenkins_home
chown 1000:1000 /var/jenkins_home
echo 'jenkins ALL=NOPASSWD: ALL' >> /etc/sudoers.d/50-jenkins
echo 'Defaults    env_keep += "DEBIAN_FRONTEND"' >> /etc/sudoers.d/env_keep
0

Source: https://habr.com/ru/post/1014434/

More articles:


All Articles