问题描述:

I get this issue when I run WordCount in eclipse.

log4j:WARN No appenders could be found for logger (org.apache.hadoop.util.Shell).

log4j:WARN Please initialize the log4j system properly.

log4j:WARN See http://logging.apache.org/log4j/1.2/faq.html#noconfig for more info.

Exception in thread "main" java.lang.NullPointerException

at java.lang.ProcessBuilder.start(Unknown Source)

at org.apache.hadoop.util.Shell.runCommand(Shell.java:482)

at org.apache.hadoop.util.Shell.run(Shell.java:455)

at org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:715)

at org.apache.hadoop.util.Shell.execCommand(Shell.java:808)

at org.apache.hadoop.util.Shell.execCommand(Shell.java:791)

at org.apache.hadoop.fs.RawLocalFileSystem.setPermission(RawLocalFileSystem.java:656)

at org.apache.hadoop.fs.RawLocalFileSystem.mkdirs(RawLocalFileSystem.java:444)

at org.apache.hadoop.fs.FilterFileSystem.mkdirs(FilterFileSystem.java:293)

at org.apache.hadoop.mapreduce.JobSubmissionFiles.getStagingDir(JobSubmissionFiles.java:133)

at org.apache.hadoop.mapreduce.JobSubmitter.submitJobInternal(JobSubmitter.java:145)

at org.apache.hadoop.mapreduce.Job$10.run(Job.java:1297)

at org.apache.hadoop.mapreduce.Job$10.run(Job.java:1294)

at java.security.AccessController.doPrivileged(Native Method)

at javax.security.auth.Subject.doAs(Unknown Source)

at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1656)

at org.apache.hadoop.mapreduce.Job.submit(Job.java:1294)

at org.apache.hadoop.mapreduce.Job.waitForCompletion(Job.java:1315)

at WordCount.main(WordCount.java:81)

Any ideas on where the problem might be?

网友答案:

you need to add these winutils files to your $HADOOP_HOME/bin folder.

网友答案:

You need to add few jars from your hadoop-version/share/hadoop folder to run the program in eclipse locally on a linux machine.

Hadoop common 1.2.1 jar

You need to download the above jar file separately. You can follow the below blog link to know the complete procedure.

https://acadgild.com/blog/running-mapreduce-in-local-mode-2/

相关阅读:
Top