java.lang.ClassNotFoundException at Map-Reduce Job

The problem is I have exported all third party libraries into the lib directory in the jar file, but I still receive java.lang.ClassNotFoundException. As we know, all libraries need to be distributed to all task nodes. In general, there are several ways to put the jar libraries:

  1. package into jar file
  2. use  “-libjars” command line option of the “hadoop jar “
  3. load libraries into nodes manually

The first method works well for manually installed Hadoop system, but failed at CDH4. My guess is the CDH has its own wrapper for Hadoop. So the simple solution is the second method: hadoop jar Model.jar -libjars lib/your_library.jar /model/input /model/output_element. Using “,” if you have more than one library.



Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s