jeudi 8 février 2018

HDFS MiniCluster Error on windows machine

I am trying to mock the hdfs with the following code but always getting this particular error.

test("some test") {

  val testDataPath = new File(PathUtils.getTestDir(getClass()), "miniclusters")

  //Configuration conf;

  //MiniDFSCluster cluster;

  //testDataPath = new File(PathUtils.getTestDir(getClass()), miniclusters");

  System.clearProperty(MiniDFSCluster.PROP_TEST_BUILD_DATA)
  val confMini = new HdfsConfiguration()

  val testDataCluster1 = new File(testDataPath, "CLUSTER_1")
  println(testDataCluster1)

  val c1Path = testDataCluster1.getAbsolutePath()
  println(c1Path)
  confMini.set(MiniDFSCluster.HDFS_MINIDFS_BASEDIR, c1Path)

  val cluster = new MiniDFSCluster.Builder(confMini).build()


  val fs = FileSystem.get(confMini);

  println(fs)

  assert(true)
}

The error is the Following

An exception or error caused a run to abort: org.apache.hadoop.io.nativeio.NativeIO$Windows.access0(Ljava/lang/String;I)Z 
java.lang.UnsatisfiedLinkError: org.apache.hadoop.io.nativeio.NativeIO$Windows.access0(Ljava/lang/String;I)Z
    at org.apache.hadoop.io.nativeio.NativeIO$Windows.access0(Native Method)

I am not sure about the error and whats the reason for the error.

Aucun commentaire:

Enregistrer un commentaire