Run the Pi Estimator Sample on Hadoop Services for Azure (video)

  • Article

View   http://youtu.be//w0BpLawwmKI

Hadoop-based Services for Windows Azure includes several samples you can use for learning and testing.In this video, Developer Brad Sarsfield walks you through the Pi Estimator sample.

See Also

Transcript (edited for readability)

Run the Pi Estimator Sample on Hadoop Services for Windows Azure (video)

Hi, my name is Brad Sarsfield and I'm a Developer on the Hadoop Services for Windows and Windows Azure team. In this video I'm going to take you through the Pi Estimator example on Windows Azure.

  1.  I’ve already created and allocated a Hadoop cluster.  I start by clicking on the Samples tile and going to the Sample Gallery page. 

  2. On the Sample Gallery page I see four samples (more coming soon). The first one is the Pi Estimator.
    The Pi Estimator job is an example of how to run a JAR job that estimates the value of Pi. In this example we’re going to use 16 maps and each of the maps will compute 10 million samples.

  3. I can download the PiEstimator.java file from here to see the source code, or I can f-download the hadoop-examples.jar which contains the compiled resources that are necessary to submit the job to the Hadoop cluster. But I’m going to click Deploy to your cluster.  This prepopulates the fields necessary for me to run this example on Hadoop.

  4. On the Create Job page I see that the Job Name, the JAR File, and the Parameters have been prepopulated.

  5. When I click Execute job, what will happen in the background is:

    • the Final Command, that I see here, is executed on the headnode of the Hadoop cluster,
    • and then I’ll be taken to another page which monitors the status of this job.

    After I click Execute job, the JAR file is uploaded to the headnode and the Hadoop command is run.

  6. What I see now is two sections: the Output and Errors.

    •  The Output section is used by the Java MapReduce program to output informational messages that are specific to, in this case, the Pi Estimator example.
    • The Standard Error or the Errors section displays information that Hadoop prints out and these are informational messages from Hadoop that describe how far the maps have progressed and how far the reduces have progressed.

  7. When the job finishes, as we see now, the informational messages about the statistics of the job have been outputted.

And that is the Pi example.

 Thanks for watching. Check back for more videos as we’re continually adding new videos.  This is the first in a series of increasingly-complex videos you will find on our website.