how to execute Cloudera Hadoop(CDH4) Quickstart VM(CentOS 6.2) in Windows with VMware workstation

In order to execute Cloudera Hadoop CDH4 VM in Windows , you need to download the quickstart VM from here according to your VM version(i.e VMware/VirtualBox/KVM). It requires a 64 bit host OS. This VM runs CentOS 6.2 and includes CDH4.3, Cloudera Manager 4.6, Cloudera Impala 1.0.1 and Cloudera Search .9 Beta.

For this demo, I have used VMware version of Cloudera Quickstart VM for running on Windows 8 64 bit host OS.

Few points to ponder:

• This is a 64-bit VM, and requires a 64-bit host OS and a virtualization product that can support a
  64-bit guest OS.
• This VM uses 4 GB of total RAM. The total system memory required varies depending on the size
  of your data set and on the other processes that are running.
• The demo VM file is approximately 2 GB. Feel free to mirror internally or externally to minimize
  bandwidth usage.
• To use the VMware VM, you must use a player compatible with WorkStation 8.x or higher: Player 4.x or higher, ESXi 5.x or higher, or Fusion 4.x or higher. Older versions of WorkStation can be used to create a new VM using the same virtual disk (VMDK file), but some features in VMware Tools won’t be available.

• After downloading the Cloudera VM , extract it & select the virtual machine configuration (.vmx) file.

• Open the .vmx file by vmware workstation & start the VM.

 

 

 

• To start work on hadoop console, click on hue & login with default Id ‘admin’ & password ‘admin’

 

 

• Similarly, login to cloudera manager console with default user id ‘admin‘ & password ‘admin‘ in order to check hadoop cluster’s health.

 

• You can check the hadoop namenode cluster details with summary along with namenode logs along with HDFS clusters.

 

 

• Cloudera Hadoop (CDH4) VM contains inbuilt eclipse integrated with apache hadoop to write mapreduce jobs with ease.

 

• Once you open the eclipse integrated with hadoop, a default MapReduce java project is available which runs on Java SE 1.6

What’s new in Windows Azure SDK 2.1, Visual Studio 2013 Preview along with .NET framework 4.5.1

Recently announced the release of Windows Azure SDK 2.1 which can downloaded here comply with .NET framework 4.5.1 along with support of Visual Studio 2013 preview. 

Windows Azure for .NET SDK version number (2.1) is determined by the core Windows Azure service runtime assembly using semantic versioning. Other client libraries packaged inside the Windows Azure SDK for .NET, along with their respective NuGet packages, follow independent versioning based on the semantic versioning scheme.

The Windows Azure SDK for .NET 2.1 includes Windows Azure Storage 2.0.6, Windows Azure Service Bus 2.1, Windows Azure Caching 2.0, and Windows Azure Configuration Manager 2.0.1 libraries. You can also download these libraries from their respective NuGet packages.

• you need to install separately windows azure SDK 2.1 for VS 2013 preview.

 

• Install Windows Azure emulators, SDK as well as VS integrator components for 2.1 version. But this version would be final version to be supported on VS 2010 as per as Side by Side debugging policy.
• Side-by-Side Cloud Service Debugging is only supported with Windows Azure SDK for .NET versions 1.7, 1.8, 2.0, and 2.1. For a full debugging experience, projects built using versions prior to version 1.7 must be upgraded. For more information, see Windows Azure Cloud Services Support Lifecycle Policy.

  For example, when running a project built on Windows Azure SDK for .NET version 1.6 on a machine with version 2.1 installed, you will see the following error message- “The system is missing a prerequisite to execute the service.”

  To work around this error, uninstall the compute emulator included in version 2.1 and install thecompute emulator for version 1.6. If you do this, however, you will not be able to run an application built on SDK version 2.1 on this machine.

 

 

 

• Lets start to work with Windows Azure SDK 2.1 & start a new cloud project  selecting .net framework 4.5.

 

• Select the suitable webrole/worker role needs to be added to the project to start execution.

 

• One noticeable change is the deprecation of Silverlight Business app webrole, instead of a new worker role template of service bus queue has been added.
• For my demo, lets select an ASP.NET 4.5 webrole.

 

• In order to have custom settings , asp.net 4.5 settings , it offers different options for webrole selection from webforms , MVC, webAPI, Facebook & mobile apps. 
• One latest addition is SPA app on asp.net 4.5 which is an interactive frontend based app using HTML5, CSS3, javascript , jquery.
• This template has a built – in ‘todolist‘ app which I selected for my demo app preview. so , lets select the app.

 

• After selecting the app, start the execution of the app. The app starts in built to-do list sign in page.

 

• After sign up , you would be prompted to the home page of the app, where you can add your latest schedule of todo programs.

 

 

The following are breaking changes in the Windows Azure SDK for .NET 2.1:

• Creating an ASP.NET MVC 5 Web API project or SPA project in a new Cloud Service in Visual Studio 2013 Preview– When creating these project types through File->New->Windows Azure Cloud Service->ASP.NET Web Application, you must manually add an assembly redirect to ODataLib version 5.5 if you are using the Storage Client Library in your project.  This can be done in one of two ways:
  • Remove and re-add the Storage Client Library NuGet package by right-clicking your project References in Solution Explorer and selecting Manage NuGet Packages
  • Or, add the assembly redirect as called out in the build warning:
    C:\Program Files (x86)\MSBuild\12.0\bin\Microsoft.Common.CurrentVersion.targets(1613,5): warning MSB3247: 
    Found conflicts between different versions of the same dependent assembly. 
    Please add the following binding redirects to the "runtime" node in your application configuration file: 
    <assemblyBinding xmlns="urn:schemas-microsoft-com:asm.v1">
      <dependentAssembly>
        <assemblyIdentity name="Microsoft.Data.Edm" culture="neutral" publicKeyToken="31bf3856ad364e35" />
        <bindingRedirect oldVersion="0.0.0.0-5.5.0.0" newVersion="5.5.0.0" />
      </dependentAssembly>
    </assemblyBinding>
    <assemblyBinding xmlns="urn:schemas-microsoft-com:asm.v1">
      <dependentAssembly>
        <assemblyIdentity name="System.Spatial" culture="neutral" publicKeyToken="31bf3856ad364e35" />
        <bindingRedirect oldVersion="0.0.0.0-5.5.0.0" newVersion="5.5.0.0" />
      </dependentAssembly>
    </assemblyBinding>
    <assemblyBinding xmlns="urn:schemas-microsoft-com:asm.v1">
      <dependentAssembly>
        <assemblyIdentity name="Microsoft.Data.OData" culture="neutral" publicKeyToken="31bf3856ad364e35" />
        <bindingRedirect oldVersion="0.0.0.0-5.5.0.0" newVersion="5.5.0.0" />
      </dependentAssembly>
    </assemblyBinding>

• Running the Windows Azure compute emulator on the Windows 8.1 Preview– Internet Explorer in Windows 8.1 Preview has a new Enhanced Security mode which breaks the Start Without Debugging (CTRL+F5) functionality of the compute emulator. When the compute emulator starts, Internet Explorer launches and fails to load the web application hosted at the local host IP because http://127.0.0.x is not listed in Trusted sites. Start With Debugging (F5) functionality is not affected.
• Loading the ServiceRuntime from SDK 2.0 or later in a startup task throws an exception– To resolve, add the useLegacyV2RuntimeActivationPolicy attribute with a value of true to the startup element of the App.config file for the executable, as in the following example:
  <?xml version="1.0" encoding="utf-8" ?>
  <configuration>
      <startup useLegacyV2RuntimeActivationPolicy="true">
           ...
           ...
           ...
      </startup>
  </configuration>

• WebPI feed failure conditions–
  • The WebPI feed will fail to install if the Microsoft Root Certificate Authority 2010 or 2011 is missing.  For more information seehttp://support.microsoft.com/kb/931125.
  • The WebPI feed will silently fail on computers that are pending a reboot after installing updates or new software. To resolve this issue, restart the computer and then restart the setup.