Deployment of Cloudera Enterprise 5.4.4(CDH 5) on Microsoft Azure Virtual Machine & Running Impala shell as single node cluster


Deployment of Cloudera Enterprise (CDH) 5.4.4 can be implemented directly on Microsoft Azure Virtual Machines  & we can start working on Impala shell & Hue itself.

The hosting process is super easy, just need to make sure the following prerequisites & troubleshooting steps should be taken care off.

Prerequisites :

  1. SELinux should be disabled,

Before disabling SELinux you may try sysctl -w vm.swappiness=0.

You have to add the line below in /etc/sysctl.conf to keep your change permanently:

vm.swappiness = 10

  1.  Change the root password
  2. Change the hostname in /etc/hosts file
  3. Add ports 7180, 7182, 9000, 9001 open
  4. Passwordless sudo user authentication
  5. Change the /etc/hosts file , the hostname from hosts IP address by $ifconfig

 Issue: Cloudera Manager site is not opening on browser after installation & the following error shows on log

cloudera-scm-server dead but pid file exists

Follow the steps:

# service cloudera-scm-server stop

# service cloudera-scm-server-db stop

# rm /var/run/cloudera-scm-server.pid

# service cloudera-scm-server-db start

# service cloudera-scm-server start

Details about the step by step process of deployment of CDH 5 on MS Azure Virtual Machine(RHEL 6.x) can be viewed on YouTube channel.

Advertisements

Developing & Deploying Node.js Web application into Windows Azure with Cloud9 IDE


While it’s common to run Node.js applications  & deploy them into Windows Azure. Lets start by building a simple Demo of building a Node.js app in Cloud9 IDE & deploy it into Windows Azure. Node.js is an open – source toolkit for developing Windows Azure based apps. Cloud9 is a browser based cross-platform development environment supports for Node.js projects which can be directly deployed to Windows Azure.

  • Cloud9 IDE is an ideal browser tool for Chrome -OS & works in OS X , Linux & Windows.
  • To use Cloud9 IDE, you need to Sign up to Cloud9 portal @ http://c9.io/
  • Alternatively you can Sign in using Github Id.
  • So, lets first create a Cloud9 Project , click on the icon ‘Create a new project’ .

  • Select the name of your cloud project  & click on Create to create the project.
  • Start Editor after the project is being created. You would be prompted for dashboard preview but is you need to code directly click on Just the editor, please.
  • Start coding by selecting New File from the menu. Paste the following code in the file.

var http = require(‘http’);

var port = process.env.PORT;

http.createServer(function(req,res)

{

res.writeHead(200, {‘Content-Type’: ‘text/plain’});

res.end(‘hello windows azure\n’); }).listen(port);

  • Next save the file as ‘server.js’ from the Save As menu.

  • Start debugging & preview the output in browser as cloudapp.net app.

  • Now, deploy the app by clicking on Deploy button from the button grid options & select Windows Azure as deployment platform.
  • Next, you will be prompted to Windows Azure portal  & start migrting your on-premise node.js web app in Cloud9 IDE app into Windows Azure.

 

 

Integration of Windows LIVE SDK Connect & Push Notifications in Windows 8 Metro Style Apps using Windows Live SDK


Windows Live SDK is quite essential for integrating Windows LIVE CONNECT with Metro Style Apps as well as with LIVE Push Notifications. In this article we will see how to integrate Windows LIVE CONNECT & Push Notification services from Windows Azure (Cloud) to Windows 8 Metro Style Apps.  First of all , you need to Download & install Windows Live SDK from

http://www.microsoft.com/download/en/details.aspx?displaylang=en&id=28195

Now, Install the Live SDK & Check out the Extension on VS 11 Developer Preview in Add Reference section.

  • Now create a new Metro Style project in VS 11 Developer Preview .

  • Now write some simple code in MainPage.xaml:

<Grid x:Name=”LayoutRoot” Background=”#FF0C0C0C”>

<Grid.ColumnDefinitions>

<ColumnDefinition Width=”251*” />

<ColumnDefinition Width=”1116*” />

</Grid.ColumnDefinitions>

<StackPanel Grid.ColumnSpan=”2″>

<live:SignInButton Name=”btnLogIn” Scopes=”wl.signin wl.basic” />

<TextBlock Name=”tbName” Width=”600″ Height=”150″ FontSize=”32″ TextWrapping=”Wrap” />

<TextBlock Name=”tbGender” Width=”600″ Height=”150″ FontSize=”32″ TextWrapping=”Wrap” />

<TextBlock Name=”tbLiveProfile” Width=”600″ Height=”150″ FontSize=”32″ TextWrapping=”Wrap” />

<TextBlock Name=”tbError” Text=”Error Message” Width=”600″ Height=”150″ FontSize=”32″ />

</StackPanel>

</Grid>

  • Same way modify MainPage.xaml.cs :

using System;

using System.Collections.Generic;

using System.Linq;

using System.Threading.Tasks;

using Windows.Foundation;

using Windows.UI.Xaml;

using Windows.UI.Xaml.Controls;

using Windows.UI.Xaml.Data;

using Microsoft.Live;

using  Microsoft.Live.Controls;

namespace LiveSDKDemo

{

partial class MainPage

{

private LiveConnectClient liveClient;

private LiveConnectSession session;

 public MainPage()

{

InitializeComponent();

      this.btnLogIn.SessionChanged += btnLogIn_OnSessionChanged;

}

private void btnLogIn_OnSessionChanged(object sender, LiveConnectSessionChangedEventArgs e)

{

if(e.Session != null && e.Status == LiveConnectSessionStatus.Connected)

{

this.liveClient = newLiveConnectClient(e.Session);

session = e.Session;

this.liveClient.GetCompleted += OnGetCompleted;

this.liveClient.GetAsync(“me”,null);

}

   else

{

this.liveClient = null;

}

}

private  void OnGetCompleted(object sender, LiveOperationCompletedEventArgs e)

{

if (e.Error == null)

{

dynamic result = e.Result;

his.tbName.Text = “Hello,” + result.first_name + ” “ + result.last_name;

this.tbGender.Text = “You are “ + result.gender + “that lives in “ + result.locale + “.”;

this.tbLiveProfile.Text = “Your Live Profile can be found:” + result.link;

}

else

{

this.tbError.Text = e.Error.ToString();

}

}

}

}

  • Now , after pressing F5 , you will be stuck with the following message:

  • You need to login to https:manage.dev.live.com  with your Windows Live ID & add the Metro Style Apps Package Name & Publisher Name in Windows Live Connect Portal to integrate LIVE SDK CONNECT & Push notifications with your Metro Style Apps.

  • Accept the  Terms & Condition to get the Live Application Package Name ID & paste it in your package.appxmanifest file.

  • Now Press F5 & Test it in Windows 8 Simulator & check out the following output.

  • Click on Yes & you will be signed in with Windows LIVE SDK Connect & enable to receive Push notifications in your Metro style apps.

SharePoint 2010 with Windows Phone 7 Traning Kit is now Available


Accessing SharePoint 2010 applications are quite easy with the help of Windows Phone 7 which integrates directly with SharePoint. The training kit helps to leverage custom development using Microsoft SharePoint 2010 applications with Windows Phone 7 for OAuth services, Twitter API, Windows Azure, SQL Azure , Office 365, LinkedIn API, Silverlight Web parts & Bing Maps API.

Download the training kit from here: http://www.microsoft.com/download/en/details.aspx?id=26813

  • One of the great tool sharepoint 2010 with Windows Phone 7 toolkit is now available in codeplex which helps the windows phone 7 apps to access the SharePoint 2010 applications using Forms authentication.
  • It downloads lists of data with prepackaged filters to limit the data without writing CAML.

 

Building Silverlight Bing Maps based API with Cloud application in SharePoint 2010


Developing Bing Maps API based Silverlight applications are quite easy integrating with Windows Azure , SQL Azure databases with Microsoft SharePoint 2010. Bing Maps API provides a rich set of geolocation APIs which helps to easy integrate with Silverlight, Windows Azure application.

  • To use Bing Maps with Silverlight Web applications , you need to get the Bing Maps Developer key which can be obtained from www.bingmapsportal.com , register there with your Windows Live ID & get a developer key by clicking on create or view key.

  •  Next, you need keep the key in clipboard & should be used while developing your Silverlight application. Now create a new Silverlight application from Visual Studio & modify the following XAML into it.

  • Write in MainPage.XAML :

<UserControl x:Class=”SimpleBingMap.MainPage”

xmlns=”http://schemas.microsoft.com/winfx/2006/xaml/presentation&#8221;

xmlns:x=”http://schemas.microsoft.com/winfx/2006/xaml&#8221;

xmlns:d=”http://schemas.microsoft.com/expression/blend/2008&#8243;

xmlns:m=”clr-namespace:Microsoft.Maps.MapControl;assembly=Microsoft.Maps.MapControl”        

xmlns:mc=”http://schemas.openxmlformats.org/markup-compatibility/2006&#8243;

mc:Ignorable=”d”

d:DesignHeight=”300″ d:DesignWidth=”400″>

<Grid x:Name=”LayoutRoot” Background=”White” HorizontalAlignment=”Stretch”

VerticalAlignment=”Stretch” Width=”888″ Height=”394″>

<Grid.ColumnDefinitions>

<ColumnDefinition Width=”888*” />

<ColumnDefinition Width=”314*” />

</Grid.ColumnDefinitions>

<Grid.RowDefinitions>

<RowDefinition Height=”auto” />

<RowDefinition Height=”*” />

</Grid.RowDefinitions>

<StackPanel Grid.ColumnSpan=”2″>

<TextBlock Text=”Map Me!” FontWeight=”Bold” FontSize=”14″ Width=”888″ />

<StackPanel Orientation=”Horizontal” HorizontalAlignment=”Stretch” Margin=”12,0,12,0″>

<TextBox x:Name=”txtbxLatitude” Width=”322″ Text=”Type your latitude here,e.g. 47.7656″ FontSize=”10″ />

</StackPanel>

<StackPanel Orientation=”Horizontal” HorizontalAlignment=”Stretch” Margin=”12,0,12,0″>

<TextBox x:Name=”txtbxLongitude” Width=”322″ Text=”Type your longitude here, e.g. -122.9957″ FontSize=”10″ />

</StackPanel>

<StackPanel Orientation=”Horizontal” HorizontalAlignment=”Stretch” Margin=”12,0,12,0″>

<Button x:Name=”btnFindMe” Content=”Map It!” Width=”65″ Click=”btnFindMe_Click” />

</StackPanel>

</StackPanel>

<m:Map CredentialsProvider=”Av1a04p5K8Ubd97aV5dMMX7tRSbDxWz8hm01_4ZA8BA2kE7o56VvsNfjMOOrTnV5″ x:Name=”MyMap” Grid.Row=”1″ Mode=”AerialWithLabels” HorizontalAlignment=”Stretch” VerticalAlignment=”Stretch” Margin=”0,14,0,0″ Grid.ColumnSpan=”2″>

<m:Map.Children>

<m:MapLayer x:Name=”PushPinLayer” />

</m:Map.Children>

</m:Map>

</Grid>

</UserControl>

  • Write in MainPage.XAML.CS:

using System;

using System.Collections.Generic;

using System.Linq;

using System.Net;

using System.Windows;

using System.Windows.Controls;

using System.Windows.Documents;

using System.Windows.Input;

using System.Windows.Media;

using System.Windows.Media.Animation;

using System.Windows.Shapes;

using  Microsoft.Maps.MapControl;


namespace SimpleBingMap

{

public partial class MainPage : UserControl

{

public MainPage()

{

InitializeComponent();

}

private  void btnFindMe_Click(object sender, RoutedEventArgs e)

{

Location myLocation = newLocation();

myLocation.Latitude = Double.Parse(txtbxLatitude.Text);

myLocation.Longitude = Double.Parse(txtbxLongitude.Text);

// Create a new pushpin to add to the map.

Pushpin myPPin = newPushpin();

myPPin.Width = 7;

myPPin.Height = 10;

myPPin.Location = myLocation;

PushPinLayer.AddChild(myPPin, myLocation, PositionOrigin.Center);

// Set the main view of the map using location with a zoom level

MyMap.SetView(myLocation, 10);

}

}

}

 

  • Map yourself in Bing Maps API by providing Latitude & Longitude:

  • Now, it’s time to integrate with your SharePoint 2010 website. Open the Add Documents Library option & Click on Add, Click browser & paste the path of Silverlight .xap file in SharePoint directory.
  • Click on Site Settings & Click on Edit Page to open the .xap file in SharePoint Web Part, Now open the insert tab-> Click on Media Content-> Silverlight Web Part -> paste the path of the Silverlight .xap file in the clipboard. Access it in Silverlight portal.

 

  • Configure the height & width according to configuration :

  • That’s it, The SharePoint applications is integrated with Silverlight web part along with Bing Maps API integrated in Cloud.

Consuming SQL Server/SQL Azure Data in Windows Phone 7.1 & Android through OData WCF Data Services & Entity Frameworks


Consuming data from SQL Server or SQL Azure database in Client Smart Devices like Windows Phone 7.1 , Android , iPhone or iPad is quite easy through the use of OData WCF Services. In previous posts , I already described how to consume data from live OData AtomPub feed showed in XML format. But, in this article I want to show how to consume SQL Server / SQL Azure data in Windows Phone & Android devices with the help of OData WCF services & entity frameworks where the service will be hosted in remote IIS & deployed SQL Server/SQL Azure database instance.

  • For creating solutions, let’s first start by creating an empty asp.net web application from Visual Studio.

  • Start by connecting with your SQL Server / SQL Azure database with Visual Studio .

  • Next, add data in the tables of SQL Server / SQL Azure database.

  • Next, Add an ADO.NET Entity Framework Model from Visual Studio to connect with SQL Server / SQL Azure database & save the connection string in web.config.

  • Connect with your database with proper credentials

  • Click on Next to select Tables, Views , Stored Procedures to select data.

  •  Now Add a WCF Service in the project & modify the code as like this:

using System;

using System.Collections.Generic;

using System.Data.Services;

using System.Data.Services.Common;

using System.Linq;

using System.ServiceModel.Web;

using System.Web;

namespace ODataSQLWP7

{

public class CustomerService : DataService<CustomersEntities>

{

// This method is called only once to initialize service-wide policies.

public static void InitializeService(DataServiceConfiguration config)

{

// TODO: set rules to indicate which entity sets and service operations are visible, updatable, etc.

// Examples:

// config.SetEntitySetAccessRule(“MyEntityset”, EntitySetRights.AllRead);

// config.SetServiceOperationAccessRule(“MyServiceOperation”, ServiceOperationRights.All);

config.SetEntitySetAccessRule( “CustomerInfoes”, EntitySetRights.All);

config.DataServiceBehavior.MaxProtocolVersion = DataServiceProtocolVersion.V2;

}

}

}

  •  Now check the service in browser & it looks like this:

  • Reason of the error: It’s due to the permission set not configured for IIS to deploy the SQL Server/SQL Azure database data. For SQL Azure database while creating the database in Windows Azure portal , you need to set the firewall settings definitely to 0.0.0.0 to 255.255.255.255 in order to deploy the SQL Azure database in IIS.
  • Now Right-Click on the project , select Property -> Package/Publish Web tab: 

  • Remember to check the settings: Include all databases in  Package/Publish SQL tab.
  •  Next , Click on Package/Publish SQL options: Click on Import from Web.config to generate your SQL Server/SQL Azure database connection strings.
  • Click on Connection String for destination database & add your destination database which will be hosted in IIS.
  • Click on the checkbox under Source database information: Pull data/or schema from existing database.
  • Under database scripting options select Schema only (if you would like generate schema only) or Schema & Data for generating schema & data deployement both.

  • Now , add user to login to the deployed database from SQL Server Management Studio by right clicking on Security -> New Login 

  • Add user for login as ODataUser from Windows Add Users – Groups utility , alternatively , you can add user from right-clicking My Computer -> Manage-> Users & Groups -> User -> Add User.

  • Now under the User Mapping tab add the database that you would like to deploy & click on db_datareader & db_datawriter. Click OK.

  • Deploy the Web Service in IIS from Web Deploy wizard in Visual Studio.

  • Next, open the Application Pool in IIS manager which holds the deployed service & click on advanced settings : Select Process Model as Application Pool Idenity -> Under Custom account Set the account credentials which you have set under the SQL Server logins section.

  • After this settings, now access the Customer Info  with the Service URL settings & it will show the entire RSS feed in AtomPub/ JSON format from the SQL Server / SQL Azure database.

  • It means OData WCF service now able to access the SQL Server/ SQL Azure data in AtomPub format hosted in remote IIS. Next , It’s time to create the Client applications:
  • First take a Windows Phone Panorama Project & modify the MainPage.XAML as like this:

<controls:PanoramaItem Header=”first item”>

<!–Double line list with text wrapping–>

<ListBox x:Name=”lst” Margin=”0,0,-12,0″ ItemsSource=”{Binding}”>

<ListBox.ItemTemplate>

<DataTemplate>

<StackPanel Margin=”0,0,0,17″ Width=”432″>

<TextBlock Text=”{Binding FirstName}” TextWrapping=”Wrap” Margin=”12,-6,12,0″ Style=”{StaticResource PhoneTextExtraLargeStyle}” />

<TextBlock Text=”{Binding LastName}” TextWrapping=”Wrap” Margin=”12,-6,12,0″ Style=”{StaticResource PhoneTextExtraLargeStyle}” />

<TextBlock Text=”{Binding Address}” TextWrapping=”Wrap” Margin=”12,-6,12,0″ Style=”{StaticResource PhoneTextSubtleStyle}” />

<TextBlock Text=”{Binding City}” TextWrapping=”Wrap” Margin=”12,-6,12,0″ Style=”{StaticResource PhoneTextSubtleStyle}” />

<TextBlock Text=”{Binding State}” TextWrapping=”Wrap” Margin=”12,-6,12,0″ Style=”{StaticResource PhoneTextSubtleStyle}” />

<TextBlock Text=”{Binding Zip}” TextWrapping=”Wrap” Margin=”12,-6,12,0″ Style=”{StaticResource PhoneTextSubtleStyle}” />

</StackPanel>

</DataTemplate>

</ListBox.ItemTemplate>

</ListBox>

</controls:PanoramaItem>

  • Modify MainPage.XAML.CS:

 

using System;

using System.Collections.Generic;

using System.Linq;

using System.Net;

using System.Windows;

using System.Windows.Controls;

using System.Windows.Documents;

using System.Windows.Input;

using System.Windows.Media;

using System.Windows.Media.Animation;

using System.Windows.Shapes;

using Microsoft.Phone.Controls;

using WP7PanoramaOData.CustomersModel;

using System.Data.Services.Client;


namespace WP7PanoramaOData

{

public partial classMainPage : PhoneApplicationPage

{

// Constructor

public MainPage()

{

InitializeComponent();

// Set the data context of the listbox control to the sample data

DataContext = App.ViewModel;

this.Loaded += newRoutedEventHandler(MainPage_Loaded);

}

//   Load data for the ViewModel Items

private void MainPage_Loaded(object sender, RoutedEventArgs e)

{

var ctx = newCustomersEntities(newUri(http://10.12.1.223/ODataSQLWP7/CustomerService.svc/&#8221;));

var coll = new System.Data.Services.Client.DataServiceCollection<CustomerInfo>(ctx);

lst.ItemsSource = coll;

coll.LoadCompleted += new EventHandler<LoadCompletedEventArgs>(coll_LoadCompleted);

var qry = “/CustomerInfoes”;

coll.LoadAsync(new Uri(qry, UriKind.Relative));

}

void coll_LoadCompleted(object sender, LoadCompletedEventArgs e)

{

if (e.Error != null)

{

MessageBox.Show(e.Error.Message);

}

}

  • Remember to add Service Reference to your Windows Phone application by referencing the service hosted in IIS:

  • Check the View in Windows Phone 7 :

  • Adding View for Android SmartPhones consuming data from SQL Server / SQL Azure database through OData WCF Data Services:

package com.example.ODataAndroid;

import java.util.ArrayList;

import java.util.List;

import android.app.Activity;

import android.os.Bundle;

import org.odata4j.consumer.ODataConsumer;

import org.odata4j.core.OEntity;

import android.app.ListActivity;

import android.widget.ArrayAdapter;

public class ODataSQLAndroidActivity extends ListActivity {

/** Called when the activity is first created. */

@Override

public void  onCreate(Bundle savedInstanceState) {

super.onCreate(savedInstanceState);

        setListAdapter(new ArrayAdapter<String>(this,  android.R.layout.simple_list_item_1, GetExpenseReports()));

        getListView().setTextFilterEnabled(true);

    }

    ArrayList<String> GetExpenseReports()

    {

    ArrayList<String> listUI =new ArrayList<String>();

    ODataConsumer c = ODataConsumer.create( http://10.12.1.223/ODataSQLWP7/CustomerService.svc/);

    List<OEntity> listExpenses = c.getEntities( “CustomerInfoes”).execute().toList();

for(OEntity expense:listExpenses) {

    listUI.add(expense.getProperty( “ID”).getValue().toString()

    + “:” + expense.getProperty(“FirstName”).getValue().toString()

    + expense.getProperty(“LastName”).getValue().toString()

    + “\n”+ expense.getProperty(“Address”).getValue().toString()

    + “\n” + expense.getProperty(“City”).getValue().toString()

    + “\n”+ expense.getProperty(“State”).getValue().toString()

    );

    }

return  listUI;

    }

    }

  • Check out the SQL Server / SQL Azure database data in Android device:

Consuming OData WCF REST Service from Android Client Device


As in my previous blog , already mentioned to create about OData Service feed in ATOM format which can be consumed in Windows Phone 7, Android, iPhone/iPAD , Silverlight,PHP, Windows Azure Table Storage clients.

  • Lets check , how to consume the OData WCF REST service in Android client. To work with OData Android client lets download Odata4j Android client library from RESTlet: http://www.restlet.org/downloads/ & select the OData4j-Bundle.jar in Java Build Path of your Android project.
  • Next, to consume OData service from android device, we need to host the service in IIS. In my demo, i have hosted it in IIS 7.5.
  • Now, Start develop an android client application which can consume OData feed.
  • Modify AndroidManifest.xml as it can consume feed from internet from native application:

<uses-permissionandroid:name=“android.permission.INTERNET”>

</uses-permission>

   Write code for  Activity.java :

package com.example.android.OData;

import java.util.ArrayList;

import java.util.List;

import org.odata4j.consumer.ODataConsumer;

import org.odata4j.core.OEntity;

import android.app.ListActivity;

import android.os.Bundle;

import android.widget.ArrayAdapter;

public class JsonGrabbingConsumerExampleActivity  extends  ListActivity {

/** Called when the activity is first created. */

@Override

public void  onCreate(Bundle savedInstanceState) {

 super.onCreate(savedInstanceState);

        setListAdapter(new ArrayAdapter<String>(this,   android.R.layout.simple_list_item_1, GetExpenseReports()));

        getListView().setTextFilterEnabled(true);

    }

// read expenses odata feed

    ArrayList<String> GetExpenseReports()

    {

// build a simple array list of strings to test things out

        ArrayList<String> listUI = new ArrayList<String>();

// use odata4j consumer

        ODataConsumer c = ODataConsumer.create(http://10.12.1.223/ODataSample1/Service.svc);

// run a query just for Pending states

        List<OEntity> listExpenses = c.getEntities(“SampleCustomerData”).execute().toList();

for(OEntity expense : listExpenses) {

    listUI.add(expense.getProperty(

“CustomerID”).getValue().toString()

    +

“; “ + expense.getProperty(“CustomerName”).getValue().toString()

    +

“; “ + expense.getProperty(“CustomerNotes”).getValue().toString()

    );

        }

return  listUI;   

    }

}

  • Checkout the output in Android 2.3:
%d bloggers like this: