Installing HDP for Windows

Last week I was busy for some time installing Hortonworks Data Platform for Windows and below were my notes as I proceeded. It might be useful for you as well.

Following link was quite handy for me:
http://docs.hortonworks.com/HDPDocuments/HDP1/HDP-Win-1.3.0/bk_installing_hdp_for_windows/content/win-chap2-singlenode.html

Operating system used: Windows Server 2008 R2 (64 bit)

Download the below listed software and save them to a folder. For example C:\HDP-Installation

Installing Pre-requisites

  1. Install the Microsoft Visual C++ 2010 Redistributable Package (64 bit) and Microsoft.NET framework 4.0 by simply clicking the installers respectively.
  2. Install Python 2.7 and follow the below steps to Update System variable Path
    • Open Control Panel -> System -> Advanced system settings
    • Go to Advanced tab and click Environment Variables button
    • Under System variables find path and Edit
    • Update the value of path variable by appending the installation path of Python. For example c:\python27\ ( use semi-colon as delamination)

installation path of python

    • Verify the setting by using command shell, type python

verify python setting

  1. Install Java JDK to a directory that has no whitespace characters in its path. For example C:\Java\jdk.1.6.0_45\

After installation Create a system variable named JAVA_HOME

    • Open Control Panel -> System -> Advanced system settings
    • Go to Advanced tab and click Environment Variables button
    • Under System variable click New
    • Set Variable name as JAVA_HOME and Variable value as installation path of JDK. For example C:\Java\jdk.1.6.0_45\

    • Click OK

Configuring Firewall

HDP uses multiple ports for communication with clients and between service components.

You must enable the required ports. You can find the list of ports in this link

Use the following command to open these ports:

netsh advfirewall firewall add rule name=AllowRPCCommunication dir=in action=allow protocol=TCP localport=$PORT_NUMBER

For example,
To open port 80

netsh advfirewall firewall add rule name=AllowRPCCommunication dir=in action=allow protocol=TCP localport=135

To open ports 49152-65535

netsh advfirewall firewall add rule name=AllowRPCCommunication dir=in action=allow protocol=TCP localport=49152-65535

Specify the configuration for HDP on a single node machine

Create a clusterproperties.txt file. (The clusterproperties.txt is a text file and contains parameter definitions like the hostnames of the nodes in your cluster, the roles for each of them, etc.)

Copy the text provided below and modify it according to the hostname of the machine. Save the text file to C:\HDP-Installation folder

#Log directory
HDP_LOG_DIR=c:\hadoop\logs#Data directory
HDP_DATA_DIR=c:\hdp\data#Hosts (Roles for the host machines in your cluster)
NAMENODE_HOST=${Hostname for your single node cluster}
SECONDARY_NAMENODE_HOST=${Hostname for your single node cluster}
JOBTRACKER_HOST=${Hostname for your single node cluster}
HIVE_SERVER_HOST=${Hostname for your single node cluster}
OOZIE_SERVER_HOST=${Hostname for your single node cluster}
WEBHCAT_HOST=${Hostname for your single node cluster}
FLUME_HOSTS=${Hostname for your single node cluster}
HBASE_MASTER=${Hostname for your single node cluster}
HBASE_REGIONSERVERS=${Hostname for your single node cluster}
ZOOKEEPER_HOSTS=${Hostname for your single node cluster}
SLAVE_HOSTS=${Hostname for your single node cluster}#Database host
DB_FLAVOR=derby
DB_HOSTNAME=${Hostname for your single node cluster}

#Hive properties
HIVE_DB_NAME=hive
HIVE_DB_USERNAME=hive
HIVE_DB_PASSWORD=hive

#Oozie properties
OOZIE_DB_NAME=oozie
OOZIE_DB_USERNAME=oozie
OOZIE_DB_PASSWORD=oozie

Install HDP

Unzip the HDP-1.3.0.0-GA.zip and keep the installation file hdp-1.3.0.0.winpkg.msi in HDP-Installation folder.

Open a command prompt with Administrator privileges and execute the MSI installer command:

msiexec /i “c:\HDP-Installation\hdp-1.3.0.0.winpkg.msi” /lv “C:\HDP-Installation\hdp.log” HDP_LAYOUT=”C:\HDP-Installation\clusterproperties.txt” HDP_DIR=”C:\hdp\hadoop” DESTROY_DATA=”no”

Create a Hadoop Distributed File System (HDFS)

Open Command Shell and change directory to the hadoop bin folder. For example C:\hdp\hadoop\hadoop-1.2.0.1.3.0.0-0380\bin

And type: hadoop namenode -format

Hadoop namenode

Starting Services

Open Command Shell and change directory to the hadoop folder. For example C:\hdp\hadoop

And type: start_local_hdp_services

Start Hadoop Service

Runnig SmokeTest

Open Command Shell and change directory to the hadoop folder. For example C:\hdp\hadoop

And type: run-smoketests

Running Smoketest

Now my HDP is ready to go. Let me know your thoughts…

Leave a Reply

Close Menu