Last week I was busy for some time installing Hortonworks Data Platform for Windows and below were my notes as I proceeded. It might be useful for you as well.
Following link was quite handy for me:
http://docs.hortonworks.com/HDPDocuments/HDP1/HDP-Win-1.3.0/bk_installing_hdp_for_windows/content/win-chap2-singlenode.html
Operating system used: Windows Server 2008 R2 (64 bit)
Download the below listed software and save them to a folder. For example C:\HDP-Installation
- Microsoft Visual C++ 2010 Redistributable Package (64 bit) Download
- Microsoft.NET framework 4.0 Download
- Python 2.7 Download
- Java JDK Download
- HDP for Windows Download
Installing Pre-requisites
- Install the Microsoft Visual C++ 2010 Redistributable Package (64 bit) and Microsoft.NET framework 4.0 by simply clicking the installers respectively.
- Install Python 2.7 and follow the below steps to Update System variable Path
- Open Control Panel -> System -> Advanced system settings
- Go to Advanced tab and click Environment Variables button
- Under System variables find path and Edit
- Update the value of path variable by appending the installation path of Python. For example c:\python27\ ( use semi-colon as delamination)
-
- Verify the setting by using command shell, type python
- Install Java JDK to a directory that has no whitespace characters in its path. For example C:\Java\jdk.1.6.0_45\
After installation Create a system variable named JAVA_HOME
-
- Open Control Panel -> System -> Advanced system settings
- Go to Advanced tab and click Environment Variables button
- Under System variable click New
- Set Variable name as JAVA_HOME and Variable value as installation path of JDK. For example C:\Java\jdk.1.6.0_45\
-
- Click OK
Configuring Firewall
HDP uses multiple ports for communication with clients and between service components.
You must enable the required ports. You can find the list of ports in this link
Use the following command to open these ports:
For example,
To open port 80
To open ports 49152-65535
Specify the configuration for HDP on a single node machine
Create a clusterproperties.txt file. (The clusterproperties.txt is a text file and contains parameter definitions like the hostnames of the nodes in your cluster, the roles for each of them, etc.)
Copy the text provided below and modify it according to the hostname of the machine. Save the text file to C:\HDP-Installation folder
HDP_LOG_DIR=c:\hadoop\logs#Data directory
HDP_DATA_DIR=c:\hdp\data#Hosts (Roles for the host machines in your cluster)
NAMENODE_HOST=${Hostname for your single node cluster}
SECONDARY_NAMENODE_HOST=${Hostname for your single node cluster}
JOBTRACKER_HOST=${Hostname for your single node cluster}
HIVE_SERVER_HOST=${Hostname for your single node cluster}
OOZIE_SERVER_HOST=${Hostname for your single node cluster}
WEBHCAT_HOST=${Hostname for your single node cluster}
FLUME_HOSTS=${Hostname for your single node cluster}
HBASE_MASTER=${Hostname for your single node cluster}
HBASE_REGIONSERVERS=${Hostname for your single node cluster}
ZOOKEEPER_HOSTS=${Hostname for your single node cluster}
SLAVE_HOSTS=${Hostname for your single node cluster}#Database host
DB_FLAVOR=derby
DB_HOSTNAME=${Hostname for your single node cluster}
#Hive properties
HIVE_DB_NAME=hive
HIVE_DB_USERNAME=hive
HIVE_DB_PASSWORD=hive
#Oozie properties
OOZIE_DB_NAME=oozie
OOZIE_DB_USERNAME=oozie
OOZIE_DB_PASSWORD=oozie
Install HDP
Unzip the HDP-1.3.0.0-GA.zip and keep the installation file hdp-1.3.0.0.winpkg.msi in HDP-Installation folder.
Open a command prompt with Administrator privileges and execute the MSI installer command:
Create a Hadoop Distributed File System (HDFS)
Open Command Shell and change directory to the hadoop bin folder. For example C:\hdp\hadoop\hadoop-1.2.0.1.3.0.0-0380\bin
And type: hadoop namenode -format
Starting Services
Open Command Shell and change directory to the hadoop folder. For example C:\hdp\hadoop
And type: start_local_hdp_services
Runnig SmokeTest
Open Command Shell and change directory to the hadoop folder. For example C:\hdp\hadoop
And type: run-smoketests
Now my HDP is ready to go. Let me know your thoughts…