<?xml version="1.0" encoding="UTF-8"?>
<!--
  Licensed to the Apache Software Foundation (ASF) under one or more
  contributor license agreements.  See the NOTICE file distributed with
  this work for additional information regarding copyright ownership.
  The ASF licenses this file to You under the Apache License, Version 2.0
  (the "License"); you may not use this file except in compliance with
  the License.  You may obtain a copy of the License at

      http://www.apache.org/licenses/LICENSE-2.0

  Unless required by applicable law or agreed to in writing, software
  distributed under the License is distributed on an "AS IS" BASIS,
  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
  See the License for the specific language governing permissions and
  limitations under the License.
-->
<!DOCTYPE document PUBLIC "-//APACHE//DTD Documentation V2.0//EN" "http://forrest.apache.org/dtd/document-v20.dtd">

<document>
  <header>
    <title>Installation from Tarball</title>
  </header>
  <body>

  <section>
    <title>Server Installation from Source</title>

    <p><strong>Prerequisites</strong></p>
    <ul>
        <li>machine to build the installation tar on</li>
        <li>machine on which the server can be installed &#8212; this should have
        access to the Hadoop cluster in question, and be accessible from
        the machines you launch jobs from</li>
        <li>an RDBMS &#8212; we recommend MySQL and provide instructions for it</li>
        <li>Hadoop cluster</li>
        <li>Unix user that the server will run as, and, if you are running your
          cluster in secure mode, an associated Kerberos service principal and keytabs.</li>
        <li>Apache Ant 1.8 or greater. Version 1.7.x is not supported.</li>
    </ul>

    <p>Throughout these instructions when you see a word in <em>italics</em> it
    indicates a place where you should replace the word with a locally 
    appropriate value such as a hostname or password.</p>

    <p><strong>Building a tarball </strong></p>

    <p>If you downloaded HCatalog from Apache or another site as a source release,
    you will need to first build a tarball to install.  You can tell if you have
    a source release by looking at the name of the object you downloaded.  If
    it is named hcatalog-src-0.5.0-incubating.tar.gz (notice the
    <strong>src</strong> in the name) then you have a source release.</p>
    
    <p>If you do not already have Apache Ant installed on your machine, you 
    will need to obtain it.  You can get it from the <a href="http://ant.apache.org/">
    Apache Ant website</a>.  Once you download it, you will need to unpack it
    somewhere on your machine.  The directory where you unpack it will be referred
    to as <em>ant_home</em> in this document.</p>

    <p>To produce a binary tarball from downloaded src tarball, execute the following steps:</p>
    <p><code>tar xzf hcatalog-src-0.5.0-incubating.tar.gz</code></p>
    <p><code>cd hcatalog-src-0.5.0-incubating</code></p>
    <p><em>ant_home</em><code>/bin/ant package </code></p>
    <p>The tarball for installation should now be at
    <code>build/hcatalog-0.5.0-incubating.tar.gz</code></p>

    <p><strong>Database Setup</strong></p>

    <p>If you do not already have Hive installed with MySQL, the following will
    walk you through how to do so.  If you have already set this up, you can skip
    this step.</p>

    <p>Select a machine to install the database on.  This need not be the same
    machine as the Thrift server, which we will set up later.  For large
    clusters we recommend that they not be the same machine.  For the 
    purposes of these instructions we will refer to this machine as
    <em>hivedb.acme.com</em></p>

    <p>Install MySQL server on <em>hivedb.acme.com</em>.  You can obtain
    packages for MySQL from <a href="http://www.mysql.com/downloads/">MySQL's
    download site</a>.  We have developed and tested with versions 5.1.46
    and 5.1.48.  We suggest you use these versions or later.
    Once you have MySQL up and running, use the <code>mysql</code> command line
    tool to add the <code>hive</code> user and <code>hivemetastoredb</code>
    database.  You will need to pick a password for your <code>hive</code>
    user, and replace <em>dbpassword</em> in the following commands with it.</p>

    <p><code>mysql -u root</code></p>
    <p><code>mysql> CREATE USER 'hive'@'</code><em>hivedb.acme.com</em><code>' IDENTIFIED BY '</code><em>dbpassword</em><code>';</code></p>
    <p><code>mysql> CREATE DATABASE hivemetastoredb DEFAULT CHARACTER SET latin1 DEFAULT COLLATE latin1_swedish_ci;</code></p>
    <p><code>mysql> GRANT ALL PRIVILEGES ON hivemetastoredb.* TO 'hive'@'</code><em>hivedb.acme.com</em><code>' WITH GRANT OPTION;</code></p>
    <p><code>mysql> flush privileges;</code></p>
    <p><code>mysql> quit;</code></p>

    <p>Use the database installation script found in the Hive package to create the
    database.  <code>hive_home</code> in the line below refers to the directory
    where you have installed Hive.  If you are using Hive rpms, then this will
    be <code>/usr/lib/hive</code>.</p>

    <p><code>mysql -u hive -D hivemetastoredb -h</code><em>hivedb.acme.com</em><code> -p &lt; </code><em>hive_home</em><code>/scripts/metastore/upgrade/mysql/hive-schema-0.10.0.mysql.sql</code></p>

    <p><strong>Thrift Server Setup</strong></p>

    <p>If you do not already have Hive running a metastore server using Thrift,
    you can use the following instructions to setup and run one.  You may skip
    this step if you already are using a Hive metastore server.</p>

    <p>Select a machine to install your Thrift server on.  For smaller and test
    installations this can be the same machine as the database.  For the
    purposes of these instructions we will refer to this machine as
    <em>hcatsvr.acme.com</em>.</p>

    <p>If you have not already done so, install Hive 0.10.0 (say) on this machine.  You
    can use the
    <a href="http://hive.apache.org/releases.html">binary distributions</a> 
    provided by Hive or rpms available from
    <a href="http://incubator.apache.org/bigtop/">Apache Bigtop</a>.  If you use
    the Apache Hive binary distribution, select a directory, henceforth 
    referred to as <code>hive_home</code>, and untar the distribution there.
    If you use the rpms, <code>hive_home</code> will be
    <code>/usr/lib/hive</code>.</p>

    <p>Install the MySQL Java connector libraries on <em>hcatsvr.acme.com</em>.
    You can obtain these from
    <a href="http://www.mysql.com/downloads/connector/j/5.1.html">MySQL's
    download site</a>.</p>

    <p>Select a user to run the Thrift server as.  This user should not be a
    human user, and must be able to act as a proxy for other users.  We suggest
    the name "hive" for the user.  Throughout the rest of this documentation 
    we will refer to this user as <em>hive</em>.  If necessary, add the user to 
    <em>hcatsvr.acme.com</em>.</p>

    <p>Select a <em>root</em> directory for your installation of HCatalog.  This 
    directory must be owned by the <em>hive</em> user.  We recommend
    <code>/usr/local/hive</code>.  If necessary, create the directory.  You will
    need to be the <em>hive</em> user for the operations described in the remainder
    of this Thrift Server Setup section.</p>

    <p>Copy the HCatalog installation tarball into a temporary directory, and untar
    it.  Then change directories into the new distribution and run the HCatalog
    server installation script.  You will need to know the directory you chose
    as <em>root</em> and the
    directory you installed the MySQL Java connector libraries into (referred
    to in the command below as <em>dbroot</em>).  You will also need your
    <em>hadoop_home</em>, the directory where you have Hadoop installed, and 
    the port number you wish HCatalog to operate on which you will use to set
    <em>portnum</em>.</p>

    <p><code>tar zxf hcatalog-0.5.0-incubating.tar.gz</code></p>
    <p><code>cd hcatalog-0.5.0-incubating</code></p>
    <p><code>share/hcatalog/scripts/hcat_server_install.sh -r </code><em>root</em><code> -d </code><em>dbroot</em><code> -h </code><em>hadoop_home</em><code> -p </code><em>portnum</em></p>

    <p>Now you need to edit your <em>hive_home</em><code>/conf/hive-site.xml</code> file.
    If there is no such file in hive conf directory, copy <em>hcat_home</em><code>/etc/hcatalog/proto-hive-site.xml</code>
    and rename it hive-site.xml in <em>hive_home</em><code>/conf/</code>.
    Open this file in your favorite text editor.  The following table shows the
    values you need to configure.</p>

    <table>
        <tr>
            <th>Parameter</th>
            <th>Value to Set it to</th>
        </tr>
        <tr>
            <td>javax.jdo.option.ConnectionURL</td>
            <td>jdbc:mysql://<em>hostname</em>/hivemetastoredb?createDatabaseIfNotExist=true where <em>hostname</em> is the name of the machine you installed MySQL on.</td>
        </tr>
        <tr>
            <td>javax.jdo.option.ConnectionDriverName</td>
            <td>com.mysql.jdbc.Driver</td>
        </tr>

        <tr>
            <td>javax.jdo.option.ConnectionUserName</td>
            <td>hive</td>
        </tr>
        <tr>
            <td>javax.jdo.option.ConnectionPassword</td>
            <td><em>dbpassword</em> value you used in setting up the MySQL server
            above.</td>
        </tr>
        <tr>
            <td>hive.semantic.analyzer.factory.impl</td>
            <td>org.apache.hive.hcatalog.cli.HCatSemanticAnalyzerFactory</td>
        </tr>
        <tr>
            <td>hive.metastore.warehouse.dir</td>
            <td>The directory can be a URI or an absolute file path. If it is an absolute file path, it will be resolved to a URI by the metastore:
            <p>-- If default hdfs was specified in core-site.xml, path resolves to HDFS location. </p>
            <p>-- Otherwise, path is resolved as local file: URI.</p>
            <p>This setting becomes effective when creating new tables (it takes precedence over default DBS.DB_LOCATION_URI at the time of table creation).</p>
            <p>You only need to set this if you have not yet configured Hive to run on your system.</p>
            </td>
        </tr>
        <tr>
            <td>hive.metastore.uris</td>
            <td>thrift://<em>hostname</em>:<em>portnum</em> where <em>hostname</em> is the name of the machine hosting the Thrift server, and <em>portnum</em> is the port number
            used above in the installation script.</td>
        </tr>
        <tr>
            <td>hive.metastore.execute.setugi</td>
            <td>true</td>
        </tr>
        <tr>
            <td>hive.metastore.sasl.enabled</td>
            <td>Set to true if you are using Kerberos security with your Hadoop
            cluster, false otherwise.</td>
        </tr>
        <tr>
            <td>hive.metastore.kerberos.keytab.file</td>
            <td>The path to the Kerberos keytab file containing the metastore
            Thrift server's service principal.  Only required if you set
            hive.metastore.sasl.enabled above to true.</td>
        </tr>
        <tr>
            <td>hive.metastore.kerberos.principal</td>
            <td>The service principal for the metastore Thrift server.  You can
            reference your host as _HOST and it will be replaced with your
            actual hostname.  Only required if you set
            hive.metastore.sasl.enabled above to true.</td>
        </tr>
    </table>

    <p>You can now proceed to starting the server.</p>
  </section>

  <section>
    <title>Starting the Server</title>
            
    <p>To start your server, HCatalog needs to know where Hive is installed.
    This is communicated by setting the environment variable <code>HIVE_HOME</code>
    to the location you installed Hive.  Start the HCatalog server by switching directories to
    <em>root</em> and invoking
    "<code>HIVE_HOME=</code><em>hive_home</em><code> sbin/hcat_server.sh start</code>".</p>

  </section>

  <section>
    <title>Logging</title>

    <p>Server activity logs are located in
    <em>root</em><code>/var/log/hcat_server</code>.  Logging configuration is located at
    <em>root</em><code>/conf/log4j2.properties</code>.  Server logging uses
    <code>DailyRollingFileAppender</code> by default. It will generate a new
    file per day and does not expire old log files automatically.</p>

  </section>

  <section>
    <title>Stopping the Server</title>

    <p>To stop the HCatalog server, change directories to the <em>root</em>
    directory and invoke 
    "<code>HIVE_HOME=</code><em>hive_home</em><code> sbin/hcat_server.sh stop</code>".</p>

  </section>

  <section>
    <title>Client Installation</title>

    <p>Select a <em>root</em> directory for your installation of HCatalog client.
    We recommend <code>/usr/local/hcat</code>.  If necessary, create the directory.</p>

    <p>Copy the HCatalog installation tarball into a temporary directory, and untar
    it.</p>

    <p><code>tar zxf hcatalog-0.5.0-incubating.tar.gz</code></p>

    <p>Now you need to edit your <em>hive_home</em><code>/conf/hive-site.xml</code> file.
    You can use the same file as on the server <strong>except the value of 
    </strong><code>javax.jdo.option.ConnectionPasswordh</code><strong> should be
    removed</strong>.  This avoids having the password available in plain text on
    all of your clients.</p>

    <p>The HCatalog command line interface (CLI) can now be invoked as
    <code>HIVE_HOME=</code><em>hive_home root</em><code>/bin/hcat</code>.</p>

  </section>

  </body>
</document>
