Skip to main content

Enable Hive LLAP Via Knox

 

Enable Hive LLAP Via Knox


What is Hive llap: Live Long And Process (LLAP)

LLAP provides a hybrid execution model.  It consists of a long-lived daemon which replaces direct interactions with the HDFS DataNode, and a tightly integrated DAG-based framework.

For more details on hive llap follow below url:

https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=62689557

What is knox:

 The Apache Knox™ Gateway is an Application Gateway for interacting with the REST APIs and UIs

of Apache Hadoop deployments.

The Knox Gateway provides a single access point for all REST and HTTP interactions with Apache Hadoop clusters.

For more details on knox please follow below url

https://knox.apache.org/


Purpose:

This page explains the process of enabling Hive LLAP Via Knox.

 Process:

Step 1: Login to ambari and add the below properties in hive configs

Hive.server2.thrift.http.path = cliservice

Hive.server2.thrift.http.port = 10001

Hive.server2.transport.mode = http

Step 2: Login to knox server and perform he below actions

Go to /var/lib/knox/data-2.6.5.1108-1/services and cp hive folder

cp -r hive llap/0.13.0

Go to llap/0.13.0 folder and and edit the xml files as below

[root@techzone-n1 0.13.0]# ls -ltr

total 8

-rw-r----- 1 knox knox 1052 Apr 16 08:36 service.xml

-rw-r----- 1 knox knox  950 Apr 16 08:37 rewrite.xml

[root@techzone-n1 0.13.0]# cat

^C

[root@techzone-n1 0.13.0]# cat service.xml

<!--

   Licensed to the Apache Software Foundation (ASF) under one or more

   contributor license agreements.  See the NOTICE file distributed with

   this work for additional information regarding copyright ownership.

   The ASF licenses this file to You under the Apache License, Version 2.0

   (the "License"); you may not use this file except in compliance with

   the License.  You may obtain a copy of the License at

       http://www.apache.org/licenses/LICENSE-2.0

 

   Unless required by applicable law or agreed to in writing, software

   distributed under the License is distributed on an "AS IS" BASIS,

   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.

   See the License for the specific language governing permissions and

   limitations under the License.

-->

<service role="LLAP" name="llap" version="0.13.0">

    <routes>

        <route path="/llap"/>

    </routes>

    <dispatch classname="org.apache.hadoop.gateway.hive.HiveDispatch" ha-classname="org.apache.hadoop.gateway.hive.HiveHaDispatch"/>

</service>

[root@techzone-n1 0.13.0]# cat rewrite.xml

<!--

   Licensed to the Apache Software Foundation (ASF) under one or more

   contributor license agreements.  See the NOTICE file distributed with

   this work for additional information regarding copyright ownership.

   The ASF licenses this file to You under the Apache License, Version 2.0

   (the "License"); you may not use this file except in compliance with

   the License.  You may obtain a copy of the License at

 

       http://www.apache.org/licenses/LICENSE-2.0

 

   Unless required by applicable law or agreed to in writing, software

   distributed under the License is distributed on an "AS IS" BASIS,

   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.

 

See the License for the specific language governing permissions and

   limitations under the License.

-->

<rules>

    <rule dir="IN" name="LLAP/llap/inbound" pattern="*://*:*/**/llap">

        <rewrite template="{$serviceUrl[LLAP]}"/>

    </rule>

</rules>

 

Step 3: Restart the hive,knox services in Ambari.

Step 4: Go to knox server and login to the beeline shell and give the below connection string.

You will be able to connect with the username and password.

!connect jdbc:hive2://nodefqdn-n1.connected.xyz.com:8443/;

ssl=true;transportMode=http;httpPath=gateway/default/llap

above steps are tested and performed on hdp 2.6 version, if you face any error while configuring hive llap via knox,

I request you to post your error in comment section and I will have a look.

 

Thank you !!!

 

Thank you !! Example HTML page Pleaes provide your valuable feedback.
Labels:

Comments

Post a Comment

Popular posts from this blog

Script To Monitor disk Usage

Hello All, Many times we have a requirement to automate few of the tasks, and one of the important task is to delete old log files from log destination. We usually get the conditions where we have to delete the log from particular directory when it reaches to some thresh hold value. Now a day the requirement of automation is highly increased where we are seeing most of the day to day tasks are getting automated using different scripting languages. Its always a good idea to automate your daily tasks which will save your working hours and will increase your productivity. Most of the time log destination gets full and due to which we face other issues like application down or particular service stop working. And manual deleting of log is hectic process so below script will help you to delete your log data when it meets your condition, and also it will trigger mail to recipient after deletion of logs. You can schedule this script in your crontab according to your feasible ti...
7 comments
Read more

Azure Active Directory

  What is Azure Active Directory? Azure Active Directory is Microsoft’s multi-tenant, cloud-based directory and identity management service. For an organization, Azure AD helps employees sign up to multiple services and access them anywhere over the cloud with a single set of login credentials. Azure Active Directory Concept: It is important to understand these Azure AD concepts. Identity . An object that can get authenticated. An identity can be a user with a username and password. Account . An identity that has data associated with it. You can't have an account without an identity. Azure subscription . Used to pay for Azure cloud services. You can have many subscriptions and they're linked to a credit card. Azure tenant/directory . A dedicated and trusted instance of Azure AD, a Tenant is automatically created when your organization signs up for a Microsoft cloud service subscription. More instances of Azure AD can be created. Azure AD is the underlying product ...
1 comment
Read more

Azure Storage

Azure Storage is Microsoft's cloud storage solution for modern data storage scenarios.    Why Azure Storage: Durable and Highly Available: ·        Data is safe during hardware failure. ·        Replicate data across data centres. ·        In local catastrophe or natural disaster data replicated to other data centres remains highly available. Secure: ·        Data encrypted by Azure service Storage Service encryption (SSE) for data at rest. ·        Completed control over who access your data. Scalable: ·        Designed to massive scale. Managed: ·        Azure handles maintenance, updates and critical issues. Accessible: ·        Accessible from anywhere in the world. ·        You can access using http, https,...
Post a Comment
Read more

Connect SparkThriftServer with Tableau/PowerBI

  Connect SparkThriftServer with Tableau/PowerBI REFERENCE : https://docs.microsoft.com/en-us/azure/hdinsight/hdinsight-apache-spark-use-bi-tools Use Power BI for Spark data visualization Note This section is applicable only for Spark 1.6 on HDInsight 3.4 and Spark 2.0 on HDInsight 3.5.   Once you have saved the data as a table, you can use Power BI to connect to the data and visualize it to create reports, dashboards, etc.   1.       Make sure you have access to Power BI. You can get a free preview subscription of Power BI from http://www.powerbi.com/ . 2.       Sign in to Power BI . 3.       From the bottom of the left pane, click Get Data . 4.       On the Get Data page, under Import or Connect to Data , for Databases , click Get . 5.       On the next screen, click Spark on Azure HDInsight and then click Connect . When prompted, enter th...
Post a Comment
Read more

Continuous Integration with Jenkins and GitLab

  Enter the credential manager Go into system credentials Add a new credential (private key of slave node where you want to run the job) We need a SSH private key credential. I’m not sure the username matters, but I set it to “git” as that is what the login for gitlab uses. I then pointed it to my *private* key on the machine and gave it a description. Now the key shows up in the system. Once I had setup the ssh private key, which allows Jenkins to connect to the git host over ssh (i.e. git clone git@server:user/repo.git), I needed to setup API Access for Jenkins to get metadata from GitLab. In GitLab go to your user profile. Select “Access Tokens”. Now make a new token, I gave it a descriptive name and made it expire at the end of the decade (I haven’t investigated the pros/cons of expiration date length yet). You are then presented with a screen showing the token. Copy it now, as it won’t be accessible again! You can always create a new one if you mess it up t...
Post a Comment
Read more