Skip to main content

Enable Hive LLAP Via Knox

 

Enable Hive LLAP Via Knox


What is Hive llap: Live Long And Process (LLAP)

LLAP provides a hybrid execution model.  It consists of a long-lived daemon which replaces direct interactions with the HDFS DataNode, and a tightly integrated DAG-based framework.

For more details on hive llap follow below url:

https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=62689557

What is knox:

 The Apache Knox™ Gateway is an Application Gateway for interacting with the REST APIs and UIs

of Apache Hadoop deployments.

The Knox Gateway provides a single access point for all REST and HTTP interactions with Apache Hadoop clusters.

For more details on knox please follow below url

https://knox.apache.org/


Purpose:

This page explains the process of enabling Hive LLAP Via Knox.

 Process:

Step 1: Login to ambari and add the below properties in hive configs

Hive.server2.thrift.http.path = cliservice

Hive.server2.thrift.http.port = 10001

Hive.server2.transport.mode = http

Step 2: Login to knox server and perform he below actions

Go to /var/lib/knox/data-2.6.5.1108-1/services and cp hive folder

cp -r hive llap/0.13.0

Go to llap/0.13.0 folder and and edit the xml files as below

[root@techzone-n1 0.13.0]# ls -ltr

total 8

-rw-r----- 1 knox knox 1052 Apr 16 08:36 service.xml

-rw-r----- 1 knox knox  950 Apr 16 08:37 rewrite.xml

[root@techzone-n1 0.13.0]# cat

^C

[root@techzone-n1 0.13.0]# cat service.xml

<!--

   Licensed to the Apache Software Foundation (ASF) under one or more

   contributor license agreements.  See the NOTICE file distributed with

   this work for additional information regarding copyright ownership.

   The ASF licenses this file to You under the Apache License, Version 2.0

   (the "License"); you may not use this file except in compliance with

   the License.  You may obtain a copy of the License at

       http://www.apache.org/licenses/LICENSE-2.0

 

   Unless required by applicable law or agreed to in writing, software

   distributed under the License is distributed on an "AS IS" BASIS,

   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.

   See the License for the specific language governing permissions and

   limitations under the License.

-->

<service role="LLAP" name="llap" version="0.13.0">

    <routes>

        <route path="/llap"/>

    </routes>

    <dispatch classname="org.apache.hadoop.gateway.hive.HiveDispatch" ha-classname="org.apache.hadoop.gateway.hive.HiveHaDispatch"/>

</service>

[root@techzone-n1 0.13.0]# cat rewrite.xml

<!--

   Licensed to the Apache Software Foundation (ASF) under one or more

   contributor license agreements.  See the NOTICE file distributed with

   this work for additional information regarding copyright ownership.

   The ASF licenses this file to You under the Apache License, Version 2.0

   (the "License"); you may not use this file except in compliance with

   the License.  You may obtain a copy of the License at

 

       http://www.apache.org/licenses/LICENSE-2.0

 

   Unless required by applicable law or agreed to in writing, software

   distributed under the License is distributed on an "AS IS" BASIS,

   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.

 

See the License for the specific language governing permissions and

   limitations under the License.

-->

<rules>

    <rule dir="IN" name="LLAP/llap/inbound" pattern="*://*:*/**/llap">

        <rewrite template="{$serviceUrl[LLAP]}"/>

    </rule>

</rules>

 

Step 3: Restart the hive,knox services in Ambari.

Step 4: Go to knox server and login to the beeline shell and give the below connection string.

You will be able to connect with the username and password.

!connect jdbc:hive2://nodefqdn-n1.connected.xyz.com:8443/;

ssl=true;transportMode=http;httpPath=gateway/default/llap

above steps are tested and performed on hdp 2.6 version, if you face any error while configuring hive llap via knox,

I request you to post your error in comment section and I will have a look.

 

Thank you !!!

 

Thank you !! Example HTML page Pleaes provide your valuable feedback.
Labels:

Comments

Post a Comment

Popular posts from this blog

Connect SparkThriftServer with Tableau/PowerBI

  Connect SparkThriftServer with Tableau/PowerBI REFERENCE : https://docs.microsoft.com/en-us/azure/hdinsight/hdinsight-apache-spark-use-bi-tools Use Power BI for Spark data visualization Note This section is applicable only for Spark 1.6 on HDInsight 3.4 and Spark 2.0 on HDInsight 3.5.   Once you have saved the data as a table, you can use Power BI to connect to the data and visualize it to create reports, dashboards, etc.   1.       Make sure you have access to Power BI. You can get a free preview subscription of Power BI from http://www.powerbi.com/ . 2.       Sign in to Power BI . 3.       From the bottom of the left pane, click Get Data . 4.       On the Get Data page, under Import or Connect to Data , for Databases , click Get . 5.       On the next screen, click Spark on Azure HDInsight and then click Connect . When prompted, enter th...
Post a Comment
Read more

Azure Active Directory

  What is Azure Active Directory? Azure Active Directory is Microsoft’s multi-tenant, cloud-based directory and identity management service. For an organization, Azure AD helps employees sign up to multiple services and access them anywhere over the cloud with a single set of login credentials. Azure Active Directory Concept: It is important to understand these Azure AD concepts. Identity . An object that can get authenticated. An identity can be a user with a username and password. Account . An identity that has data associated with it. You can't have an account without an identity. Azure subscription . Used to pay for Azure cloud services. You can have many subscriptions and they're linked to a credit card. Azure tenant/directory . A dedicated and trusted instance of Azure AD, a Tenant is automatically created when your organization signs up for a Microsoft cloud service subscription. More instances of Azure AD can be created. Azure AD is the underlying product ...
1 comment
Read more

Roadmap to DevOps

    DevOps is nothing but the combination of process and philosophies which contains four basic component culture, collaboration, tools, and practices. In return, this gives a good automated system and infrastructure which helps an organisation to deliver a quality and reliable build. The beauty of this culture is it enables a quality for organizations to better serve their customers and compete more effectively in the market and also add some promised benefits which include confidence and trust, faster software releases, ability to solve critical issues quickly, and better manage unplanned work.   1. What are the tasks of a DevOps Engineer? Design, build, test and deploy scalable, distributed systems from development through production Manage the code repository(such as Git, SVN, BitBucket, etc.) including code merging and integrating, branching and maintenance and remote repository management Manage, configure and maintain infra...
Post a Comment
Read more

Azure Storage

Azure Storage is Microsoft's cloud storage solution for modern data storage scenarios.    Why Azure Storage: Durable and Highly Available: ·        Data is safe during hardware failure. ·        Replicate data across data centres. ·        In local catastrophe or natural disaster data replicated to other data centres remains highly available. Secure: ·        Data encrypted by Azure service Storage Service encryption (SSE) for data at rest. ·        Completed control over who access your data. Scalable: ·        Designed to massive scale. Managed: ·        Azure handles maintenance, updates and critical issues. Accessible: ·        Accessible from anywhere in the world. ·        You can access using http, https,...
Post a Comment
Read more

Docker In Details

  Course Contents:- 1. Overview of Docker 2. Difference between Virtualization & Containerization 3. Installation & Configuration of Docker Runtime on Linux & Windows 4. Practice on Docker commands 5. launch a Webserver in a container 6. Launch public & official images of application like Jenkins, Nginx, DB etc.. 7. Launch a base OS Container 8. How to save changes inside the container & create a fresh image(commit) 9. How to ship image & container from one hardware to another. 10. How to remove stop/rm multiple container/images 11. Docker Registry 12. Docker Networking       Check current docker network                  Docker Network Bridge                     Docker Network Weaving                  Launch our own Docker Cluster with our defined Network             ...
Post a Comment
Read more