Skip to main content

Enable Hive LLAP Via Knox

 

Enable Hive LLAP Via Knox


What is Hive llap: Live Long And Process (LLAP)

LLAP provides a hybrid execution model.  It consists of a long-lived daemon which replaces direct interactions with the HDFS DataNode, and a tightly integrated DAG-based framework.

For more details on hive llap follow below url:

https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=62689557

What is knox:

 The Apache Knox™ Gateway is an Application Gateway for interacting with the REST APIs and UIs

of Apache Hadoop deployments.

The Knox Gateway provides a single access point for all REST and HTTP interactions with Apache Hadoop clusters.

For more details on knox please follow below url

https://knox.apache.org/


Purpose:

This page explains the process of enabling Hive LLAP Via Knox.

 Process:

Step 1: Login to ambari and add the below properties in hive configs

Hive.server2.thrift.http.path = cliservice

Hive.server2.thrift.http.port = 10001

Hive.server2.transport.mode = http

Step 2: Login to knox server and perform he below actions

Go to /var/lib/knox/data-2.6.5.1108-1/services and cp hive folder

cp -r hive llap/0.13.0

Go to llap/0.13.0 folder and and edit the xml files as below

[root@techzone-n1 0.13.0]# ls -ltr

total 8

-rw-r----- 1 knox knox 1052 Apr 16 08:36 service.xml

-rw-r----- 1 knox knox  950 Apr 16 08:37 rewrite.xml

[root@techzone-n1 0.13.0]# cat

^C

[root@techzone-n1 0.13.0]# cat service.xml

<!--

   Licensed to the Apache Software Foundation (ASF) under one or more

   contributor license agreements.  See the NOTICE file distributed with

   this work for additional information regarding copyright ownership.

   The ASF licenses this file to You under the Apache License, Version 2.0

   (the "License"); you may not use this file except in compliance with

   the License.  You may obtain a copy of the License at

       http://www.apache.org/licenses/LICENSE-2.0

 

   Unless required by applicable law or agreed to in writing, software

   distributed under the License is distributed on an "AS IS" BASIS,

   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.

   See the License for the specific language governing permissions and

   limitations under the License.

-->

<service role="LLAP" name="llap" version="0.13.0">

    <routes>

        <route path="/llap"/>

    </routes>

    <dispatch classname="org.apache.hadoop.gateway.hive.HiveDispatch" ha-classname="org.apache.hadoop.gateway.hive.HiveHaDispatch"/>

</service>

[root@techzone-n1 0.13.0]# cat rewrite.xml

<!--

   Licensed to the Apache Software Foundation (ASF) under one or more

   contributor license agreements.  See the NOTICE file distributed with

   this work for additional information regarding copyright ownership.

   The ASF licenses this file to You under the Apache License, Version 2.0

   (the "License"); you may not use this file except in compliance with

   the License.  You may obtain a copy of the License at

 

       http://www.apache.org/licenses/LICENSE-2.0

 

   Unless required by applicable law or agreed to in writing, software

   distributed under the License is distributed on an "AS IS" BASIS,

   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.

 

See the License for the specific language governing permissions and

   limitations under the License.

-->

<rules>

    <rule dir="IN" name="LLAP/llap/inbound" pattern="*://*:*/**/llap">

        <rewrite template="{$serviceUrl[LLAP]}"/>

    </rule>

</rules>

 

Step 3: Restart the hive,knox services in Ambari.

Step 4: Go to knox server and login to the beeline shell and give the below connection string.

You will be able to connect with the username and password.

!connect jdbc:hive2://nodefqdn-n1.connected.xyz.com:8443/;

ssl=true;transportMode=http;httpPath=gateway/default/llap

above steps are tested and performed on hdp 2.6 version, if you face any error while configuring hive llap via knox,

I request you to post your error in comment section and I will have a look.

 

Thank you !!!

 

Thank you !! Example HTML page Pleaes provide your valuable feedback.
Labels:

Comments

Post a Comment

Popular posts from this blog

Jenkins

Pre-requisites 1. Install a Webserver https://gitlab.com/Azam-devops/webserver/-/blob/main/README.md Code for index.html https://gitlab.com/Azam-devops/webserver 2. Maven Code https://gitlab.com/Azam-devops/imperial-maven-project 1. Install & configure Jenkins Automation Server on Linux Vm. 2. Go through at some of the important options in Jenkins. 3. Manage Jenkins. 4. Plugins 5. Global Tools Configuration. 6. Credentials 7. Users 8. Slave Nodes 9. Configuring CI pipeline using Gitlab. 10. Configuring standalone CICD pipeline using. 11. Automating the CICD pipeline. 12. Jenkins log 13. Introduction to Jenkins file. 14. Basic groovy syntax & file formation. 15. Launching a Pipeline using Jenkins file. 3. DevOps Architecture Description of above DevOps plan. Create Maven based source code in Gitlab. Create a Jenkins job which will execute below stages. Checkout code from Gitlab Build/compile the source code using Maven as a build tool. scan the code virtually. Test...
Post a Comment
Read more

Docker In Details

  Course Contents:- 1. Overview of Docker 2. Difference between Virtualization & Containerization 3. Installation & Configuration of Docker Runtime on Linux & Windows 4. Practice on Docker commands 5. launch a Webserver in a container 6. Launch public & official images of application like Jenkins, Nginx, DB etc.. 7. Launch a base OS Container 8. How to save changes inside the container & create a fresh image(commit) 9. How to ship image & container from one hardware to another. 10. How to remove stop/rm multiple container/images 11. Docker Registry 12. Docker Networking       Check current docker network                  Docker Network Bridge                     Docker Network Weaving                  Launch our own Docker Cluster with our defined Network             ...
Post a Comment
Read more

Ansible

  Ansible is an open-source software provisioning, configuration management, and application-deployment tool. It runs on many Unix-like systems, and can configure both Unix-like systems as well as Microsoft Windows. It includes its own declarative language to describe system configuration. Ansible was written by Michael DeHaan and acquired by Red Hat in 2015. Ansible is agentless, temporarily connecting remotely via SSH or Windows Remote Management (allowing remote PowerShell execution) to do its tasks. Platform support Control machines have to be a Linux/Unix host (for example SUSE Linux Enterprise, Red Hat Enterprise Linux, Debian, CentOS, macOS, BSD, Ubuntu, and Python 2.7 or 3.5 is required. Managed nodes, if they are Unix-like, must have Python 2.4 or later. For managed nodes with Python 2.5 or earlier, the python-simplejson package is also required. Since version 1.7, Ansible can also manage Windows nodes. In this case, native PowerShell remoting supported by the WS-Managemen...
Post a Comment
Read more

Basic Linux Commands

  Linux Command Cheat Sheet Hello All, Below are the most common commands used in a day to day life of  linux user. if you are new to linux i will recommend you to go through all of the commands.  this commands will help you to troubleshoot linux issues.   Command Description ls Lists all files and directories from present working directory ls-R Lists files in sub-directories ls-a to list down hidden files. ls-al Lists files and directories with complete details like permissions, size, owner cd or cd ~ To go back to home directory cd .. Move one level up cd To change to a particular directory cd / Move to the root directory cat > filename Creates a new file cat filename Displays the content of a file cat file...
Post a Comment
Read more

Kubernetes-Update

                                                    https://kubernetes.io/ Kubernetes (K8s)  is an open-source system for automating deployment, scaling, and management of containerized applications. It groups containers that make up an application into logical units for easy management and discovery. Kubernetes builds upon  15 years of experience of running production workloads at Google , combined with best-of-breed ideas and practices from the community. Latest Verion:-  1.19 Kubernetes Objects Kubernetes defines a set of building blocks ("primitives"), which collectively provide mechanisms that deploy, maintain, and scale applications based on CPU, memory or custom metrics. Kubernetes is loosely coupled and extensible to meet different workloads. This extensibility is provided in large part by the Kubernetes API, which is used by int...
Post a Comment
Read more