Commit 07e7e669 authored by Edward J. Stembler's avatar Edward J. Stembler

Initial commit

parents
/target
/classes
/checkouts
pom.xml
pom.xml.asc
*.jar
!resources/*.jar
/.lein-*
/.nrepl-port
The MIT License (MIT)
Copyright © 2018 Edward J. Stembler
Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the “Software”), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED “AS IS”, WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
# emr-hive-jdbc-example
Example of how to connect to AWS EMR via Hive JDBC.
See my blog post for more details: [Using Clojure to query AWS EMR via Hive JDBC](https://ejstembler.com/2018/09/using-clojure-to-query-aws-emr-via-hive-jdbc).
## Installation
Download from https://gitlab.com/ejstembler/emr-hive-jdbc-example.
## Usage
To run via lein:
$ lein run
To run the packaged JAR:
$ java -jar emr-hive-jdbc-example-0.1.0-standalone.jar
## License
The MIT License (MIT)
Copyright © 2018 Edward J. Stembler
Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the “Software”), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED “AS IS”, WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
(defproject emr-hive-jdbc-example "0.1.0-SNAPSHOT"
:description "Example of how to connect to AWS EMR via Hive JDBC"
:url "https://gitlab.com/ejstembler/emr-hive-jdbc-example"
:license {:name "MIT"
:url "https://mit-license.org"}
:dependencies [[org.clojure/clojure "1.8.0"]
[org.clojure/tools.logging "0.4.1"]
[org.clojure/java.jdbc "0.7.8"]]
:resource-paths ["resources/HiveJDBC41.jar"
"resources/commons-codec-1.3.jar"
"resources/commons-logging-1.1.1.jar"
"resources/hive_metastore.jar"
"resources/hive_service.jar"
"resources/httpclient-4.1.3.jar"
"resources/httpcore-4.1.3.jar"
"resources/libfb303-0.9.0.jar"
"resources/libthrift-0.9.0.jar"
"resources/log4j-1.2.14.jar"
"resources/ql.jar"
"resources/slf4j-api-1.5.11.jar"
"resources/slf4j-log4j12-1.5.11.jar"
"resources/TCLIServiceClient.jar"
"resources/zookeeper-3.4.6.jar"]
:main ^:skip-aot emr-hive-jdbc-example.core
:target-path "target/%s"
:profiles {:uberjar {:aot :all}})
(ns emr-hive-jdbc-example.core
(:require [clojure.tools.logging :as log])
(:require [clojure.java.jdbc :as jdbc])
(:gen-class))
;; Define the user/password/subname from ENV variables
(def user (System/getenv "EMR_HIVE_JDBC_USER"))
(def password (System/getenv "EMR_HIVE_JDBC_PASSWORD"))
(def subname (System/getenv "EMR_HIVE_JDBC_SUBNAME"))
;; Preconditions
(when (nil? user)
(throw (RuntimeException. "user is nil")))
(when (nil? password)
(throw (RuntimeException. "password is nil")))
(when (nil? subname)
(throw (RuntimeException. "subname is nil")))
;; Define the connection specification
(def mdl-spec {:classname "com.amazon.hive.jdbc41.HS2Driver"
:subprotocol "hive2"
:subname subname
:user user
:password password})
;; Define the SQL statement to execute
(def sql ["SELECT
COUNT(*) AS row_count
FROM
my_table"]) ; TODO: Change to your desired SQL statement
;; The main entry-point
;; 1. Logs the SQL to execute
;; 2. Executes the SQL
;; 3. Returns the row_count value of the first row
(defn -main
[& args]
(log/info "Executing SQL:\n" (first sql))
(println (jdbc/query mdl-spec sql {:row-fn :row_count :result-set-fn first})))
log4j.rootLogger=INFO, console
log4j.logger.example=DEBUG
log4j.appender.console=org.apache.log4j.ConsoleAppender
log4j.appender.console.layout=org.apache.log4j.PatternLayout
log4j.appender.console.layout.ConversionPattern=%-5p %c: %m%n
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment