Commit 30275575 authored by Toralf Wittner's avatar Toralf Wittner

Merge branch 'develop'

parents 2bebf3d7 daf87ea9
Pipeline #3543491 passed with stage
in 19 minutes and 12 seconds
before_script:
- apt-get update
- apt-get install -y libstdc++-4.9-dev g++
- cabal update
test:7.10:
image: haskell:7.10
script:
- cabal install --enable-test --enable-bench --only-dep -j
- cabal build
- cabal test
- cabal bench
test:7.8:
image: haskell:7.8
script:
- cabal install --enable-test --enable-bench --only-dep -j
- cabal build
- cabal test
- cabal bench
language: haskell
ghc:
- 7.8
- 7.6
3.1.0
-----
- Bugfix: Previous versions used an encoding for `CqlVarInt`
and `CqlDecimal` which is *incompatible* with the actual CQL binary
protocol specification. If you want to continue using the previous
encoding use `-f incompatible-varint` when building this release.
- The `Keyspace` parameter was removed from `UdtColumn` (for details
see https://gitlab.com/twittner/cql/merge_requests/2)
3.0.7
-----
- Bugfix release. Fixes UDT and tuple serialisation.
......
Thank you for your interest in this project!
Unless you explicitly state otherwise, any contribution intentionally
submitted for inclusion in the software by you shall be under the terms
and conditions of the Free Public License 1.0.0, without any additional
terms or conditions.
If you agree to the above and you can certify that your contribution
was created in whole or in part by you and that you have the right to
submit it under the Free Public License 1.0.0, then add a line saying
Signed-off-by: Your real name <your e-mail address>
to your contribution.
For details in regards to the Free Public License read the file LICENSE
or visit https://opensource.org/licenses/FPL-1.0.0
Free Public License 1.0.0
Permission to use, copy, modify, and/or distribute this software for
any purpose with or without fee is hereby granted.
THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL
WARRANTIES WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES
OF MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE
FOR ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY
DAMAGES WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN
AN ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT
OF OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE.
=============================================================================
Cereal License
Copyright (c) Lennart Kolmodin, Galois, Inc.
All rights reserved.
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions
are met:
1. Redistributions of source code must retain the above copyright
notice, this list of conditions and the following disclaimer.
2. Redistributions in binary form must reproduce the above copyright
notice, this list of conditions and the following disclaimer in the
documentation and/or other materials provided with the distribution.
3. Neither the name of the author nor the names of his contributors
may be used to endorse or promote products derived from this software
without specific prior written permission.
THIS SOFTWARE IS PROVIDED BY THE CONTRIBUTORS ``AS IS'' AND ANY EXPRESS
OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE
DISCLAIMED. IN NO EVENT SHALL THE AUTHORS OR CONTRIBUTORS BE LIABLE FOR
ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS
OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION)
HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT,
STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN
ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE
POSSIBILITY OF SUCH DAMAGE.
This diff is collapsed.
= CQL Binary Protocol Implementation
// URL references
:cql2: https://github.com/apache/cassandra/blob/trunk/doc/native_protocol_v2.spec[2]
:cql3: https://github.com/apache/cassandra/blob/trunk/doc/native_protocol_v3.spec[3]
// Badges
image:https://img.shields.io/badge/license-MPL_2.0-blue.svg["License: MPL-2.0", link="https://www.mozilla.org/MPL/2.0/"]
image:https://travis-ci.org/twittner/cql.svg?branch=develop["Build Status", link="https://travis-ci.org/twittner/cql"]
This Haskell library implements Cassandra's CQL Binary Protocol versions
{cql2} and {cql3}. It provides encoding and decoding functionality as well
as representations of the various protocol related types.
CQL Binary Protocol Implementation
==================================
This Haskell library implements Cassandra's CQL Binary Protocol versions
[2] and [3]. It provides encoding and decoding functionality as well
as representations of the various protocol related types.
Contributing
------------
If you want to contribute to this project please read the file
CONTRIBUTING first.
[2]: https://github.com/apache/cassandra/blob/trunk/doc/native_protocol_v2.spec
[3]: https://github.com/apache/cassandra/blob/trunk/doc/native_protocol_v3.spec
name: cql
version: 3.0.7
version: 3.1.0
synopsis: Cassandra CQL binary protocol.
stability: experimental
license: MPL-2.0
license-file: LICENSE.txt
license: OtherLicense
license-file: LICENSE
author: Toralf Wittner, Roman S. Borschel
maintainer: Toralf Wittner <tw@dtex.org>
copyright: (C) 2014-2015 Toralf Wittner, Roman S. Borschel
homepage: https://github.com/twittner/cql/
bug-reports: https://github.com/twittner/cql/issues
homepage: https://gitlab.com/twittner/cql/
bug-reports: https://gitlab.com/twittner/cql/issues
category: Database
build-type: Simple
cabal-version: >= 1.10
extra-source-files: README.asciidoc
CHANGELOG.txt
AUTHORS.txt
extra-source-files: README.md
CHANGELOG
CONTRIBUTING
AUTHORS
description:
Implementation of Cassandra's CQL Binary Protocol
......@@ -28,9 +29,13 @@ description:
Thus it can serve as a building block for writing Cassandra drivers, such
as <http://hackage.haskell.org/package/cql-io cql-io>.
flag incompatible-varint
description: Use incompatible legacy encoding for varint and decimal.
default: False
source-repository head
type: git
location: git@github.com:twittner/cql.git
type: git
location: https://gitlab.com/twittner/cql
library
default-language: Haskell2010
......@@ -52,19 +57,22 @@ library
Database.CQL.Protocol.Request
Database.CQL.Protocol.Response
if flag(incompatible-varint)
cpp-options: -DINCOMPATIBLE_VARINT
build-depends:
base >= 4.5 && < 5.0
, bytestring >= 0.10 && < 1.0
, cereal >= 0.3 && < 1.0
, Decimal >= 0.3 && < 1.0
, iproute >= 1.3 && < 2.0
, network >= 2.4 && < 3.0
, text >= 0.11 && < 2.0
, bytestring >= 0.10
, cereal >= 0.3
, Decimal >= 0.3
, iproute >= 1.3
, network >= 2.4
, text >= 0.11
, template-haskell
, time >= 1.4 && < 2.0
, transformers >= 0.3 && < 0.5
, uuid >= 1.2.6 && < 2.0
, vector >= 0.10 && < 1.0
, time >= 1.4
, transformers >= 0.3
, uuid >= 1.2.6
, vector >= 0.10
test-suite cql-tests
type: exitcode-stdio-1.0
......
-- This Source Code Form is subject to the terms of the Mozilla Public
-- License, v. 2.0. If a copy of the MPL was not distributed with this
-- file, You can obtain one at http://mozilla.org/MPL/2.0/.
-- | The CQL native protocol is a binary frame-based protocol where
-- each frame has a 'Header', a 'Length' and a body. The protocol
-- distinguishes 'Request's and 'Response's.
......
-- This Source Code Form is subject to the terms of the Mozilla Public
-- License, v. 2.0. If a copy of the MPL was not distributed with this
-- file, You can obtain one at http://mozilla.org/MPL/2.0/.
{-# LANGUAGE FlexibleInstances #-}
{-# LANGUAGE ScopedTypeVariables #-}
......
-- This Source Code Form is subject to the terms of the Mozilla Public
-- License, v. 2.0. If a copy of the MPL was not distributed with this
-- file, You can obtain one at http://mozilla.org/MPL/2.0/.
{-# LANGUAGE BangPatterns #-}
{-# LANGUAGE CPP #-}
{-# LANGUAGE OverloadedStrings #-}
......@@ -56,14 +52,12 @@ module Database.CQL.Protocol.Codec
, encodeOpCode
, decodeOpCode
, encodeColumnType
, decodeColumnType
, encodePagingState
, decodePagingState
, decodeKeyspace
, decodeTable
, decodeColumnType
, decodeQueryId
, putValue
......@@ -77,7 +71,11 @@ import Data.ByteString (ByteString)
import Data.Decimal
import Data.Int
import Data.IP
#ifdef INCOMPATIBLE_VARINT
import Data.List (unfoldr)
#else
import Data.List (foldl')
#endif
import Data.Text (Text)
import Data.UUID (UUID)
import Data.Word
......@@ -356,36 +354,6 @@ decodeOpCode = decodeByte >>= mapCode
------------------------------------------------------------------------------
-- ColumnType
encodeColumnType :: Putter ColumnType
encodeColumnType (CustomColumn x) = encodeShort 0x0000 >> encodeString x
encodeColumnType AsciiColumn = encodeShort 0x0001
encodeColumnType BigIntColumn = encodeShort 0x0002
encodeColumnType BlobColumn = encodeShort 0x0003
encodeColumnType BooleanColumn = encodeShort 0x0004
encodeColumnType CounterColumn = encodeShort 0x0005
encodeColumnType DecimalColumn = encodeShort 0x0006
encodeColumnType DoubleColumn = encodeShort 0x0007
encodeColumnType FloatColumn = encodeShort 0x0008
encodeColumnType IntColumn = encodeShort 0x0009
encodeColumnType TextColumn = encodeShort 0x000A
encodeColumnType TimestampColumn = encodeShort 0x000B
encodeColumnType UuidColumn = encodeShort 0x000C
encodeColumnType VarCharColumn = encodeShort 0x000D
encodeColumnType VarIntColumn = encodeShort 0x000E
encodeColumnType TimeUuidColumn = encodeShort 0x000F
encodeColumnType InetColumn = encodeShort 0x0010
encodeColumnType (MaybeColumn x) = encodeColumnType x
encodeColumnType (ListColumn x) = encodeShort 0x0020 >> encodeColumnType x
encodeColumnType (MapColumn x y) = encodeShort 0x0021 >> encodeColumnType x >> encodeColumnType y
encodeColumnType (SetColumn x) = encodeShort 0x0022 >> encodeColumnType x
encodeColumnType (TupleColumn xs) = encodeShort 0x0031 >> mapM_ encodeColumnType xs
encodeColumnType (UdtColumn k n xs) = do
encodeShort 0x0030
encodeString (unKeyspace k)
encodeString n
encodeShort (fromIntegral (length xs))
forM_ xs $ \(x, t) -> encodeString x >> encodeColumnType t
decodeColumnType :: Get ColumnType
decodeColumnType = decodeShort >>= toType
where
......@@ -409,9 +377,12 @@ decodeColumnType = decodeShort >>= toType
toType 0x0020 = ListColumn <$> (decodeShort >>= toType)
toType 0x0021 = MapColumn <$> (decodeShort >>= toType) <*> (decodeShort >>= toType)
toType 0x0022 = SetColumn <$> (decodeShort >>= toType)
toType 0x0030 = UdtColumn <$> (Keyspace <$> decodeString) <*> decodeString <*> do
n <- fromIntegral <$> decodeShort
replicateM n ((,) <$> decodeString <*> (decodeShort >>= toType))
toType 0x0030 = do
_ <- decodeString -- Keyspace (not used by this library)
t <- decodeString -- Type name
UdtColumn t <$> do
n <- fromIntegral <$> decodeShort
replicateM n ((,) <$> decodeString <*> (decodeShort >>= toType))
toType 0x0031 = TupleColumn <$> do
n <- fromIntegral <$> decodeShort
replicateM n (decodeShort >>= toType)
......@@ -424,7 +395,7 @@ encodePagingState :: Putter PagingState
encodePagingState (PagingState s) = encodeBytes s
decodePagingState :: Get (Maybe PagingState)
decodePagingState = liftM PagingState <$> decodeBytes
decodePagingState = fmap PagingState <$> decodeBytes
------------------------------------------------------------------------------
-- Value
......@@ -476,17 +447,17 @@ putNative _ (CqlInet x) = case x of
putWord32host b
putWord32host c
putWord32host d
putNative _ (CqlVarInt x) = integer2bytes x
putNative _ (CqlDecimal x) = do
putNative _ (CqlVarInt x) = integer2bytes x
putNative _ (CqlDecimal x) = do
put (fromIntegral (decimalPlaces x) :: Int32)
integer2bytes (decimalMantissa x)
putNative V3 (CqlUdt x) = putByteString $ runPut (mapM_ (putValue V3 . snd) x)
putNative V2 v@(CqlUdt _) = fail $ "putNative: udt: " ++ show v
putNative _ v@(CqlList _) = fail $ "putNative: collection type: " ++ show v
putNative _ v@(CqlSet _) = fail $ "putNative: collection type: " ++ show v
putNative _ v@(CqlMap _) = fail $ "putNative: collection type: " ++ show v
putNative _ v@(CqlMaybe _) = fail $ "putNative: collection type: " ++ show v
putNative _ v@(CqlTuple _) = fail $ "putNative: tuple type: " ++ show v
putNative V3 (CqlUdt x) = putByteString $ runPut (mapM_ (putValue V3 . snd) x)
putNative V2 v@(CqlUdt _) = fail $ "putNative: udt: " ++ show v
putNative _ v@(CqlList _) = fail $ "putNative: collection type: " ++ show v
putNative _ v@(CqlSet _) = fail $ "putNative: collection type: " ++ show v
putNative _ v@(CqlMap _) = fail $ "putNative: collection type: " ++ show v
putNative _ v@(CqlMaybe _) = fail $ "putNative: collection type: " ++ show v
putNative _ v@(CqlTuple _) = fail $ "putNative: tuple type: " ++ show v
-- Note: Empty lists, maps and sets are represented as null in cassandra.
getValue :: Version -> ColumnType -> Get Value
......@@ -548,17 +519,17 @@ getNative _ DecimalColumn = do
x <- get :: Get Int32
y <- bytes2integer
return (CqlDecimal (Decimal (fromIntegral x) y))
getNative V3 (UdtColumn _ _ x) = do
getNative V3 (UdtColumn _ x) = do
b <- remainingBytes
either fail return $ flip runGet b $ CqlUdt <$> do
let (n, t) = unzip x
zip n <$> mapM (getValue V3) t
getNative V2 c@(UdtColumn _ _ _) = fail $ "getNative: udt: " ++ show c
getNative _ c@(ListColumn _) = fail $ "getNative: collection type: " ++ show c
getNative _ c@(SetColumn _) = fail $ "getNative: collection type: " ++ show c
getNative _ c@(MapColumn _ _) = fail $ "getNative: collection type: " ++ show c
getNative _ c@(MaybeColumn _) = fail $ "getNative: collection type: " ++ show c
getNative _ c@(TupleColumn _) = fail $ "getNative: tuple type: " ++ show c
getNative V2 c@(UdtColumn _ _) = fail $ "getNative: udt: " ++ show c
getNative _ c@(ListColumn _) = fail $ "getNative: collection type: " ++ show c
getNative _ c@(SetColumn _) = fail $ "getNative: collection type: " ++ show c
getNative _ c@(MapColumn _ _) = fail $ "getNative: collection type: " ++ show c
getNative _ c@(MaybeColumn _) = fail $ "getNative: collection type: " ++ show c
getNative _ c@(TupleColumn _) = fail $ "getNative: tuple type: " ++ show c
getList :: Get [a] -> Get [a]
getList m = do
......@@ -593,6 +564,8 @@ toBytes s p = do
_ -> put (fromIntegral (B.length bytes) :: Int32)
putByteString bytes
#ifdef INCOMPATIBLE_VARINT
-- 'integer2bytes' and 'bytes2integer' implementations are taken
-- from cereal's instance declaration of 'Serialize' for 'Integer'
-- except that no distinction between small and large integers is made.
......@@ -622,6 +595,44 @@ bytes2integer = do
where
unstep b a = a `shiftL` 8 .|. fromIntegral b
#else
integer2bytes :: Putter Integer
integer2bytes n
| n == 0 = putWord8 0x00
| n == -1 = putWord8 0xFF
| n < 0 = do
let bytes = explode (-1) n
unless (head bytes >= 0x80) $
putWord8 0xFF
mapM_ putWord8 bytes
| otherwise = do
let bytes = explode 0 n
unless (head bytes < 0x80) $
putWord8 0x00
mapM_ putWord8 bytes
explode :: Integer -> Integer -> [Word8]
explode x n = loop n []
where
loop !i !acc
| i == x = acc
| otherwise = loop (i `shiftR` 8) (fromIntegral i : acc)
bytes2integer :: Get Integer
bytes2integer = do
msb <- getWord8
bytes <- B.unpack <$> remainingBytes
if msb < 0x80
then return (implode (msb:bytes))
else return (- (implode (map complement (msb:bytes)) + 1))
implode :: [Word8] -> Integer
implode = foldl' fun 0
where
fun i b = i `shiftL` 8 .|. fromIntegral b
#endif
------------------------------------------------------------------------------
-- Various
......
-- This Source Code Form is subject to the terms of the Mozilla Public
-- License, v. 2.0. If a copy of the MPL was not distributed with this
-- file, You can obtain one at http://mozilla.org/MPL/2.0/.
module Database.CQL.Protocol.Header
( Header (..)
, HeaderType (..)
......
-- This Source Code Form is subject to the terms of the Mozilla Public
-- License, v. 2.0. If a copy of the MPL was not distributed with this
-- file, You can obtain one at http://mozilla.org/MPL/2.0/.
-- | Like "Database.CQL.Protocol" but exports the whole
-- encode/decode machinery for all types.
module Database.CQL.Protocol.Internal
......
-- This Source Code Form is subject to the terms of the Mozilla Public
-- License, v. 2.0. If a copy of the MPL was not distributed with this
-- file, You can obtain one at http://mozilla.org/MPL/2.0/.
{-# LANGUAGE CPP #-}
{-# LANGUAGE TypeFamilies #-}
......
-- This Source Code Form is subject to the terms of the Mozilla Public
-- License, v. 2.0. If a copy of the MPL was not distributed with this
-- file, You can obtain one at http://mozilla.org/MPL/2.0/.
{-# LANGUAGE GADTs #-}
{-# LANGUAGE OverloadedStrings #-}
{-# LANGUAGE ScopedTypeVariables #-}
......
-- This Source Code Form is subject to the terms of the Mozilla Public
-- License, v. 2.0. If a copy of the MPL was not distributed with this
-- file, You can obtain one at http://mozilla.org/MPL/2.0/.
{-# LANGUAGE DeriveDataTypeable #-}
{-# LANGUAGE OverloadedStrings #-}
{-# LANGUAGE ScopedTypeVariables #-}
......
-- This Source Code Form is subject to the terms of the Mozilla Public
-- License, v. 2.0. If a copy of the MPL was not distributed with this
-- file, You can obtain one at http://mozilla.org/MPL/2.0/.
{-# LANGUAGE CPP #-}
{-# LANGUAGE ScopedTypeVariables #-}
{-# LANGUAGE TemplateHaskell #-}
......
-- This Source Code Form is subject to the terms of the Mozilla Public
-- License, v. 2.0. If a copy of the MPL was not distributed with this
-- file, You can obtain one at http://mozilla.org/MPL/2.0/.
{-# LANGUAGE CPP #-}
{-# LANGUAGE ScopedTypeVariables #-}
......@@ -189,12 +185,19 @@ element :: Cql a => Version -> Tagged a ColumnType -> Get a
element v t = getValue v (untag t) >>= either fail return . fromCql
typecheck :: [ColumnType] -> [ColumnType] -> [ColumnType]
typecheck rr cc = if and (zipWith (===) rr cc) then [] else rr
typecheck rr cc = if checkAll (===) rr cc then [] else rr
where
(MaybeColumn a) === b = a === b
(ListColumn a) === (ListColumn b) = a === b
(SetColumn a) === (SetColumn b) = a === b
(MapColumn a b) === (MapColumn c d) = a === c && b === d
TextColumn === VarCharColumn = True
VarCharColumn === TextColumn = True
a === b = a == b
checkAll f as bs = and (zipWith f as bs)
checkField (a, b) (c, d) = a == c && b === d
TextColumn === VarCharColumn = True
VarCharColumn === TextColumn = True
(MaybeColumn a) === b = a === b
(ListColumn a) === (ListColumn b) = a === b
(SetColumn a) === (SetColumn b) = a === b
(MapColumn a b) === (MapColumn c d) = a === c && b === d
(UdtColumn a as) === (UdtColumn b bs) = a == b && checkAll checkField as bs
(TupleColumn as) === (TupleColumn bs) = checkAll (===) as bs
a === b = a == b
-- This Source Code Form is subject to the terms of the Mozilla Public
-- License, v. 2.0. If a copy of the MPL was not distributed with this
-- file, You can obtain one at http://mozilla.org/MPL/2.0/.
module Database.CQL.Protocol.Types where
import Data.ByteString (ByteString)
......@@ -125,7 +121,7 @@ data ColumnType
| SetColumn !ColumnType
| MapColumn !ColumnType !ColumnType
| TupleColumn [ColumnType]
| UdtColumn !Keyspace !Text [(Text, ColumnType)]
| UdtColumn !Text [(Text, ColumnType)]
deriving (Eq)
instance Show ColumnType where
......@@ -159,9 +155,7 @@ instance Show ColumnType where
. showString (List.intercalate ", " (map show a))
. showString ">"
$ ""
show (UdtColumn k n f) = showString (unpack (unKeyspace k))
. showString "."
. showString (unpack n)
show (UdtColumn t f) = showString (unpack t)
. showString "<"
. shows (List.intercalate ", " (map show f))
. showString ">"
......
-- This Source Code Form is subject to the terms of the Mozilla Public
-- License, v. 2.0. If a copy of the MPL was not distributed with this
-- file, You can obtain one at http://mozilla.org/MPL/2.0/.
module Main where
import Test.Tasty
......
-- This Source Code Form is subject to the terms of the Mozilla Public
-- License, v. 2.0. If a copy of the MPL was not distributed with this
-- file, You can obtain one at http://mozilla.org/MPL/2.0/.
{-# LANGUAGE FlexibleInstances #-}
{-# LANGUAGE OverloadedStrings #-}
......@@ -95,7 +91,7 @@ typeof (CqlMap []) = MapColumn (CustomColumn "a") (CustomColumn "b")
typeof (CqlMap ((x,y):_)) = MapColumn (typeof x) (typeof y)
typeof (CqlCustom _) = CustomColumn "a"
typeof (CqlTuple x) = TupleColumn (map typeof x)
typeof (CqlUdt x) = UdtColumn (Keyspace "") "" (map (second typeof) x)
typeof (CqlUdt x) = UdtColumn "" (map (second typeof) x)
instance Arbitrary Value where
arbitrary = oneof
......
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment