Signed-off-by: Eduardo Silva <eduardo@treasure-data.com>

Fujimoto Seiji authored 5 years ago and Eduardo Silva committed 5 years ago

We're right now seeing persistent test failures on Travis (arm64/clang).

 * https://travis-ci.org/fluent/fluent-bit/builds/601256615
 * https://travis-ci.org/fluent/fluent-bit/builds/602281393
 * https://travis-ci.org/fluent/fluent-bit/builds/601502335



I noticed that Travis downgraded the clang version to 3.8 since last week's
build and coincidentally the failure started to occur (Before that build,
Travis used clang 6.0.0 -- I'm not sure what made it switch).

This patch tweaks travis.yml so that Travis uses clang-6 consistently.

Signed-off-by: Fujimoto Seiji <fujimoto@clear-code.com>

Signed-off-by: Eduardo Silva <eduardo@treasure-data.com>

Masanori Matsumoto authored 5 years ago and Eduardo Silva committed 5 years ago

Signed-off-by: Masanori Matsumoto <gatetwom@gmail.com>

Fujimoto Seiji authored 5 years ago and Eduardo Silva committed 5 years ago

The current flb_utils_write_str() tries to read "ahead" to decode
a multibyte UTF-8 sequence. While doing so, it never checks the
boundary of the input buffer.

This means that if we pass a truncated UTF-8 string like "\xe3",
it will read _beyond_ the boundary of the buffer and potentially
cause a crash.

This fixes the bug, and also add corresponding test cases for it.

Signed-off-by: Fujimoto Seiji <fujimoto@clear-code.com>

Takahiro YAMASHITA authored 5 years ago and Eduardo Silva committed 5 years ago

Signed-off-by: Takahiro YAMASHITA <nokute78@gmail.com>

Fujimoto Seiji authored 5 years ago and Eduardo Silva committed 5 years ago

The point is that we can build 32-bit installers on a 64-bit Windows
machine just by tweaking flags in MSVC. Because of this, the target
platform of an installer does not always match the archtecture where
the build took place.

This patch adds "win32" (or "win64") to intaller's name so that we
can easily distinguish the target architecture.

    e.g. td-agent-bit-1.4.0-win32.zip for 32-bit build

Signed-off-by: Fujimoto Seiji <fujimoto@clear-code.com>

Fujimoto Seiji authored 5 years ago and Eduardo Silva committed 5 years ago

Without this, cmake emits an error while initialization, due to
bison/flex required by record-accessor but not available.

> This is a build time dependency, you can either install the dependencies or
> disable the feature setting the CMake option -DFLB_RECORD_ACCESSOR=Off .

Signed-off-by: Fujimoto Seiji <fujimoto@clear-code.com>

Fujimoto Seiji authored 5 years ago and Eduardo Silva committed 5 years ago

Evidently MSVC doesn't like string functions like strncpy(), and emits
warnings on every call of these functions.

> warning C4996: 'strncpy': This function or variable may be unsafe.
> Consider using strncpy_s instead.

Besides, we use these function A LOT. Thus, it results in more than
hundreds warnings at compile time.

This patch fixes the issue and should keep the build process cleaner.

Signed-off-by: Fujimoto Seiji <fujimoto@clear-code.com>

Even Holthe authored 5 years ago and Eduardo Silva committed 5 years ago

This patch introduces logging of response payloads from the server when
the status code is not 200 or 201.

Signed-off-by: Even Holthe <even.holthe@bekk.no>

Signed-off-by: Eduardo Silva <eduardo@treasure-data.com>

Masanori Matsumoto authored 5 years ago and Eduardo Silva committed 5 years ago

Signed-off-by: Masanori Matsumoto <gatetwom@gmail.com>

Fujimoto Seiji authored 5 years ago and Eduardo Silva committed 5 years ago

The current flb_utils_write_str() tries to read "ahead" to decode
a multibyte UTF-8 sequence. While doing so, it never checks the
boundary of the input buffer.

This means that if we pass a truncated UTF-8 string like "\xe3",
it will read _beyond_ the boundary of the buffer and potentially
cause a crash.

This fixes the bug, and also add corresponding test cases for it.

Signed-off-by: Fujimoto Seiji <fujimoto@clear-code.com>

Takahiro YAMASHITA authored 5 years ago and Eduardo Silva committed 5 years ago

Signed-off-by: Takahiro YAMASHITA <nokute78@gmail.com>

Fujimoto Seiji authored 5 years ago and Eduardo Silva committed 5 years ago

The point is that we can build 32-bit installers on a 64-bit Windows
machine just by tweaking flags in MSVC. Because of this, the target
platform of an installer does not always match the archtecture where
the build took place.

This patch adds "win32" (or "win64") to intaller's name so that we
can easily distinguish the target architecture.

    e.g. td-agent-bit-1.4.0-win32.zip for 32-bit build

Signed-off-by: Fujimoto Seiji <fujimoto@clear-code.com>

Fujimoto Seiji authored 5 years ago and Eduardo Silva committed 5 years ago

Without this, cmake emits an error while initialization, due to
bison/flex required by record-accessor but not available.

> This is a build time dependency, you can either install the dependencies or
> disable the feature setting the CMake option -DFLB_RECORD_ACCESSOR=Off .

Signed-off-by: Fujimoto Seiji <fujimoto@clear-code.com>

Fujimoto Seiji authored 5 years ago and Eduardo Silva committed 5 years ago

Evidently MSVC doesn't like string functions like strncpy(), and emits
warnings on every call of these functions.

> warning C4996: 'strncpy': This function or variable may be unsafe.
> Consider using strncpy_s instead.

Besides, we use these function A LOT. Thus, it results in more than
hundreds warnings at compile time.

This patch fixes the issue and should keep the build process cleaner.

Signed-off-by: Fujimoto Seiji <fujimoto@clear-code.com>

Even Holthe authored 5 years ago and Eduardo Silva committed 5 years ago

This patch introduces logging of response payloads from the server when
the status code is not 200 or 201.

Signed-off-by: Even Holthe <even.holthe@bekk.no>

Signed-off-by: Eduardo Silva <eduardo@treasure-data.com>

Takahiro YAMASHITA authored 5 years ago and Eduardo Silva committed 5 years ago

Signed-off-by: Takahiro YAMASHITA <nokute78@gmail.com>

Signed-off-by: Eduardo Silva <eduardo@treasure-data.com>

Takahiro YAMASHITA authored 5 years ago and Eduardo Silva committed 5 years ago

Signed-off-by: Takahiro YAMASHITA <nokute78@gmail.com>

Signed-off-by: Eduardo Silva <eduardo@treasure-data.com>

Signed-off-by: Eduardo Silva <eduardo@treasure-data.com>

Signed-off-by: Eduardo Silva <eduardo@treasure-data.com>

Signed-off-by: Eduardo Silva <eduardo@treasure-data.com>

Signed-off-by: Eduardo Silva <eduardo@treasure-data.com>

Nick Brown authored 5 years ago and Eduardo Silva committed 5 years ago

Signed-off-by: Nicholas Brown <nick.brown@att.com>

Takahiro YAMASHITA authored 5 years ago and Eduardo Silva committed 5 years ago

Signed-off-by: Takahiro YAMASHITA <nokute78@gmail.com>

Valerian authored 5 years ago and Eduardo Silva committed 5 years ago

Signed-off-by: Valerian Pereira <valerianpereira25@gmail.com>

Fujimoto Seiji authored 5 years ago and Eduardo Silva committed 5 years ago

When we encode a multi-byte character in UTF-8 strings, we encode
each character in the format of '\\uXXXX' (e.g. 'à' => "\\u00e0").
Hence we use 6 bytes per character.

However, we previously did:

    hex_bytes = flb_utf8_len(str + i);
    if ((available - written) < (2 + hex_bytes)) {
        return FLB_FALSE;
    }

to see if the buffer has an enough space, and it almost always
miscalculated the byte size required; For instance, (2 + hex_bytes)
would evaluate to 4 (instead of 6) for 'à'.

This fixes the bug and should make flb-it-pack run fine on arm64.

Signed-off-by: Fujimoto Seiji <fujimoto@clear-code.com>

Signed-off-by: Eduardo Silva <eduardo@treasure-data.com>

Signed-off-by: Eduardo Silva <eduardo@treasure-data.com>

Signed-off-by: Eduardo Silva <eduardo@treasure-data.com>

On specific bad formed payloads, Fluent Bit crashes when reading data using
the standard input plugin.

The following patch unify the validation of the record types and only packs
valid JSON maps or arrays, for data from a different type it just triggers
and error.

note: Issue reported by Microsoft, Secutiry Vulnerability Report 47959

Signed-off-by: Eduardo Silva <eduardo@treasure-data.com>

Signed-off-by: Eduardo Silva <eduardo@treasure-data.com>

Fujimoto Seiji authored 5 years ago and Eduardo Silva committed 5 years ago

According to the Forward Protocol Specification:

> *auth* a binary string of salt to generate digest for user authentication
> (authentication not required if it's empty)

But in practice, Fluentd always include an `auth` token in HELO requests
regardless of whether it actually requires password auth or not.

This patch fixes the implementation to use empty username and password
by default, instead of spitting out an error message.

Signed-off-by: Fujimoto Seiji <fujimoto@ceptord.net>

Signed-off-by: Eduardo Silva <eduardo@treasure-data.com>

Signed-off-by: Eduardo Silva <eduardo@treasure-data.com>