From 8d81dc99fb72947ae98bc1b19760aa7dc9612821 Mon Sep 17 00:00:00 2001 From: Adam Wight <adam.wight@wikimedia.de> Date: Sun, 18 Sep 2022 12:31:27 +0200 Subject: [PATCH 01/40] Clean up changelog --- CHANGELOG.md | 9 ++------- 1 file changed, 2 insertions(+), 7 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index d476f44..fc8673f 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -7,7 +7,6 @@ The distant future; these features are out of scope for v0.x: * Clients for some of the many other Wikimedia [REST API](https://www.mediawiki.org/wiki/REST_API )s beyond core, served through RESTBase. See [issue #2](https://gitlab.com/adamwight/mediawiki_client_ex/-/issues/2). -* Local API cache to prevent accidental abuse during development. * Demonstrate a cross-wiki API call (CentralAuth). * Dump-processing interface for the archives served on https://dumps.wikimedia.org/ — might be blocked by the unavailability of bzip2 @@ -16,11 +15,11 @@ and 7zip bindings for Erlang or Elixir. * Wikidata Query Service ## 0.4.0-TODO -Holding place for everything needed to finish the 0.x series. The focus is on a few robust, core apis. +Holding place for everything needed to finish the 0.x series. The focus is on a few robust, core apis. What it should already include: * Detect server and network errors, fail fast. Show helpful API debugging in dev environment. Demonstrate how to call with error handling. -* Longer, configurable default timeouts to match servers. +* Longer, configurable default timeouts to match servers. (#18) * Use atoms for selecting known server-side event streams. Similarly, for some of the action API? * Convenient logging—a global setting to inject the logging middleware into all client plugs. @@ -30,12 +29,8 @@ some of the action API? This is the development target and next steps. * Show how to integrate with the MediaWiki OAuth2 provider (to be published), to authenticate actions on behalf of a user. * Built-in Mediawiki [REST API](https://www.mediawiki.org/wiki/API:REST_API) -* Demonstrate Wikibase API calls. * ... -## 0.4.0-TODO -This is the development target and next steps. - Breaking changes: * Wiki.Site renamed to Wiki.SiteMatrix and has an updated interface. `new()` returns an opaque sitematrix. -- GitLab From f485dbec43890a61f16eaaab90766f4f76bc2095 Mon Sep 17 00:00:00 2001 From: Adam Wight <adam.wight@wikimedia.de> Date: Sun, 18 Sep 2022 12:37:15 +0200 Subject: [PATCH 02/40] License as Apache 2.0 I'm making this decision as the sole code contributor, to this date. Apache seems to be a common license for Elixir projects, and for the language itself. Closes #21. --- CHANGELOG.md | 5 +- LICENSE | 201 +++++++++++++++++++++++++++++++++++++++++++++++++++ mix.exs | 2 +- 3 files changed, 206 insertions(+), 2 deletions(-) create mode 100644 LICENSE diff --git a/CHANGELOG.md b/CHANGELOG.md index fc8673f..8373e30 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -25,7 +25,7 @@ some of the action API? * Convenient logging—a global setting to inject the logging middleware into all client plugs. * ... -## 0.3.1-TODO +## 0.3.2-TODO This is the development target and next steps. * Show how to integrate with the MediaWiki OAuth2 provider (to be published), to authenticate actions on behalf of a user. * Built-in Mediawiki [REST API](https://www.mediawiki.org/wiki/API:REST_API) @@ -37,6 +37,9 @@ Breaking changes: Other: * Wiki.Site caches sitematrix response. +## 0.3.1 (Sep 2022) +* Settle on the Apache 2 license (bug #21). + ## 0.3.0 (Dec 2021) Breaking changes: * Switched most of the API to return `{:ok | :error, ...}` tuples. For quick migration just use the bang functions like `get!`. Or match `{:ok, result}` for fine-grained error handling. Errors will be returned as exception objects, with a crude string message for now. In a future release these will include a reason atom. diff --git a/LICENSE b/LICENSE new file mode 100644 index 0000000..261eeb9 --- /dev/null +++ b/LICENSE @@ -0,0 +1,201 @@ + Apache License + Version 2.0, January 2004 + http://www.apache.org/licenses/ + + TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION + + 1. Definitions. + + "License" shall mean the terms and conditions for use, reproduction, + and distribution as defined by Sections 1 through 9 of this document. + + "Licensor" shall mean the copyright owner or entity authorized by + the copyright owner that is granting the License. + + "Legal Entity" shall mean the union of the acting entity and all + other entities that control, are controlled by, or are under common + control with that entity. For the purposes of this definition, + "control" means (i) the power, direct or indirect, to cause the + direction or management of such entity, whether by contract or + otherwise, or (ii) ownership of fifty percent (50%) or more of the + outstanding shares, or (iii) beneficial ownership of such entity. + + "You" (or "Your") shall mean an individual or Legal Entity + exercising permissions granted by this License. + + "Source" form shall mean the preferred form for making modifications, + including but not limited to software source code, documentation + source, and configuration files. + + "Object" form shall mean any form resulting from mechanical + transformation or translation of a Source form, including but + not limited to compiled object code, generated documentation, + and conversions to other media types. + + "Work" shall mean the work of authorship, whether in Source or + Object form, made available under the License, as indicated by a + copyright notice that is included in or attached to the work + (an example is provided in the Appendix below). + + "Derivative Works" shall mean any work, whether in Source or Object + form, that is based on (or derived from) the Work and for which the + editorial revisions, annotations, elaborations, or other modifications + represent, as a whole, an original work of authorship. For the purposes + of this License, Derivative Works shall not include works that remain + separable from, or merely link (or bind by name) to the interfaces of, + the Work and Derivative Works thereof. + + "Contribution" shall mean any work of authorship, including + the original version of the Work and any modifications or additions + to that Work or Derivative Works thereof, that is intentionally + submitted to Licensor for inclusion in the Work by the copyright owner + or by an individual or Legal Entity authorized to submit on behalf of + the copyright owner. For the purposes of this definition, "submitted" + means any form of electronic, verbal, or written communication sent + to the Licensor or its representatives, including but not limited to + communication on electronic mailing lists, source code control systems, + and issue tracking systems that are managed by, or on behalf of, the + Licensor for the purpose of discussing and improving the Work, but + excluding communication that is conspicuously marked or otherwise + designated in writing by the copyright owner as "Not a Contribution." + + "Contributor" shall mean Licensor and any individual or Legal Entity + on behalf of whom a Contribution has been received by Licensor and + subsequently incorporated within the Work. + + 2. Grant of Copyright License. Subject to the terms and conditions of + this License, each Contributor hereby grants to You a perpetual, + worldwide, non-exclusive, no-charge, royalty-free, irrevocable + copyright license to reproduce, prepare Derivative Works of, + publicly display, publicly perform, sublicense, and distribute the + Work and such Derivative Works in Source or Object form. + + 3. Grant of Patent License. Subject to the terms and conditions of + this License, each Contributor hereby grants to You a perpetual, + worldwide, non-exclusive, no-charge, royalty-free, irrevocable + (except as stated in this section) patent license to make, have made, + use, offer to sell, sell, import, and otherwise transfer the Work, + where such license applies only to those patent claims licensable + by such Contributor that are necessarily infringed by their + Contribution(s) alone or by combination of their Contribution(s) + with the Work to which such Contribution(s) was submitted. If You + institute patent litigation against any entity (including a + cross-claim or counterclaim in a lawsuit) alleging that the Work + or a Contribution incorporated within the Work constitutes direct + or contributory patent infringement, then any patent licenses + granted to You under this License for that Work shall terminate + as of the date such litigation is filed. + + 4. Redistribution. You may reproduce and distribute copies of the + Work or Derivative Works thereof in any medium, with or without + modifications, and in Source or Object form, provided that You + meet the following conditions: + + (a) You must give any other recipients of the Work or + Derivative Works a copy of this License; and + + (b) You must cause any modified files to carry prominent notices + stating that You changed the files; and + + (c) You must retain, in the Source form of any Derivative Works + that You distribute, all copyright, patent, trademark, and + attribution notices from the Source form of the Work, + excluding those notices that do not pertain to any part of + the Derivative Works; and + + (d) If the Work includes a "NOTICE" text file as part of its + distribution, then any Derivative Works that You distribute must + include a readable copy of the attribution notices contained + within such NOTICE file, excluding those notices that do not + pertain to any part of the Derivative Works, in at least one + of the following places: within a NOTICE text file distributed + as part of the Derivative Works; within the Source form or + documentation, if provided along with the Derivative Works; or, + within a display generated by the Derivative Works, if and + wherever such third-party notices normally appear. The contents + of the NOTICE file are for informational purposes only and + do not modify the License. You may add Your own attribution + notices within Derivative Works that You distribute, alongside + or as an addendum to the NOTICE text from the Work, provided + that such additional attribution notices cannot be construed + as modifying the License. + + You may add Your own copyright statement to Your modifications and + may provide additional or different license terms and conditions + for use, reproduction, or distribution of Your modifications, or + for any such Derivative Works as a whole, provided Your use, + reproduction, and distribution of the Work otherwise complies with + the conditions stated in this License. + + 5. Submission of Contributions. Unless You explicitly state otherwise, + any Contribution intentionally submitted for inclusion in the Work + by You to the Licensor shall be under the terms and conditions of + this License, without any additional terms or conditions. + Notwithstanding the above, nothing herein shall supersede or modify + the terms of any separate license agreement you may have executed + with Licensor regarding such Contributions. + + 6. Trademarks. This License does not grant permission to use the trade + names, trademarks, service marks, or product names of the Licensor, + except as required for reasonable and customary use in describing the + origin of the Work and reproducing the content of the NOTICE file. + + 7. Disclaimer of Warranty. Unless required by applicable law or + agreed to in writing, Licensor provides the Work (and each + Contributor provides its Contributions) on an "AS IS" BASIS, + WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or + implied, including, without limitation, any warranties or conditions + of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A + PARTICULAR PURPOSE. You are solely responsible for determining the + appropriateness of using or redistributing the Work and assume any + risks associated with Your exercise of permissions under this License. + + 8. Limitation of Liability. In no event and under no legal theory, + whether in tort (including negligence), contract, or otherwise, + unless required by applicable law (such as deliberate and grossly + negligent acts) or agreed to in writing, shall any Contributor be + liable to You for damages, including any direct, indirect, special, + incidental, or consequential damages of any character arising as a + result of this License or out of the use or inability to use the + Work (including but not limited to damages for loss of goodwill, + work stoppage, computer failure or malfunction, or any and all + other commercial damages or losses), even if such Contributor + has been advised of the possibility of such damages. + + 9. Accepting Warranty or Additional Liability. While redistributing + the Work or Derivative Works thereof, You may choose to offer, + and charge a fee for, acceptance of support, warranty, indemnity, + or other liability obligations and/or rights consistent with this + License. However, in accepting such obligations, You may act only + on Your own behalf and on Your sole responsibility, not on behalf + of any other Contributor, and only if You agree to indemnify, + defend, and hold each Contributor harmless for any liability + incurred by, or claims asserted against, such Contributor by reason + of your accepting any such warranty or additional liability. + + END OF TERMS AND CONDITIONS + + APPENDIX: How to apply the Apache License to your work. + + To apply the Apache License to your work, attach the following + boilerplate notice, with the fields enclosed by brackets "[]" + replaced with your own identifying information. (Don't include + the brackets!) The text should be enclosed in the appropriate + comment syntax for the file format. We also recommend that a + file or class name and description of purpose be included on the + same "printed page" as the copyright notice for easier + identification within third-party archives. + + Copyright [yyyy] [name of copyright owner] + + Licensed under the Apache License, Version 2.0 (the "License"); + you may not use this file except in compliance with the License. + You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an "AS IS" BASIS, + WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + See the License for the specific language governing permissions and + limitations under the License. diff --git a/mix.exs b/mix.exs index 6230cc8..4b2bbbf 100644 --- a/mix.exs +++ b/mix.exs @@ -4,7 +4,7 @@ defmodule Elixir.MixProject do def project do [ app: :mediawiki_client, - version: "0.3.0", + version: "0.3.1", elixir: "~> 1.9", elixirc_paths: ~w(lib), start_permanent: Mix.env() == :prod, -- GitLab From dcb4658e32bc683121f804c147be7de4b9af8d37 Mon Sep 17 00:00:00 2001 From: Adam Wight <adam.wight@wikimedia.de> Date: Sun, 18 Sep 2022 13:09:53 +0200 Subject: [PATCH 03/40] Lint cleanups (elixir 1.14) --- config/dev.exs | 1 - config/prod.exs | 1 - 2 files changed, 2 deletions(-) diff --git a/config/dev.exs b/config/dev.exs index 8b13789..e69de29 100644 --- a/config/dev.exs +++ b/config/dev.exs @@ -1 +0,0 @@ - diff --git a/config/prod.exs b/config/prod.exs index 8b13789..e69de29 100644 --- a/config/prod.exs +++ b/config/prod.exs @@ -1 +0,0 @@ - -- GitLab From 31bba13a5b3274d2a79bf7b5a179d1dd12960a7e Mon Sep 17 00:00:00 2001 From: Adam Wight <adam.wight@wikimedia.de> Date: Sun, 18 Sep 2022 13:20:35 +0200 Subject: [PATCH 04/40] Consolidate config This was needed because Elixir 1.13 and 1.14 mix formatters disagree on how to format a blank file! --- config/config.exs | 36 +++++++++++++++++++++++++++++------- config/dev.exs | 0 config/prod.exs | 0 config/test.exs | 29 ----------------------------- 4 files changed, 29 insertions(+), 36 deletions(-) delete mode 100644 config/dev.exs delete mode 100644 config/prod.exs delete mode 100644 config/test.exs diff --git a/config/config.exs b/config/config.exs index ab84fa4..7c4268c 100644 --- a/config/config.exs +++ b/config/config.exs @@ -19,10 +19,32 @@ import Config # config :logger, level: :info # -# It is also possible to import configuration files, relative to this -# directory. For example, you can emulate configuration per environment -# by uncommenting the line below and defining dev.exs, test.exs and such. -# Configuration from the imported file will override the ones defined -# here (which is why it is important to import them last). -# -import_config "#{Mix.env()}.exs" +if Mix.env() == "test" do + config :mediawiki_client, + eventsource_adapter: Wiki.Tests.HTTPoisonMock, + tesla_adapter: Wiki.Tests.TeslaAdapterMock, + ores_endpoint: "https://ores.test/v3/scores/", + eventstream_endpoint: "https://stream.test/v2/stream/" + + config :git_hooks, + verbose: true, + hooks: [ + pre_push: [ + tasks: [ + {:mix_task, :clean}, + {:mix_task, :compile, ["--warnings-as-errors"]}, + {:mix_task, :format, ["--check-formatted"]}, + {:mix_task, :credo, ["--strict"]}, + {:mix_task, :test}, + {:mix_task, :coveralls}, + {:mix_task, :dialyzer}, + {:mix_task, :doctor} + ] + ] + ], + extra_success_returns: [ + # The compile step is being obstinate, it seems to give a special result + # when run with --warnings-as-errors. + {:ok, []} + ] +end diff --git a/config/dev.exs b/config/dev.exs deleted file mode 100644 index e69de29..0000000 diff --git a/config/prod.exs b/config/prod.exs deleted file mode 100644 index e69de29..0000000 diff --git a/config/test.exs b/config/test.exs deleted file mode 100644 index 0f7579e..0000000 --- a/config/test.exs +++ /dev/null @@ -1,29 +0,0 @@ -import Config - -config :mediawiki_client, - eventsource_adapter: Wiki.Tests.HTTPoisonMock, - tesla_adapter: Wiki.Tests.TeslaAdapterMock, - ores_endpoint: "https://ores.test/v3/scores/", - eventstream_endpoint: "https://stream.test/v2/stream/" - -config :git_hooks, - verbose: true, - hooks: [ - pre_push: [ - tasks: [ - {:mix_task, :clean}, - {:mix_task, :compile, ["--warnings-as-errors"]}, - {:mix_task, :format, ["--check-formatted"]}, - {:mix_task, :credo, ["--strict"]}, - {:mix_task, :test}, - {:mix_task, :coveralls}, - {:mix_task, :dialyzer}, - {:mix_task, :doctor} - ] - ] - ], - extra_success_returns: [ - # The compile step is being obstinate, it seems to give a special result - # when run with --warnings-as-errors. - {:ok, []} - ] -- GitLab From 689a560d437bb2e05cea969ad4047d54c68593ae Mon Sep 17 00:00:00 2001 From: Adam Wight <adam.wight@wikimedia.de> Date: Sun, 18 Sep 2022 13:31:31 +0200 Subject: [PATCH 05/40] Update packaging license Changing the project license, as its current sole code contributor. See #21 --- mix.exs | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/mix.exs b/mix.exs index 4b2bbbf..7469f23 100644 --- a/mix.exs +++ b/mix.exs @@ -58,7 +58,7 @@ defmodule Elixir.MixProject do files: ~w(.formatter.exs lib mix.exs *.md), name: :mediawiki_client, maintainers: ["adamwight"], - licenses: ["GPLv3"], + licenses: ["Apache-2.0"], links: %{"GitLab" => "https://gitlab.com/adamwight/mediawiki_client_ex"} ] end -- GitLab From fcb132380997bcc5a53e477bf06a63e9d5ffd6c2 Mon Sep 17 00:00:00 2001 From: Adam Wight <adam.wight@wikimedia.de> Date: Sun, 18 Sep 2022 13:48:51 +0200 Subject: [PATCH 06/40] Add missing doc delimiter --- lib/ores.ex | 1 + 1 file changed, 1 insertion(+) diff --git a/lib/ores.ex b/lib/ores.ex index 8456cc5..484f825 100644 --- a/lib/ores.ex +++ b/lib/ores.ex @@ -31,6 +31,7 @@ defmodule Wiki.Ores do # } # } # } + ``` """ alias Wiki.{Error, Util} -- GitLab From ded34a6484fcf651f0e124afb0307c8dd4b5919a Mon Sep 17 00:00:00 2001 From: Adam Wight <adam.wight@wikimedia.de> Date: Sun, 18 Sep 2022 18:02:00 +0200 Subject: [PATCH 07/40] Fix CI jobs * Typo prevented git_hooks from installing * Conslidate the CI config a bit --- .gitlab-ci.yml | 10 +++++++--- config/config.exs | 14 +++++++------- mix.exs | 7 +++++-- 3 files changed, 19 insertions(+), 12 deletions(-) diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml index b460dcf..1e15dfe 100644 --- a/.gitlab-ci.yml +++ b/.gitlab-ci.yml @@ -5,11 +5,15 @@ cache: - deps/ - priv/plts/ +variables: + MIX_ENV: test + .common: script: - mkdir -p priv/plts + - mix deps.get - mix deps.clean --unused - - MIX_ENV=test mix git_hooks.run all + - mix git_hooks.run pre_push stable: extends: .common @@ -20,14 +24,14 @@ stable: - apk add git - mix local.rebar --force - mix local.hex --force - - mix deps.get bleeding-edge: extends: .common image: elixir:latest + coverage: /\[TOTAL\]/s+[0-9.]+%/ + before_script: - mix local.rebar --force - mix local.hex --force - mix deps.unlock --all - - mix deps.get diff --git a/config/config.exs b/config/config.exs index 7c4268c..af1d443 100644 --- a/config/config.exs +++ b/config/config.exs @@ -19,13 +19,7 @@ import Config # config :logger, level: :info # -if Mix.env() == "test" do - config :mediawiki_client, - eventsource_adapter: Wiki.Tests.HTTPoisonMock, - tesla_adapter: Wiki.Tests.TeslaAdapterMock, - ores_endpoint: "https://ores.test/v3/scores/", - eventstream_endpoint: "https://stream.test/v2/stream/" - +if Mix.env() == :test do config :git_hooks, verbose: true, hooks: [ @@ -47,4 +41,10 @@ if Mix.env() == "test" do # when run with --warnings-as-errors. {:ok, []} ] + + config :mediawiki_client, + eventsource_adapter: Wiki.Tests.HTTPoisonMock, + tesla_adapter: Wiki.Tests.TeslaAdapterMock, + ores_endpoint: "https://ores.test/v3/scores/", + eventstream_endpoint: "https://stream.test/v2/stream/" end diff --git a/mix.exs b/mix.exs index 7469f23..dbe1596 100644 --- a/mix.exs +++ b/mix.exs @@ -4,7 +4,7 @@ defmodule Elixir.MixProject do def project do [ app: :mediawiki_client, - version: "0.3.1", + version: "0.3.2", elixir: "~> 1.9", elixirc_paths: ~w(lib), start_permanent: Mix.env() == :prod, @@ -59,7 +59,10 @@ defmodule Elixir.MixProject do name: :mediawiki_client, maintainers: ["adamwight"], licenses: ["Apache-2.0"], - links: %{"GitLab" => "https://gitlab.com/adamwight/mediawiki_client_ex"} + links: %{ + "GitLab" => "https://gitlab.com/adamwight/mediawiki_client_ex", + "Other MediaWiki Action API clients" => "https://www.mediawiki.org/wiki/API:Client_code" + } ] end end -- GitLab From d92f1362f096008ecbed2700ef7e05006be803fb Mon Sep 17 00:00:00 2001 From: Adam Wight <adam.wight@wikimedia.de> Date: Sun, 18 Sep 2022 18:06:45 +0200 Subject: [PATCH 08/40] Upgrade dependencies --- mix.lock | 20 ++++++++++---------- 1 file changed, 10 insertions(+), 10 deletions(-) diff --git a/mix.lock b/mix.lock index a62b7b9..ddccbf0 100644 --- a/mix.lock +++ b/mix.lock @@ -1,31 +1,31 @@ %{ "blankable": {:hex, :blankable, "1.0.0", "89ab564a63c55af117e115144e3b3b57eb53ad43ba0f15553357eb283e0ed425", [:mix], [], "hexpm", "7cf11aac0e44f4eedbee0c15c1d37d94c090cb72a8d9fddf9f7aec30f9278899"}, - "bunt": {:hex, :bunt, "0.2.0", "951c6e801e8b1d2cbe58ebbd3e616a869061ddadcc4863d0a2182541acae9a38", [:mix], [], "hexpm", "7af5c7e09fe1d40f76c8e4f9dd2be7cebd83909f31fee7cd0e9eadc567da8353"}, + "bunt": {:hex, :bunt, "0.2.1", "e2d4792f7bc0ced7583ab54922808919518d0e57ee162901a16a1b6664ef3b14", [:mix], [], "hexpm", "a330bfb4245239787b15005e66ae6845c9cd524a288f0d141c148b02603777a5"}, "certifi": {:hex, :certifi, "2.9.0", "6f2a475689dd47f19fb74334859d460a2dc4e3252a3324bd2111b8f0429e7e21", [:rebar3], [], "hexpm", "266da46bdb06d6c6d35fde799bcb28d36d985d424ad7c08b5bb48f5b5cdd4641"}, "cookie": {:hex, :cookie, "0.1.2", "71810d40c3626053f20ef0189574c0a24d4757162a6b33c3ec279a62a832670b", [:mix], [], "hexpm", "8679372c5e1e1988b6c72bf756b9204ffe82708ac356aedc8d67eb2c48f772f7"}, - "credo": {:hex, :credo, "1.6.4", "ddd474afb6e8c240313f3a7b0d025cc3213f0d171879429bf8535d7021d9ad78", [:mix], [{:bunt, "~> 0.2.0", [hex: :bunt, repo: "hexpm", optional: false]}, {:file_system, "~> 0.2.8", [hex: :file_system, repo: "hexpm", optional: false]}, {:jason, "~> 1.0", [hex: :jason, repo: "hexpm", optional: false]}], "hexpm", "c28f910b61e1ff829bffa056ef7293a8db50e87f2c57a9b5c3f57eee124536b7"}, + "credo": {:hex, :credo, "1.6.7", "323f5734350fd23a456f2688b9430e7d517afb313fbd38671b8a4449798a7854", [:mix], [{:bunt, "~> 0.2.1", [hex: :bunt, repo: "hexpm", optional: false]}, {:file_system, "~> 0.2.8", [hex: :file_system, repo: "hexpm", optional: false]}, {:jason, "~> 1.0", [hex: :jason, repo: "hexpm", optional: false]}], "hexpm", "41e110bfb007f7eda7f897c10bf019ceab9a0b269ce79f015d54b0dcf4fc7dd3"}, "decimal": {:hex, :decimal, "2.0.0", "a78296e617b0f5dd4c6caf57c714431347912ffb1d0842e998e9792b5642d697", [:mix], [], "hexpm", "34666e9c55dea81013e77d9d87370fe6cb6291d1ef32f46a1600230b1d44f577"}, - "dialyxir": {:hex, :dialyxir, "1.1.0", "c5aab0d6e71e5522e77beff7ba9e08f8e02bad90dfbeffae60eaf0cb47e29488", [:mix], [{:erlex, ">= 0.2.6", [hex: :erlex, repo: "hexpm", optional: false]}], "hexpm", "07ea8e49c45f15264ebe6d5b93799d4dd56a44036cf42d0ad9c960bc266c0b9a"}, - "doctor": {:hex, :doctor, "0.18.0", "114934c1740239953208a39db617699b7e2660770e81129d7f95cdf7837ab766", [:mix], [{:decimal, "~> 2.0", [hex: :decimal, repo: "hexpm", optional: false]}], "hexpm", "829c88c365f72c0666e443ea670ffb6f180de7b90c23d536edabdd8c722b88f4"}, - "earmark_parser": {:hex, :earmark_parser, "1.4.25", "2024618731c55ebfcc5439d756852ec4e85978a39d0d58593763924d9a15916f", [:mix], [], "hexpm", "56749c5e1c59447f7b7a23ddb235e4b3defe276afc220a6227237f3efe83f51e"}, + "dialyxir": {:hex, :dialyxir, "1.2.0", "58344b3e87c2e7095304c81a9ae65cb68b613e28340690dfe1a5597fd08dec37", [:mix], [{:erlex, ">= 0.2.6", [hex: :erlex, repo: "hexpm", optional: false]}], "hexpm", "61072136427a851674cab81762be4dbeae7679f85b1272b6d25c3a839aff8463"}, + "doctor": {:hex, :doctor, "0.19.0", "f7974836bc85756b38b99de46cc2c6ba36741f21d8eabcbef78f6806ca6769ed", [:mix], [{:decimal, "~> 2.0", [hex: :decimal, repo: "hexpm", optional: false]}], "hexpm", "504f17473dc6b39618e693c5198d85e274b056b73eb4a4605431aec0f42f0023"}, + "earmark_parser": {:hex, :earmark_parser, "1.4.26", "f4291134583f373c7d8755566122908eb9662df4c4b63caa66a0eabe06569b0a", [:mix], [], "hexpm", "48d460899f8a0c52c5470676611c01f64f3337bad0b26ddab43648428d94aabc"}, "erlex": {:hex, :erlex, "0.2.6", "c7987d15e899c7a2f34f5420d2a2ea0d659682c06ac607572df55a43753aa12e", [:mix], [], "hexpm", "2ed2e25711feb44d52b17d2780eabf998452f6efda104877a3881c2f8c0c0c75"}, "eventsource_ex": {:hex, :eventsource_ex, "1.1.0", "41ec298216bb6bd48d92ca8256bd730393fe9534c7c8400282a6abf795bad103", [:mix], [{:httpoison, "~> 1.5", [hex: :httpoison, repo: "hexpm", optional: false]}], "hexpm", "d11d13ab4d5845426f90c055983e89e8d70fab6b57dcc2125743be195c6953a3"}, - "ex_doc": {:hex, :ex_doc, "0.28.4", "001a0ea6beac2f810f1abc3dbf4b123e9593eaa5f00dd13ded024eae7c523298", [:mix], [{:earmark_parser, "~> 1.4.19", [hex: :earmark_parser, repo: "hexpm", optional: false]}, {:makeup_elixir, "~> 0.14", [hex: :makeup_elixir, repo: "hexpm", optional: false]}, {:makeup_erlang, "~> 0.1", [hex: :makeup_erlang, repo: "hexpm", optional: false]}], "hexpm", "bf85d003dd34911d89c8ddb8bda1a958af3471a274a4c2150a9c01c78ac3f8ed"}, - "excoveralls": {:hex, :excoveralls, "0.14.5", "5c685449596e962c779adc8f4fb0b4de3a5b291c6121097572a3aa5400c386d3", [:mix], [{:hackney, "~> 1.16", [hex: :hackney, repo: "hexpm", optional: false]}, {:jason, "~> 1.0", [hex: :jason, repo: "hexpm", optional: false]}], "hexpm", "e9b4a9bf10e9a6e48b94159e13b4b8a1b05400f17ac16cc363ed8734f26e1f4e"}, + "ex_doc": {:hex, :ex_doc, "0.28.5", "3e52a6d2130ce74d096859e477b97080c156d0926701c13870a4e1f752363279", [:mix], [{:earmark_parser, "~> 1.4.19", [hex: :earmark_parser, repo: "hexpm", optional: false]}, {:makeup_elixir, "~> 0.14", [hex: :makeup_elixir, repo: "hexpm", optional: false]}, {:makeup_erlang, "~> 0.1", [hex: :makeup_erlang, repo: "hexpm", optional: false]}], "hexpm", "d2c4b07133113e9aa3e9ba27efb9088ba900e9e51caa383919676afdf09ab181"}, + "excoveralls": {:hex, :excoveralls, "0.14.6", "610e921e25b180a8538229ef547957f7e04bd3d3e9a55c7c5b7d24354abbba70", [:mix], [{:hackney, "~> 1.16", [hex: :hackney, repo: "hexpm", optional: false]}, {:jason, "~> 1.0", [hex: :jason, repo: "hexpm", optional: false]}], "hexpm", "0eceddaa9785cfcefbf3cd37812705f9d8ad34a758e513bb975b081dce4eb11e"}, "file_system": {:hex, :file_system, "0.2.10", "fb082005a9cd1711c05b5248710f8826b02d7d1784e7c3451f9c1231d4fc162d", [:mix], [], "hexpm", "41195edbfb562a593726eda3b3e8b103a309b733ad25f3d642ba49696bf715dc"}, "gen_stage": {:hex, :gen_stage, "1.1.2", "b1656cd4ba431ed02c5656fe10cb5423820847113a07218da68eae5d6a260c23", [:mix], [], "hexpm", "9e39af23140f704e2b07a3e29d8f05fd21c2aaf4088ff43cb82be4b9e3148d02"}, "git_hooks": {:hex, :git_hooks, "0.7.3", "09489e94d88dfc767662e22aff2b6208bd7cf555a19dd0e1477cca4683ce0701", [:mix], [{:blankable, "~> 1.0.0", [hex: :blankable, repo: "hexpm", optional: false]}, {:recase, "~> 0.7.0", [hex: :recase, repo: "hexpm", optional: false]}], "hexpm", "d6ddedeb4d3a8602bc3f84e087a38f6150a86d9e790628ed8bc70e6d90681659"}, "hackney": {:hex, :hackney, "1.18.1", "f48bf88f521f2a229fc7bae88cf4f85adc9cd9bcf23b5dc8eb6a1788c662c4f6", [:rebar3], [{:certifi, "~>2.9.0", [hex: :certifi, repo: "hexpm", optional: false]}, {:idna, "~>6.1.0", [hex: :idna, repo: "hexpm", optional: false]}, {:metrics, "~>1.0.0", [hex: :metrics, repo: "hexpm", optional: false]}, {:mimerl, "~>1.1", [hex: :mimerl, repo: "hexpm", optional: false]}, {:parse_trans, "3.3.1", [hex: :parse_trans, repo: "hexpm", optional: false]}, {:ssl_verify_fun, "~>1.1.0", [hex: :ssl_verify_fun, repo: "hexpm", optional: false]}, {:unicode_util_compat, "~>0.7.0", [hex: :unicode_util_compat, repo: "hexpm", optional: false]}], "hexpm", "a4ecdaff44297e9b5894ae499e9a070ea1888c84afdd1fd9b7b2bc384950128e"}, - "httpoison": {:hex, :httpoison, "1.8.1", "df030d96de89dad2e9983f92b0c506a642d4b1f4a819c96ff77d12796189c63e", [:mix], [{:hackney, "~> 1.17", [hex: :hackney, repo: "hexpm", optional: false]}], "hexpm", "35156a6d678d6d516b9229e208942c405cf21232edd632327ecfaf4fd03e79e0"}, + "httpoison": {:hex, :httpoison, "1.8.2", "9eb9c63ae289296a544842ef816a85d881d4a31f518a0fec089aaa744beae290", [:mix], [{:hackney, "~> 1.17", [hex: :hackney, repo: "hexpm", optional: false]}], "hexpm", "2bb350d26972e30c96e2ca74a1aaf8293d61d0742ff17f01e0279fef11599921"}, "idna": {:hex, :idna, "6.1.1", "8a63070e9f7d0c62eb9d9fcb360a7de382448200fbbd1b106cc96d3d8099df8d", [:rebar3], [{:unicode_util_compat, "~>0.7.0", [hex: :unicode_util_compat, repo: "hexpm", optional: false]}], "hexpm", "92376eb7894412ed19ac475e4a86f7b413c1b9fbb5bd16dccd57934157944cea"}, - "jason": {:hex, :jason, "1.3.0", "fa6b82a934feb176263ad2df0dbd91bf633d4a46ebfdffea0c8ae82953714946", [:mix], [{:decimal, "~> 1.0 or ~> 2.0", [hex: :decimal, repo: "hexpm", optional: true]}], "hexpm", "53fc1f51255390e0ec7e50f9cb41e751c260d065dcba2bf0d08dc51a4002c2ac"}, + "jason": {:hex, :jason, "1.4.0", "e855647bc964a44e2f67df589ccf49105ae039d4179db7f6271dfd3843dc27e6", [:mix], [{:decimal, "~> 1.0 or ~> 2.0", [hex: :decimal, repo: "hexpm", optional: true]}], "hexpm", "79a3791085b2a0f743ca04cec0f7be26443738779d09302e01318f97bdb82121"}, "makeup": {:hex, :makeup, "1.1.0", "6b67c8bc2882a6b6a445859952a602afc1a41c2e08379ca057c0f525366fc3ca", [:mix], [{:nimble_parsec, "~> 1.2.2 or ~> 1.3", [hex: :nimble_parsec, repo: "hexpm", optional: false]}], "hexpm", "0a45ed501f4a8897f580eabf99a2e5234ea3e75a4373c8a52824f6e873be57a6"}, "makeup_elixir": {:hex, :makeup_elixir, "0.16.0", "f8c570a0d33f8039513fbccaf7108c5d750f47d8defd44088371191b76492b0b", [:mix], [{:makeup, "~> 1.0", [hex: :makeup, repo: "hexpm", optional: false]}, {:nimble_parsec, "~> 1.2.3", [hex: :nimble_parsec, repo: "hexpm", optional: false]}], "hexpm", "28b2cbdc13960a46ae9a8858c4bebdec3c9a6d7b4b9e7f4ed1502f8159f338e7"}, "makeup_erlang": {:hex, :makeup_erlang, "0.1.1", "3fcb7f09eb9d98dc4d208f49cc955a34218fc41ff6b84df7c75b3e6e533cc65f", [:mix], [{:makeup, "~> 1.0", [hex: :makeup, repo: "hexpm", optional: false]}], "hexpm", "174d0809e98a4ef0b3309256cbf97101c6ec01c4ab0b23e926a9e17df2077cbb"}, "metrics": {:hex, :metrics, "1.0.1", "25f094dea2cda98213cecc3aeff09e940299d950904393b2a29d191c346a8486", [:rebar3], [], "hexpm", "69b09adddc4f74a40716ae54d140f93beb0fb8978d8636eaded0c31b6f099f16"}, "mime": {:hex, :mime, "1.6.0", "dabde576a497cef4bbdd60aceee8160e02a6c89250d6c0b29e56c0dfb00db3d2", [:mix], [], "hexpm", "31a1a8613f8321143dde1dafc36006a17d28d02bdfecb9e95a880fa7aabd19a7"}, "mimerl": {:hex, :mimerl, "1.2.0", "67e2d3f571088d5cfd3e550c383094b47159f3eee8ffa08e64106cdf5e981be3", [:rebar3], [], "hexpm", "f278585650aa581986264638ebf698f8bb19df297f66ad91b18910dfc6e19323"}, - "mox": {:hex, :mox, "1.0.1", "b651bf0113265cda0ba3a827fcb691f848b683c373b77e7d7439910a8d754d6e", [:mix], [], "hexpm", "35bc0dea5499d18db4ef7fe4360067a59b06c74376eb6ab3bd67e6295b133469"}, + "mox": {:hex, :mox, "1.0.2", "dc2057289ac478b35760ba74165b4b3f402f68803dd5aecd3bfd19c183815d64", [:mix], [], "hexpm", "f9864921b3aaf763c8741b5b8e6f908f44566f1e427b2630e89e9a73b981fef2"}, "nimble_parsec": {:hex, :nimble_parsec, "1.2.3", "244836e6e3f1200c7f30cb56733fd808744eca61fd182f731eac4af635cc6d0b", [:mix], [], "hexpm", "c8d789e39b9131acf7b99291e93dae60ab48ef14a7ee9d58c6964f59efb570b0"}, "parse_trans": {:hex, :parse_trans, "3.3.1", "16328ab840cc09919bd10dab29e431da3af9e9e7e7e6f0089dd5a2d2820011d8", [:rebar3], [], "hexpm", "07cd9577885f56362d414e8c4c4e6bdf10d43a8767abb92d24cbe8b24c54888b"}, "recase": {:hex, :recase, "0.7.0", "3f2f719f0886c7a3b7fe469058ec539cb7bbe0023604ae3bce920e186305e5ae", [:mix], [], "hexpm", "36f5756a9f552f4a94b54a695870e32f4e72d5fad9c25e61bc4a3151c08a4e0c"}, -- GitLab From d9ddf74b0b5a5ef64a980ddf4af18d6747a1140d Mon Sep 17 00:00:00 2001 From: Adam Wight <adam.wight@wikimedia.de> Date: Sun, 18 Sep 2022 18:26:38 +0200 Subject: [PATCH 09/40] Mysterious CI failure It looks like :git_hooks :extra_success_returns isn't defined. Playing with the env variables just in case. --- .gitlab-ci.yml | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml index 1e15dfe..9bed008 100644 --- a/.gitlab-ci.yml +++ b/.gitlab-ci.yml @@ -6,6 +6,7 @@ cache: - priv/plts/ variables: + # FIXME: doesn't get fully exported down to mix git_hooks? MIX_ENV: test .common: @@ -13,7 +14,7 @@ variables: - mkdir -p priv/plts - mix deps.get - mix deps.clean --unused - - mix git_hooks.run pre_push + - MIX_ENV=test mix git_hooks.run pre_push stable: extends: .common -- GitLab From 5df0b6179518ddd01d042d4429f513b3d98bc32f Mon Sep 17 00:00:00 2001 From: Adam Wight <adam.wight@wikimedia.de> Date: Sun, 18 Sep 2022 22:41:06 +0200 Subject: [PATCH 10/40] Work around broken hook by running compile in simpler mode This isn't about the warnings, but about the response format of `mix compile` confusing git_hooks despite the already worked-around extra_success_returns config. --- config/config.exs | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/config/config.exs b/config/config.exs index af1d443..18816ce 100644 --- a/config/config.exs +++ b/config/config.exs @@ -26,7 +26,9 @@ if Mix.env() == :test do pre_push: [ tasks: [ {:mix_task, :clean}, - {:mix_task, :compile, ["--warnings-as-errors"]}, + # TODO: Restore the flag once githooks can handle the response again. + # {:mix_task, :compile, ["--warnings-as-errors"]}, + {:mix_task, :compile}, {:mix_task, :format, ["--check-formatted"]}, {:mix_task, :credo, ["--strict"]}, {:mix_task, :test}, -- GitLab From ce3f04d2fdad790985494aee42d4c8f6bc04eb7d Mon Sep 17 00:00:00 2001 From: Adam Wight <adam.wight@wikimedia.de> Date: Sun, 18 Sep 2022 22:41:09 +0000 Subject: [PATCH 11/40] Ci fix --- .doctor.exs | 4 ++-- .gitlab-ci.yml | 4 ++++ README.md | 13 ++++--------- config/config.exs | 6 ++---- coveralls.json | 5 +---- lib/event_streams.ex | 15 ++++++++++++--- mix.exs | 9 ++++++++- 7 files changed, 33 insertions(+), 23 deletions(-) diff --git a/.doctor.exs b/.doctor.exs index dc53696..5d40aa5 100644 --- a/.doctor.exs +++ b/.doctor.exs @@ -1,9 +1,9 @@ %Doctor.Config{ ignore_modules: [], ignore_paths: [~r(contrib/.*)], - min_module_doc_coverage: 85, + min_module_doc_coverage: 100, min_module_spec_coverage: 100, - min_overall_doc_coverage: 94, + min_overall_doc_coverage: 100, min_overall_spec_coverage: 100, moduledoc_required: true, raise: false, diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml index 9bed008..226b007 100644 --- a/.gitlab-ci.yml +++ b/.gitlab-ci.yml @@ -30,6 +30,10 @@ bleeding-edge: extends: .common image: elixir:latest + artifacts: + expose_as: coverage report + paths: + - cover/excoveralls.html coverage: /\[TOTAL\]/s+[0-9.]+%/ before_script: diff --git a/README.md b/README.md index 64f82b6..d9462d6 100644 --- a/README.md +++ b/README.md @@ -92,8 +92,9 @@ For example: config :mediawiki_client, # HTTP client to use for EventStreams. Defaults to `HTTPoison`. eventsource_adapter: Wiki.Tests.HTTPoisonMock, - # API endpoint for `Wiki.EventStreams`. Defaults to - eventstream_endpoint: + # API endpoint for `Wiki.EventStreams`. + # Defaults to https://stream.wikimedia.org/v2/stream/ + eventstream_endpoint: "http://stream.test/v2/stream/" tesla_adapter: Wiki.Tests.TeslaAdapterMock, ores_endpoint: "https://ores.test/v3/scores/", ``` @@ -103,13 +104,7 @@ config :mediawiki_client, Find the [project homepage](https://gitlab.com/adamwight/mediawiki_client_ex) on GitLab. To contribute, feel free to write an issue, push a merge request, or contact the author. -Several linters are configured, these are called on `git push`. To install hooks: - -```shell script -mix git_hooks.install -``` - -To push without running tests call `git push --no-verify`, but please don't do this on the main branch. +Several linters are configured, these are called on `git push`. To skip tests call `git push --no-verify`, but please don't do this on the main branch. To generate a test coverage report, ```shell script diff --git a/config/config.exs b/config/config.exs index 18816ce..deb4092 100644 --- a/config/config.exs +++ b/config/config.exs @@ -26,13 +26,11 @@ if Mix.env() == :test do pre_push: [ tasks: [ {:mix_task, :clean}, - # TODO: Restore the flag once githooks can handle the response again. - # {:mix_task, :compile, ["--warnings-as-errors"]}, - {:mix_task, :compile}, + {:mix_task, :compile, ["--warnings-as-errors"]}, {:mix_task, :format, ["--check-formatted"]}, {:mix_task, :credo, ["--strict"]}, {:mix_task, :test}, - {:mix_task, :coveralls}, + {:mix_task, :"coveralls.html"}, {:mix_task, :dialyzer}, {:mix_task, :doctor} ] diff --git a/coveralls.json b/coveralls.json index f68e4d9..a33b0a5 100644 --- a/coveralls.json +++ b/coveralls.json @@ -1,8 +1,5 @@ { "coverage_options": { "minimum_coverage": 94 - }, - "skip_files": [ - "contrib" - ] + } } diff --git a/lib/event_streams.ex b/lib/event_streams.ex index 52e544e..6a430f7 100644 --- a/lib/event_streams.ex +++ b/lib/event_streams.ex @@ -43,6 +43,11 @@ defmodule Wiki.EventStreams do @type reply :: {:noreply, [map], state} + @doc """ + ## Arguments + + * {:producer, state} + """ @spec start_link(keyword) :: GenServer.on_start() def start_link(args) do GenStage.start_link(__MODULE__, args, name: __MODULE__) @@ -133,6 +138,13 @@ defmodule Wiki.EventStreams do alias Wiki.EventStreams # TODO: Should this logic be moved to init/1? + @doc """ + ## Arguments + + * `endpoint` - Optionally override the default endpoint URL. + * `streams` - One or more atoms with the stream names to subscribe to. + * `send_to` - Optional application which will receive the events. + """ @spec start_link(keyword) :: GenServer.on_start() def start_link(args) do endpoint = args[:endpoint] || EventStreams.default_endpoint() @@ -148,9 +160,6 @@ defmodule Wiki.EventStreams do end @impl true - @spec init(term) :: - {:ok, {:supervisor.sup_flags(), [:supervisor.child_spec()]}} - | :ignore def init(args) do {:ok, args} end diff --git a/mix.exs b/mix.exs index dbe1596..33439ba 100644 --- a/mix.exs +++ b/mix.exs @@ -20,7 +20,14 @@ defmodule Elixir.MixProject do extras: ~w(CHANGELOG.md README.md), main: "readme" ], - test_coverage: [tool: ExCoveralls] + test_coverage: [tool: ExCoveralls], + preferred_cli_env: [ + "coveralls.html": :test, + credo: :test, + dialyzer: :test, + doctor: :test, + "git_hooks.run": :test + ] ] end -- GitLab From b6d343d4dd29cdc72a04ea46c45c754f0acdf954 Mon Sep 17 00:00:00 2001 From: Adam Wight <adam.wight@wikimedia.de> Date: Mon, 19 Sep 2022 22:29:15 +0200 Subject: [PATCH 12/40] Mix alias test.all rather than git_hooks MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Simpler to call explicitly—also it was a nuisance for development. --- .gitlab-ci.yml | 2 +- config/config.exs | 35 ----------------------------------- mix.exs | 22 +++++++++++++++++----- 3 files changed, 18 insertions(+), 41 deletions(-) diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml index 226b007..9e0064f 100644 --- a/.gitlab-ci.yml +++ b/.gitlab-ci.yml @@ -14,7 +14,7 @@ variables: - mkdir -p priv/plts - mix deps.get - mix deps.clean --unused - - MIX_ENV=test mix git_hooks.run pre_push + - mix test.all stable: extends: .common diff --git a/config/config.exs b/config/config.exs index deb4092..57f016f 100644 --- a/config/config.exs +++ b/config/config.exs @@ -6,42 +6,7 @@ import Config # if you want to provide default values for your application for # third-party users, it should be done in your "mix.exs" file. -# You can configure your application as: -# -# config :elixir, key: :value -# -# and access this configuration in your application as: -# -# Application.get_env(:elixir, :key) -# -# You can also configure a third-party app: -# -# config :logger, level: :info -# - if Mix.env() == :test do - config :git_hooks, - verbose: true, - hooks: [ - pre_push: [ - tasks: [ - {:mix_task, :clean}, - {:mix_task, :compile, ["--warnings-as-errors"]}, - {:mix_task, :format, ["--check-formatted"]}, - {:mix_task, :credo, ["--strict"]}, - {:mix_task, :test}, - {:mix_task, :"coveralls.html"}, - {:mix_task, :dialyzer}, - {:mix_task, :doctor} - ] - ] - ], - extra_success_returns: [ - # The compile step is being obstinate, it seems to give a special result - # when run with --warnings-as-errors. - {:ok, []} - ] - config :mediawiki_client, eventsource_adapter: Wiki.Tests.HTTPoisonMock, tesla_adapter: Wiki.Tests.TeslaAdapterMock, diff --git a/mix.exs b/mix.exs index 33439ba..36ce9b3 100644 --- a/mix.exs +++ b/mix.exs @@ -23,10 +23,23 @@ defmodule Elixir.MixProject do test_coverage: [tool: ExCoveralls], preferred_cli_env: [ "coveralls.html": :test, - credo: :test, - dialyzer: :test, - doctor: :test, - "git_hooks.run": :test + "test.all": :test + ], + aliases: aliases() + ] + end + + defp aliases do + [ + "test.all": [ + "format --check-formatted", + "clean", + "compile --warnings-as-errors", + "credo --strict", + "test", + "coveralls.html", + "dialyzer", + "doctor" ] ] end @@ -47,7 +60,6 @@ defmodule Elixir.MixProject do {:ex_doc, "~> 0.0", only: :dev, runtime: false}, {:excoveralls, "~> 0.14", only: :test, runtime: false}, {:gen_stage, "~> 1.0"}, - {:git_hooks, "~> 0.0", only: :test, runtime: false}, {:httpoison, "~> 1.0"}, {:jason, "~> 1.0"}, {:mime, "~> 1.0"}, -- GitLab From 0f0e057c0014a25fc8c9dba03c05b667e54ffaff Mon Sep 17 00:00:00 2001 From: Adam Wight <adam.wight@wikimedia.de> Date: Mon, 19 Sep 2022 22:34:10 +0200 Subject: [PATCH 13/40] Update, repair changelog The SiteMatrix breaking change should have been made in v0.4, plus a version of deprecation. Next time... --- CHANGELOG.md | 11 +++++++---- 1 file changed, 7 insertions(+), 4 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 8373e30..2c309fd 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -25,21 +25,24 @@ some of the action API? * Convenient logging—a global setting to inject the logging middleware into all client plugs. * ... -## 0.3.2-TODO +## 0.3.3-TODO This is the development target and next steps. * Show how to integrate with the MediaWiki OAuth2 provider (to be published), to authenticate actions on behalf of a user. * Built-in Mediawiki [REST API](https://www.mediawiki.org/wiki/API:REST_API) * ... +## 0.3.2-TODO +* `mix test.all` task replaces git_hooks for development. +* ... + +## 0.3.1 (Sep 2022) Breaking changes: * Wiki.Site renamed to Wiki.SiteMatrix and has an updated interface. `new()` returns an opaque sitematrix. +* Settle on the Apache 2 license (bug #21). Other: * Wiki.Site caches sitematrix response. -## 0.3.1 (Sep 2022) -* Settle on the Apache 2 license (bug #21). - ## 0.3.0 (Dec 2021) Breaking changes: * Switched most of the API to return `{:ok | :error, ...}` tuples. For quick migration just use the bang functions like `get!`. Or match `{:ok, result}` for fine-grained error handling. Errors will be returned as exception objects, with a crude string message for now. In a future release these will include a reason atom. -- GitLab From 4174b540db05076db7b5f9a946fc983fa6ff6ada Mon Sep 17 00:00:00 2001 From: Adam Wight <adam.wight@wikimedia.de> Date: Tue, 20 Sep 2022 06:35:44 +0200 Subject: [PATCH 14/40] Convert eventsource config variables to keyword opts --- CHANGELOG.md | 12 ++++-- README.md | 5 --- config/config.exs | 4 +- lib/event_streams.ex | 75 +++++++++++++++++-------------------- test/event_streams_test.exs | 14 ++++++- 5 files changed, 56 insertions(+), 54 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 2c309fd..cbbb296 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -14,7 +14,7 @@ and 7zip bindings for Erlang or Elixir. * meter how many calls are made, when, and bandwidth used. * Wikidata Query Service -## 0.4.0-TODO +## 0.6.0-TODO Holding place for everything needed to finish the 0.x series. The focus is on a few robust, core apis. What it should already include: @@ -25,13 +25,17 @@ some of the action API? * Convenient logging—a global setting to inject the logging middleware into all client plugs. * ... -## 0.3.3-TODO +## 0.5.0-TODO This is the development target and next steps. * Show how to integrate with the MediaWiki OAuth2 provider (to be published), to authenticate actions on behalf of a user. * Built-in Mediawiki [REST API](https://www.mediawiki.org/wiki/API:REST_API) * ... -## 0.3.2-TODO +## 0.4.0-TODO +Breaking changes: +* Config variables are deprecated, all customization is done via keyword options. + +Other: * `mix test.all` task replaces git_hooks for development. * ... @@ -41,7 +45,7 @@ Breaking changes: * Settle on the Apache 2 license (bug #21). Other: -* Wiki.Site caches sitematrix response. +* Wiki.SiteMatrix caches the response. ## 0.3.0 (Dec 2021) Breaking changes: diff --git a/README.md b/README.md index d9462d6..a71bad9 100644 --- a/README.md +++ b/README.md @@ -90,11 +90,6 @@ For example: ```elixir config :mediawiki_client, - # HTTP client to use for EventStreams. Defaults to `HTTPoison`. - eventsource_adapter: Wiki.Tests.HTTPoisonMock, - # API endpoint for `Wiki.EventStreams`. - # Defaults to https://stream.wikimedia.org/v2/stream/ - eventstream_endpoint: "http://stream.test/v2/stream/" tesla_adapter: Wiki.Tests.TeslaAdapterMock, ores_endpoint: "https://ores.test/v3/scores/", ``` diff --git a/config/config.exs b/config/config.exs index 57f016f..52c5af2 100644 --- a/config/config.exs +++ b/config/config.exs @@ -8,8 +8,6 @@ import Config if Mix.env() == :test do config :mediawiki_client, - eventsource_adapter: Wiki.Tests.HTTPoisonMock, tesla_adapter: Wiki.Tests.TeslaAdapterMock, - ores_endpoint: "https://ores.test/v3/scores/", - eventstream_endpoint: "https://stream.test/v2/stream/" + ores_endpoint: "https://ores.test/v3/scores/" end diff --git a/lib/event_streams.ex b/lib/event_streams.ex index 6a430f7..0a052cd 100644 --- a/lib/event_streams.ex +++ b/lib/event_streams.ex @@ -49,8 +49,15 @@ defmodule Wiki.EventStreams do * {:producer, state} """ @spec start_link(keyword) :: GenServer.on_start() - def start_link(args) do - GenStage.start_link(__MODULE__, args, name: __MODULE__) + def start_link(opts) do + {:ok, relay_pid} = GenStage.start_link(__MODULE__, opts, name: __MODULE__) + # FIXME: Rationalize the supervision tree. + source_opts = Keyword.put_new(opts, :stream_to, relay_pid) + + {:ok, _} = + Supervisor.start_link([{Wiki.EventStreams.Source, source_opts}], strategy: :one_for_one) + + {:ok, relay_pid} end @impl true @@ -103,12 +110,16 @@ defmodule Wiki.EventStreams do alias Wiki.Util + @default_endpoint "https://stream.wikimedia.org/v2/stream/" + @doc false - @spec child_spec(String.t()) :: map - def child_spec(endpoint) do - headers = [ - {"user-agent", Util.user_agent()} - ] + @spec child_spec(Keyword.t()) :: map + def child_spec(opts \\ []) do + adapter = opts[:adapter] + endpoint = opts[:endpoint] || @default_endpoint + sink = opts[:stream_to] + user_agent = opts[:user_agent] || Util.user_agent() + url = endpoint <> normalize_streams(opts[:streams]) %{ id: Source, @@ -118,16 +129,23 @@ defmodule Wiki.EventStreams do EventsourceEx, :new, [ - endpoint, + url, [ - adapter: Application.get_env(:mediawiki_client, :eventsource_adapter), - headers: headers, - stream_to: Relay + adapter: adapter, + headers: [{"user-agent", user_agent}], + stream_to: sink ] ] } } end + + @spec normalize_streams(atom | [atom]) :: atom | String.t() + defp normalize_streams(streams) + + defp normalize_streams(streams) when is_list(streams), do: Enum.join(streams, ",") + + defp normalize_streams(streams), do: streams end defmodule RelaySupervisor do @@ -135,8 +153,6 @@ defmodule Wiki.EventStreams do use Supervisor, restart: :permanent - alias Wiki.EventStreams - # TODO: Should this logic be moved to init/1? @doc """ ## Arguments @@ -146,36 +162,24 @@ defmodule Wiki.EventStreams do * `send_to` - Optional application which will receive the events. """ @spec start_link(keyword) :: GenServer.on_start() - def start_link(args) do - endpoint = args[:endpoint] || EventStreams.default_endpoint() - url = endpoint <> normalize_streams(args[:streams]) - sink = args[:send_to] || self() - - children = [ - {Relay, sink}, - {Source, url} - ] + def start_link(opts) do + children = [{Relay, opts}] {:ok, _} = Supervisor.start_link(children, strategy: :one_for_one) end @impl true def init(args) do + # TODO: I don't know why this is necessary or what it's doing. {:ok, args} end - - @spec normalize_streams(atom | [atom]) :: atom | String.t() - defp normalize_streams(streams) - - defp normalize_streams(streams) when is_list(streams), do: Enum.join(streams, ",") - - defp normalize_streams(streams), do: streams end @type options :: [option] @type option :: - {:endpoint, String.t()} + {:adapter, module()} + | {:endpoint, String.t()} | {:send_to, GenServer.server()} | {:streams, atom | [atom]} @@ -193,6 +197,7 @@ defmodule Wiki.EventStreams do ## Arguments - `options` - Keyword list, + - `{:adapter, module}` - Override HTTPoison adapter. - `{:endpoint, url}` - Override default endpoint. - `{:send_to, pid | module}` - Instead of using the built-in streaming relay, send the events directly to your own process. @@ -211,14 +216,4 @@ defmodule Wiki.EventStreams do def stream(options \\ []) do GenStage.stream([Relay], options) end - - @doc false - @spec default_endpoint() :: String.t() - def default_endpoint do - Application.get_env( - :mediawiki_client, - :eventstream_endpoint, - "https://stream.wikimedia.org/v2/stream/" - ) - end end diff --git a/test/event_streams_test.exs b/test/event_streams_test.exs index 6acd32f..ebee4e7 100644 --- a/test/event_streams_test.exs +++ b/test/event_streams_test.exs @@ -29,7 +29,12 @@ defmodule EventStreamsTest do |> Enum.map(&send(target, %{chunk: &1})) end) - start_supervised({EventStreams, streams: ~w(revision-create revision-score)}) + start_supervised( + {EventStreams, + adapter: HTTPoisonMock, + endpoint: "https://stream.test/v2/stream/", + streams: ~w(revision-create revision-score)} + ) result = EventStreams.stream() @@ -48,7 +53,12 @@ defmodule EventStreamsTest do assert url == "https://stream.test/v2/stream/revision-create" end) - start_supervised({EventStreams, streams: "revision-create"}) + start_supervised( + {EventStreams, + adapter: HTTPoisonMock, + endpoint: "https://stream.test/v2/stream/", + streams: "revision-create"} + ) EventStreams.stream() |> Stream.take(1) -- GitLab From c4a9fdb91d08d8e32a7c509a864c52d5dbf26a84 Mon Sep 17 00:00:00 2001 From: Adam Wight <adam.wight@wikimedia.de> Date: Tue, 20 Sep 2022 18:30:14 +0200 Subject: [PATCH 15/40] Convert ORES environment config to a keyword option --- README.md | 4 +--- config/config.exs | 3 +-- lib/ores.ex | 17 +++++++---------- test/ores_test.exs | 2 +- 4 files changed, 10 insertions(+), 16 deletions(-) diff --git a/README.md b/README.md index a71bad9..61f4e64 100644 --- a/README.md +++ b/README.md @@ -81,7 +81,6 @@ the `Wiki.Action.get!()` method raises an error directly. Configuration variables (TODO: refactor away from config) under the `:mediawiki_client` application will override the built-in adapters. -* `:ores_endpoint` - API endpoint for `Wiki.Ores`. * `:tesla_adapter` - Defaults to `Tesla.Adapter.Hackney`, a stable client which performs certificate validation. * `:user_agent` - Sent in request headers, defaults to `mediawiki_client_ex/<version>`... @@ -90,8 +89,7 @@ For example: ```elixir config :mediawiki_client, - tesla_adapter: Wiki.Tests.TeslaAdapterMock, - ores_endpoint: "https://ores.test/v3/scores/", + tesla_adapter: Wiki.Tests.TeslaAdapterMock ``` ## Development diff --git a/config/config.exs b/config/config.exs index 52c5af2..d470a37 100644 --- a/config/config.exs +++ b/config/config.exs @@ -8,6 +8,5 @@ import Config if Mix.env() == :test do config :mediawiki_client, - tesla_adapter: Wiki.Tests.TeslaAdapterMock, - ores_endpoint: "https://ores.test/v3/scores/" + tesla_adapter: Wiki.Tests.TeslaAdapterMock end diff --git a/lib/ores.ex b/lib/ores.ex index 484f825..22900d7 100644 --- a/lib/ores.ex +++ b/lib/ores.ex @@ -36,6 +36,8 @@ defmodule Wiki.Ores do alias Wiki.{Error, Util} + @default_endpoint "https://ores.wikimedia.org/v3/scores/" + # TODO: # * Wrap models? # * Chunk at 50 revisions per request. @@ -47,28 +49,23 @@ defmodule Wiki.Ores do ## Arguments - `project` - Short code for the wiki where your articles appear. For example, "enwiki" for English Wikipedia. + - `opts` - Keyword options + - `:endpoint` - Override the base URL to query ## Return value Returns an opaque client object, which should be passed to `request/2`. """ @spec new(String.t()) :: Tesla.Client.t() - def new(project) do - url = endpoint() <> project <> "/" + def new(project, opts \\ []) do + endpoint = opts[:endpoint] || @default_endpoint + url = endpoint <> project <> "/" client([ {Tesla.Middleware.BaseUrl, url} ]) end - defp endpoint do - Application.get_env( - :mediawiki_client, - :ores_endpoint, - "https://ores.wikimedia.org/v3/scores/" - ) - end - @doc """ Make an ORES request. diff --git a/test/ores_test.exs b/test/ores_test.exs index 33e602b..9b95d27 100644 --- a/test/ores_test.exs +++ b/test/ores_test.exs @@ -45,7 +45,7 @@ defmodule OresTest do end) session = - Ores.new("testwiki") + Ores.new("testwiki", endpoint: "https://ores.test/v3/scores/") |> Ores.request!(%{ models: "damaging", revids: 12_345 -- GitLab From 02ba3b9d8dec23811e65c1d01fe22f84eab13e7f Mon Sep 17 00:00:00 2001 From: Adam Wight <adam.wight@wikimedia.de> Date: Tue, 20 Sep 2022 23:14:26 +0200 Subject: [PATCH 16/40] Convert http adapter configuration to keyword options --- README.md | 4 +-- config/config.exs | 12 -------- lib/action.ex | 59 ++++++++++++++++++++------------------- lib/ores.ex | 39 +++++++++++++------------- lib/site_matrix.ex | 22 +++++++++------ lib/util.ex | 7 ----- test/action_test.exs | 37 +++++++++++------------- test/ores_test.exs | 14 +++++----- test/site_matrix_test.exs | 8 ++++-- 9 files changed, 94 insertions(+), 108 deletions(-) delete mode 100644 config/config.exs diff --git a/README.md b/README.md index 61f4e64..895e164 100644 --- a/README.md +++ b/README.md @@ -81,15 +81,13 @@ the `Wiki.Action.get!()` method raises an error directly. Configuration variables (TODO: refactor away from config) under the `:mediawiki_client` application will override the built-in adapters. -* `:tesla_adapter` - Defaults to `Tesla.Adapter.Hackney`, a stable client which -performs certificate validation. * `:user_agent` - Sent in request headers, defaults to `mediawiki_client_ex/<version>`... For example: ```elixir config :mediawiki_client, - tesla_adapter: Wiki.Tests.TeslaAdapterMock + user_agent: Foobar app <foo@bar> ``` ## Development diff --git a/config/config.exs b/config/config.exs deleted file mode 100644 index d470a37..0000000 --- a/config/config.exs +++ /dev/null @@ -1,12 +0,0 @@ -import Config - -# This configuration is loaded before any dependency and is restricted -# to this project. If another project depends on this project, this -# file won't be loaded nor affect the parent project. For this reason, -# if you want to provide default values for your application for -# third-party users, it should be done in your "mix.exs" file. - -if Mix.env() == :test do - config :mediawiki_client, - tesla_adapter: Wiki.Tests.TeslaAdapterMock -end diff --git a/lib/action.ex b/lib/action.ex index df78030..3645faf 100644 --- a/lib/action.ex +++ b/lib/action.ex @@ -188,6 +188,8 @@ defmodule Wiki.Action do alias Wiki.{Action.Session, Error, SiteMatrix, Util} + @default_adapter Tesla.Adapter.Hackney + @doc """ Create a new client session @@ -208,17 +210,8 @@ defmodule Wiki.Action do end def new(url, opts) do - # TODO: This belongs in client/1, maybe pass options through? - middleware = - if opts[:accumulate] do - [Wiki.StatefulClient.CumulativeResult] - else - [] - end ++ - [{Tesla.Middleware.BaseUrl, url}] - %Session{ - __client__: client(middleware) + __client__: client(url, opts) } end @@ -484,25 +477,33 @@ defmodule Wiki.Action do end end - @spec client(list) :: Tesla.Client.t() - defp client(extra) do - middleware = - extra ++ - [ - {Tesla.Middleware.Compression, format: "gzip"}, - Wiki.StatefulClient.CookieJar, - Tesla.Middleware.FormUrlencoded, - {Tesla.Middleware.Headers, - [ - {"user-agent", Util.user_agent()} - ]}, - Tesla.Middleware.FollowRedirects, - Tesla.Middleware.JSON - # Debugging only: - # Tesla.Middleware.Logger - ] - - Tesla.client(middleware, Util.default_adapter()) + @spec client(binary(), keyword()) :: Tesla.Client.t() + defp client(url, opts) do + adapter = opts[:adapter] || @default_adapter + + extra = + if opts[:accumulate] do + [Wiki.StatefulClient.CumulativeResult] + else + [] + end + + (extra ++ + [ + {Tesla.Middleware.BaseUrl, url}, + {Tesla.Middleware.Compression, format: "gzip"}, + Wiki.StatefulClient.CookieJar, + Tesla.Middleware.FormUrlencoded, + {Tesla.Middleware.Headers, + [ + {"user-agent", Util.user_agent()} + ]}, + Tesla.Middleware.FollowRedirects, + Tesla.Middleware.JSON + # Debugging only: + # Tesla.Middleware.Logger + ]) + |> Tesla.client(adapter) end end diff --git a/lib/ores.ex b/lib/ores.ex index 22900d7..837b251 100644 --- a/lib/ores.ex +++ b/lib/ores.ex @@ -36,6 +36,7 @@ defmodule Wiki.Ores do alias Wiki.{Error, Util} + @default_adapter Tesla.Adapter.Hackney @default_endpoint "https://ores.wikimedia.org/v3/scores/" # TODO: @@ -61,9 +62,7 @@ defmodule Wiki.Ores do endpoint = opts[:endpoint] || @default_endpoint url = endpoint <> project <> "/" - client([ - {Tesla.Middleware.BaseUrl, url} - ]) + client(url, opts) end @doc """ @@ -151,22 +150,22 @@ defmodule Wiki.Ores do "unknown" end - @spec client(list) :: Tesla.Client.t() - defp client(extra) do - middleware = - extra ++ - [ - {Tesla.Middleware.Compression, format: "gzip"}, - {Tesla.Middleware.Headers, - [ - {"user-agent", Util.user_agent()} - ]}, - Tesla.Middleware.FollowRedirects, - Tesla.Middleware.JSON - # Debugging only: - # Tesla.Middleware.Logger - ] - - Tesla.client(middleware, Util.default_adapter()) + @spec client(binary(), keyword()) :: Tesla.Client.t() + defp client(url, opts) do + adapter = opts[:adapter] || @default_adapter + + [ + {Tesla.Middleware.BaseUrl, url}, + {Tesla.Middleware.Compression, format: "gzip"}, + {Tesla.Middleware.Headers, + [ + {"user-agent", Util.user_agent()} + ]}, + Tesla.Middleware.FollowRedirects, + Tesla.Middleware.JSON + # Debugging only: + # Tesla.Middleware.Logger + ] + |> Tesla.client(adapter) end end diff --git a/lib/site_matrix.ex b/lib/site_matrix.ex index a961fd1..9b87d62 100644 --- a/lib/site_matrix.ex +++ b/lib/site_matrix.ex @@ -47,11 +47,11 @@ defmodule Wiki.SiteMatrix do @spec new(keyword()) :: sitematrix_state() def new(opts \\ []) do - api = Keyword.get(opts, :api, @metawiki_api) + api = opts[:api] || @metawiki_api %{ api: api, - sites: do_get_all(api) + sites: do_get_all(api, opts) } end @@ -71,9 +71,16 @@ defmodule Wiki.SiteMatrix do {:ok, Map.values(sitematrix.sites)} end - @spec do_get_all(binary()) :: map() - defp do_get_all(api) do - fetch_sitematrix(api) + @spec action_client(binary(), keyword()) :: Wiki.Action.Session.t() + defp action_client(api, opts) do + api + |> Wiki.Action.new(opts) + end + + @spec do_get_all(binary(), keyword()) :: map() + defp do_get_all(api, opts) do + action_client(api, opts) + |> fetch_sitematrix() |> Enum.map(&site_spec/1) |> Map.new(fn site -> {site.dbname, site} end) end @@ -90,9 +97,8 @@ defmodule Wiki.SiteMatrix do } end - defp fetch_sitematrix(api) do - api - |> Wiki.Action.new() + defp fetch_sitematrix(client) do + client |> Wiki.Action.stream( action: :sitematrix, smsiteprop: [:dbname, :lang, :sitename, :url] diff --git a/lib/util.ex b/lib/util.ex index f9cfb53..3b4aeb6 100644 --- a/lib/util.ex +++ b/lib/util.ex @@ -3,16 +3,9 @@ defmodule Wiki.Util do @version Mix.Project.config()[:version] - @doc false - @spec default_adapter() :: atom - def default_adapter do - Application.get_env(:mediawiki_client, :tesla_adapter, Tesla.Adapter.Hackney) - end - @doc false @spec user_agent() :: String.t() def user_agent do - # TODO: Is there a way to use Elixir.MixProject.project()[:version]? Application.get_env( :mediawiki_client, :user_agent, diff --git a/test/action_test.exs b/test/action_test.exs index d8b599d..de22ecf 100644 --- a/test/action_test.exs +++ b/test/action_test.exs @@ -13,6 +13,12 @@ defmodule ActionTest do setup :verify_on_exit! + defp create_session(opts \\ []) do + site = opts[:site] || @url + opts = Keyword.put_new(opts, :adapter, TeslaAdapterMock) + Action.new(site, opts) + end + test "parses site structure" do site = %Wiki.SiteMatrix.Spec{base_url: "https://aawiki.test", dbname: "aawiki"} @@ -29,12 +35,11 @@ defmodule ActionTest do }} end) - site |> Action.new() |> Action.get!(foo: :bar) + create_session(site: site) |> Action.get!(foo: :bar) end test "returns new session" do - session = Action.new(@url) - %{__client__: client} = session + %{__client__: client} = create_session() assert length(client.pre) >= 1 end @@ -71,10 +76,7 @@ defmodule ActionTest do end) session = - Action.new( - @url, - accumulate: true - ) + create_session(accumulate: true) |> Action.get!( action: :query, meta: :siteinfo, @@ -116,11 +118,7 @@ defmodule ActionTest do {:ok, %Env{env | body: canned_response, headers: [], status: 200}} end) - session = - Action.new( - @url, - accumulate: true - ) + session = create_session(accumulate: true) session = %Session{ session @@ -161,7 +159,7 @@ defmodule ActionTest do {:ok, %Env{env | status: 200, body: %{a: 'b'}}} end) - Action.new(@url) + create_session() |> Action.get!(multivalue: ["Foo|Bar", "Baz"]) end @@ -213,11 +211,9 @@ defmodule ActionTest do }} end) - session = Action.new(@url) - session = %Session{ - session + create_session() | state: [ cookies: %{"a" => "b"} ] @@ -277,7 +273,7 @@ defmodule ActionTest do end) recent_changes = - Action.new(@url) + create_session() |> Action.stream( action: :query, list: :recentchanges, @@ -297,7 +293,8 @@ defmodule ActionTest do end) assert_raise Error, ~r/404/, fn -> - Action.new(@url) |> Action.get!(foo: :bar) + create_session() + |> Action.get!(foo: :bar) end end @@ -308,7 +305,7 @@ defmodule ActionTest do end) assert_raise Error, ":econnrefused", fn -> - Action.new(@url) + create_session() |> Action.get!(foo: :bar) end end @@ -375,7 +372,7 @@ defmodule ActionTest do end) assert_raise Error, message, fn -> - Action.new(@url) |> Action.get!(foo: :bar) + create_session() |> Action.get!(foo: :bar) end end end diff --git a/test/ores_test.exs b/test/ores_test.exs index 9b95d27..565bcca 100644 --- a/test/ores_test.exs +++ b/test/ores_test.exs @@ -45,7 +45,7 @@ defmodule OresTest do end) session = - Ores.new("testwiki", endpoint: "https://ores.test/v3/scores/") + Ores.new("testwiki", adapter: TeslaAdapterMock, endpoint: "https://ores.test/v3/scores/") |> Ores.request!(%{ models: "damaging", revids: 12_345 @@ -122,7 +122,7 @@ defmodule OresTest do end) session = - Ores.new("testwiki") + Ores.new("testwiki", adapter: TeslaAdapterMock) |> Ores.request!(%{ models: ~w(damaging wp10), revids: [12_345, 67_890] @@ -138,7 +138,7 @@ defmodule OresTest do end) assert_raise Error, ":econnrefused", fn -> - Ores.new("testwiki") + Ores.new("testwiki", adapter: TeslaAdapterMock) |> Ores.request!(%{}) end end @@ -150,7 +150,7 @@ defmodule OresTest do end) assert_raise Error, "Empty response", fn -> - Ores.new("testwiki") + Ores.new("testwiki", adapter: TeslaAdapterMock) |> Ores.request!(%{}) end end @@ -164,7 +164,7 @@ defmodule OresTest do # FIXME: should report malformed, not empty assert_raise Error, "Empty response", fn -> - Ores.new("testwiki") + Ores.new("testwiki", adapter: TeslaAdapterMock) |> Ores.request!(%{}) end end @@ -176,7 +176,7 @@ defmodule OresTest do end) assert_raise Error, "Error received with HTTP status 500", fn -> - Ores.new("testwiki") + Ores.new("testwiki", adapter: TeslaAdapterMock) |> Ores.request!(%{}) end end @@ -195,7 +195,7 @@ defmodule OresTest do end) assert_raise Error, "No scorers available for zenwiki", fn -> - Ores.new("zenwiki") + Ores.new("zenwiki", adapter: TeslaAdapterMock) |> Ores.request!(%{}) end end diff --git a/test/site_matrix_test.exs b/test/site_matrix_test.exs index db95ecf..fd28458 100644 --- a/test/site_matrix_test.exs +++ b/test/site_matrix_test.exs @@ -59,6 +59,10 @@ defmodule SiteTest do } ) |> Jason.decode!() + defp create_sitematrix() do + SiteMatrix.new(adapter: TeslaAdapterMock) + end + test "get_all sites" do TeslaAdapterMock |> expect(:call, fn env, _opts -> @@ -66,7 +70,7 @@ defmodule SiteTest do {:ok, %Env{env | body: @response, headers: headers, status: 200}} end) - assert SiteMatrix.new() |> SiteMatrix.get_all() == + assert create_sitematrix() |> SiteMatrix.get_all() == {:ok, [ %SiteMatrix.Spec{ @@ -115,7 +119,7 @@ defmodule SiteTest do {:ok, %Env{env | body: @response, headers: headers, status: 200}} end) - sitematrix = SiteMatrix.new() + sitematrix = create_sitematrix() assert sitematrix |> SiteMatrix.get!("aawiktionary") == %SiteMatrix.Spec{ base_url: "https://aa.wiktionary.org", -- GitLab From 2562100b5f46b1d1fa09a50c0be954a53a148256 Mon Sep 17 00:00:00 2001 From: Adam Wight <adam.wight@wikimedia.de> Date: Tue, 20 Sep 2022 18:51:59 +0200 Subject: [PATCH 17/40] Convert user agent config to keyword option --- README.md | 14 -------------- coveralls.json | 2 +- lib/action.ex | 3 ++- lib/event_streams.ex | 2 +- lib/ores.ex | 5 +++-- lib/util.ex | 10 +++------- test/action_test.exs | 19 ++++++++++++++----- test/site_matrix_test.exs | 2 +- 8 files changed, 25 insertions(+), 32 deletions(-) diff --git a/README.md b/README.md index 895e164..c2728d1 100644 --- a/README.md +++ b/README.md @@ -76,20 +76,6 @@ Methods come in an assertive and a non-assertive form, for example the `Wiki.Action.get()` method returns an `{:ok, ...}` or a `{:error, ...}` tuple, and the `Wiki.Action.get!()` method raises an error directly. -### Configuration - -Configuration variables (TODO: refactor away from config) under the -`:mediawiki_client` application will override the built-in adapters. - -* `:user_agent` - Sent in request headers, defaults to `mediawiki_client_ex/<version>`... - -For example: - -```elixir -config :mediawiki_client, - user_agent: Foobar app <foo@bar> -``` - ## Development Find the [project homepage](https://gitlab.com/adamwight/mediawiki_client_ex) on GitLab. diff --git a/coveralls.json b/coveralls.json index a33b0a5..4f9e7da 100644 --- a/coveralls.json +++ b/coveralls.json @@ -1,5 +1,5 @@ { "coverage_options": { - "minimum_coverage": 94 + "minimum_coverage": 93 } } diff --git a/lib/action.ex b/lib/action.ex index 3645faf..92fa760 100644 --- a/lib/action.ex +++ b/lib/action.ex @@ -480,6 +480,7 @@ defmodule Wiki.Action do @spec client(binary(), keyword()) :: Tesla.Client.t() defp client(url, opts) do adapter = opts[:adapter] || @default_adapter + user_agent = opts[:user_agent] || Util.default_user_agent() extra = if opts[:accumulate] do @@ -496,7 +497,7 @@ defmodule Wiki.Action do Tesla.Middleware.FormUrlencoded, {Tesla.Middleware.Headers, [ - {"user-agent", Util.user_agent()} + {"user-agent", user_agent} ]}, Tesla.Middleware.FollowRedirects, Tesla.Middleware.JSON diff --git a/lib/event_streams.ex b/lib/event_streams.ex index 0a052cd..6fed5bf 100644 --- a/lib/event_streams.ex +++ b/lib/event_streams.ex @@ -118,7 +118,7 @@ defmodule Wiki.EventStreams do adapter = opts[:adapter] endpoint = opts[:endpoint] || @default_endpoint sink = opts[:stream_to] - user_agent = opts[:user_agent] || Util.user_agent() + user_agent = opts[:user_agent] || Util.default_user_agent() url = endpoint <> normalize_streams(opts[:streams]) %{ diff --git a/lib/ores.ex b/lib/ores.ex index 837b251..a97114f 100644 --- a/lib/ores.ex +++ b/lib/ores.ex @@ -57,7 +57,7 @@ defmodule Wiki.Ores do Returns an opaque client object, which should be passed to `request/2`. """ - @spec new(String.t()) :: Tesla.Client.t() + @spec new(String.t(), keyword()) :: Tesla.Client.t() def new(project, opts \\ []) do endpoint = opts[:endpoint] || @default_endpoint url = endpoint <> project <> "/" @@ -153,13 +153,14 @@ defmodule Wiki.Ores do @spec client(binary(), keyword()) :: Tesla.Client.t() defp client(url, opts) do adapter = opts[:adapter] || @default_adapter + user_agent = opts[:user_agent] || Util.default_user_agent() [ {Tesla.Middleware.BaseUrl, url}, {Tesla.Middleware.Compression, format: "gzip"}, {Tesla.Middleware.Headers, [ - {"user-agent", Util.user_agent()} + {"user-agent", user_agent} ]}, Tesla.Middleware.FollowRedirects, Tesla.Middleware.JSON diff --git a/lib/util.ex b/lib/util.ex index 3b4aeb6..ad914b4 100644 --- a/lib/util.ex +++ b/lib/util.ex @@ -4,13 +4,9 @@ defmodule Wiki.Util do @version Mix.Project.config()[:version] @doc false - @spec user_agent() :: String.t() - def user_agent do - Application.get_env( - :mediawiki_client, - :user_agent, - "mediawiki_client_ex/#{@version} (spam@ludd.net)" - ) + @spec default_user_agent() :: String.t() + def default_user_agent do + "mediawiki_client_ex/#{@version} (spam@ludd.net)" end end diff --git a/test/action_test.exs b/test/action_test.exs index de22ecf..99832cd 100644 --- a/test/action_test.exs +++ b/test/action_test.exs @@ -102,11 +102,6 @@ defmodule ActionTest do TeslaAdapterMock |> expect(:call, fn env, _opts -> - [{"user-agent", user_agent}] = env.headers - - assert String.match?(user_agent, ~r/mediawiki_client_ex.*\d.*/) - assert env.method == :get - assert env.query == [ action: :query, format: :json, @@ -375,4 +370,18 @@ defmodule ActionTest do create_session() |> Action.get!(foo: :bar) end end + + test "can override user agent" do + TeslaAdapterMock + |> expect(:call, fn env, _opts -> + [{"user-agent", user_agent}] = env.headers + + assert user_agent == "foo bar" + + {:ok, %Env{env | body: %{bar: :baz}, status: 200}} + end) + + create_session(user_agent: "foo bar") + |> Action.get!(action: :query) + end end diff --git a/test/site_matrix_test.exs b/test/site_matrix_test.exs index fd28458..452ebc9 100644 --- a/test/site_matrix_test.exs +++ b/test/site_matrix_test.exs @@ -59,7 +59,7 @@ defmodule SiteTest do } ) |> Jason.decode!() - defp create_sitematrix() do + defp create_sitematrix do SiteMatrix.new(adapter: TeslaAdapterMock) end -- GitLab From e791298a182a72885e533d063ee78745c7872725 Mon Sep 17 00:00:00 2001 From: Adam Wight <adam.wight@wikimedia.de> Date: Tue, 20 Sep 2022 23:37:06 +0200 Subject: [PATCH 18/40] Release 0.4 Bumping the major version for visibility: application config deprecation might affect some people. --- CHANGELOG.md | 6 +++++- README.md | 2 +- mix.exs | 2 +- mix.lock | 3 --- 4 files changed, 7 insertions(+), 6 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index cbbb296..bb06edc 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -31,7 +31,11 @@ This is the development target and next steps. * Built-in Mediawiki [REST API](https://www.mediawiki.org/wiki/API:REST_API) * ... -## 0.4.0-TODO +## 0.4.1-TODO +Small changes in progress... +* ... + +## 0.4.0 (Sep 2022) Breaking changes: * Config variables are deprecated, all customization is done via keyword options. diff --git a/README.md b/README.md index c2728d1..d508a8f 100644 --- a/README.md +++ b/README.md @@ -21,7 +21,7 @@ Install this package by adding `mediawiki_client` to your dependencies in `mix.e ```elixir def deps do [ - {:mediawiki_client, "~> 0.3"} + {:mediawiki_client, "~> 0.4"} ] end ``` diff --git a/mix.exs b/mix.exs index 36ce9b3..e7827af 100644 --- a/mix.exs +++ b/mix.exs @@ -4,7 +4,7 @@ defmodule Elixir.MixProject do def project do [ app: :mediawiki_client, - version: "0.3.2", + version: "0.4.0", elixir: "~> 1.9", elixirc_paths: ~w(lib), start_permanent: Mix.env() == :prod, diff --git a/mix.lock b/mix.lock index ddccbf0..ae5c393 100644 --- a/mix.lock +++ b/mix.lock @@ -1,5 +1,4 @@ %{ - "blankable": {:hex, :blankable, "1.0.0", "89ab564a63c55af117e115144e3b3b57eb53ad43ba0f15553357eb283e0ed425", [:mix], [], "hexpm", "7cf11aac0e44f4eedbee0c15c1d37d94c090cb72a8d9fddf9f7aec30f9278899"}, "bunt": {:hex, :bunt, "0.2.1", "e2d4792f7bc0ced7583ab54922808919518d0e57ee162901a16a1b6664ef3b14", [:mix], [], "hexpm", "a330bfb4245239787b15005e66ae6845c9cd524a288f0d141c148b02603777a5"}, "certifi": {:hex, :certifi, "2.9.0", "6f2a475689dd47f19fb74334859d460a2dc4e3252a3324bd2111b8f0429e7e21", [:rebar3], [], "hexpm", "266da46bdb06d6c6d35fde799bcb28d36d985d424ad7c08b5bb48f5b5cdd4641"}, "cookie": {:hex, :cookie, "0.1.2", "71810d40c3626053f20ef0189574c0a24d4757162a6b33c3ec279a62a832670b", [:mix], [], "hexpm", "8679372c5e1e1988b6c72bf756b9204ffe82708ac356aedc8d67eb2c48f772f7"}, @@ -14,7 +13,6 @@ "excoveralls": {:hex, :excoveralls, "0.14.6", "610e921e25b180a8538229ef547957f7e04bd3d3e9a55c7c5b7d24354abbba70", [:mix], [{:hackney, "~> 1.16", [hex: :hackney, repo: "hexpm", optional: false]}, {:jason, "~> 1.0", [hex: :jason, repo: "hexpm", optional: false]}], "hexpm", "0eceddaa9785cfcefbf3cd37812705f9d8ad34a758e513bb975b081dce4eb11e"}, "file_system": {:hex, :file_system, "0.2.10", "fb082005a9cd1711c05b5248710f8826b02d7d1784e7c3451f9c1231d4fc162d", [:mix], [], "hexpm", "41195edbfb562a593726eda3b3e8b103a309b733ad25f3d642ba49696bf715dc"}, "gen_stage": {:hex, :gen_stage, "1.1.2", "b1656cd4ba431ed02c5656fe10cb5423820847113a07218da68eae5d6a260c23", [:mix], [], "hexpm", "9e39af23140f704e2b07a3e29d8f05fd21c2aaf4088ff43cb82be4b9e3148d02"}, - "git_hooks": {:hex, :git_hooks, "0.7.3", "09489e94d88dfc767662e22aff2b6208bd7cf555a19dd0e1477cca4683ce0701", [:mix], [{:blankable, "~> 1.0.0", [hex: :blankable, repo: "hexpm", optional: false]}, {:recase, "~> 0.7.0", [hex: :recase, repo: "hexpm", optional: false]}], "hexpm", "d6ddedeb4d3a8602bc3f84e087a38f6150a86d9e790628ed8bc70e6d90681659"}, "hackney": {:hex, :hackney, "1.18.1", "f48bf88f521f2a229fc7bae88cf4f85adc9cd9bcf23b5dc8eb6a1788c662c4f6", [:rebar3], [{:certifi, "~>2.9.0", [hex: :certifi, repo: "hexpm", optional: false]}, {:idna, "~>6.1.0", [hex: :idna, repo: "hexpm", optional: false]}, {:metrics, "~>1.0.0", [hex: :metrics, repo: "hexpm", optional: false]}, {:mimerl, "~>1.1", [hex: :mimerl, repo: "hexpm", optional: false]}, {:parse_trans, "3.3.1", [hex: :parse_trans, repo: "hexpm", optional: false]}, {:ssl_verify_fun, "~>1.1.0", [hex: :ssl_verify_fun, repo: "hexpm", optional: false]}, {:unicode_util_compat, "~>0.7.0", [hex: :unicode_util_compat, repo: "hexpm", optional: false]}], "hexpm", "a4ecdaff44297e9b5894ae499e9a070ea1888c84afdd1fd9b7b2bc384950128e"}, "httpoison": {:hex, :httpoison, "1.8.2", "9eb9c63ae289296a544842ef816a85d881d4a31f518a0fec089aaa744beae290", [:mix], [{:hackney, "~> 1.17", [hex: :hackney, repo: "hexpm", optional: false]}], "hexpm", "2bb350d26972e30c96e2ca74a1aaf8293d61d0742ff17f01e0279fef11599921"}, "idna": {:hex, :idna, "6.1.1", "8a63070e9f7d0c62eb9d9fcb360a7de382448200fbbd1b106cc96d3d8099df8d", [:rebar3], [{:unicode_util_compat, "~>0.7.0", [hex: :unicode_util_compat, repo: "hexpm", optional: false]}], "hexpm", "92376eb7894412ed19ac475e4a86f7b413c1b9fbb5bd16dccd57934157944cea"}, @@ -28,7 +26,6 @@ "mox": {:hex, :mox, "1.0.2", "dc2057289ac478b35760ba74165b4b3f402f68803dd5aecd3bfd19c183815d64", [:mix], [], "hexpm", "f9864921b3aaf763c8741b5b8e6f908f44566f1e427b2630e89e9a73b981fef2"}, "nimble_parsec": {:hex, :nimble_parsec, "1.2.3", "244836e6e3f1200c7f30cb56733fd808744eca61fd182f731eac4af635cc6d0b", [:mix], [], "hexpm", "c8d789e39b9131acf7b99291e93dae60ab48ef14a7ee9d58c6964f59efb570b0"}, "parse_trans": {:hex, :parse_trans, "3.3.1", "16328ab840cc09919bd10dab29e431da3af9e9e7e7e6f0089dd5a2d2820011d8", [:rebar3], [], "hexpm", "07cd9577885f56362d414e8c4c4e6bdf10d43a8767abb92d24cbe8b24c54888b"}, - "recase": {:hex, :recase, "0.7.0", "3f2f719f0886c7a3b7fe469058ec539cb7bbe0023604ae3bce920e186305e5ae", [:mix], [], "hexpm", "36f5756a9f552f4a94b54a695870e32f4e72d5fad9c25e61bc4a3151c08a4e0c"}, "ssl_verify_fun": {:hex, :ssl_verify_fun, "1.1.6", "cf344f5692c82d2cd7554f5ec8fd961548d4fd09e7d22f5b62482e5aeaebd4b0", [:make, :mix, :rebar3], [], "hexpm", "bdb0d2471f453c88ff3908e7686f86f9be327d065cc1ec16fa4540197ea04680"}, "tesla": {:hex, :tesla, "1.4.4", "bb89aa0c9745190930366f6a2ac612cdf2d0e4d7fff449861baa7875afd797b2", [:mix], [{:castore, "~> 0.1", [hex: :castore, repo: "hexpm", optional: true]}, {:exjsx, ">= 3.0.0", [hex: :exjsx, repo: "hexpm", optional: true]}, {:finch, "~> 0.3", [hex: :finch, repo: "hexpm", optional: true]}, {:fuse, "~> 2.4", [hex: :fuse, repo: "hexpm", optional: true]}, {:gun, "~> 1.3", [hex: :gun, repo: "hexpm", optional: true]}, {:hackney, "~> 1.6", [hex: :hackney, repo: "hexpm", optional: true]}, {:ibrowse, "4.4.0", [hex: :ibrowse, repo: "hexpm", optional: true]}, {:jason, ">= 1.0.0", [hex: :jason, repo: "hexpm", optional: true]}, {:mime, "~> 1.0 or ~> 2.0", [hex: :mime, repo: "hexpm", optional: false]}, {:mint, "~> 1.0", [hex: :mint, repo: "hexpm", optional: true]}, {:poison, ">= 1.0.0", [hex: :poison, repo: "hexpm", optional: true]}, {:telemetry, "~> 0.4 or ~> 1.0", [hex: :telemetry, repo: "hexpm", optional: true]}], "hexpm", "d5503a49f9dec1b287567ea8712d085947e247cb11b06bc54adb05bfde466457"}, "unicode_util_compat": {:hex, :unicode_util_compat, "0.7.0", "bc84380c9ab48177092f43ac89e4dfa2c6d62b40b8bd132b1059ecc7232f9a78", [:rebar3], [], "hexpm", "25eee6d67df61960cf6a794239566599b09e17e668d3700247bc498638152521"}, -- GitLab From cc1ec64725cacbb07e4e608348d31c0ed6ad67a8 Mon Sep 17 00:00:00 2001 From: Adam Wight <adam.wight@wikimedia.de> Date: Tue, 20 Sep 2022 23:50:48 +0200 Subject: [PATCH 19/40] minor inlining --- lib/action.ex | 9 +-------- 1 file changed, 1 insertion(+), 8 deletions(-) diff --git a/lib/action.ex b/lib/action.ex index 92fa760..ae04153 100644 --- a/lib/action.ex +++ b/lib/action.ex @@ -482,14 +482,7 @@ defmodule Wiki.Action do adapter = opts[:adapter] || @default_adapter user_agent = opts[:user_agent] || Util.default_user_agent() - extra = - if opts[:accumulate] do - [Wiki.StatefulClient.CumulativeResult] - else - [] - end - - (extra ++ + (if(opts[:accumulate], do: [Wiki.StatefulClient.CumulativeResult], else: []) ++ [ {Tesla.Middleware.BaseUrl, url}, {Tesla.Middleware.Compression, format: "gzip"}, -- GitLab From 3b7d503ae5152b05ba17fc26c486d0cb4f15e501 Mon Sep 17 00:00:00 2001 From: Adam Wight <adam.wight@wikimedia.de> Date: Wed, 21 Sep 2022 00:08:32 +0200 Subject: [PATCH 20/40] Clean up test reuse --- test/action_test.exs | 2 +- test/event_streams_test.exs | 25 +++++++++++++------------ test/ores_test.exs | 19 ++++++++++++------- 3 files changed, 26 insertions(+), 20 deletions(-) diff --git a/test/action_test.exs b/test/action_test.exs index 99832cd..6c0bc42 100644 --- a/test/action_test.exs +++ b/test/action_test.exs @@ -15,7 +15,7 @@ defmodule ActionTest do defp create_session(opts \\ []) do site = opts[:site] || @url - opts = Keyword.put_new(opts, :adapter, TeslaAdapterMock) + opts = Keyword.put(opts, :adapter, TeslaAdapterMock) Action.new(site, opts) end diff --git a/test/event_streams_test.exs b/test/event_streams_test.exs index ebee4e7..e4f027d 100644 --- a/test/event_streams_test.exs +++ b/test/event_streams_test.exs @@ -9,6 +9,17 @@ defmodule EventStreamsTest do setup :set_mox_from_context setup :verify_on_exit! + @spec start_stream(keyword()) :: any() + defp start_stream(opts) do + opts = + Keyword.merge(opts, + adapter: HTTPoisonMock, + endpoint: "https://stream.test/v2/stream/" + ) + + start_supervised({EventStreams, opts}) + end + test "follows events" do HTTPoisonMock |> expect(:get!, fn url, headers, options -> @@ -29,12 +40,7 @@ defmodule EventStreamsTest do |> Enum.map(&send(target, %{chunk: &1})) end) - start_supervised( - {EventStreams, - adapter: HTTPoisonMock, - endpoint: "https://stream.test/v2/stream/", - streams: ~w(revision-create revision-score)} - ) + start_stream(streams: ~w(revision-create revision-score)) result = EventStreams.stream() @@ -53,12 +59,7 @@ defmodule EventStreamsTest do assert url == "https://stream.test/v2/stream/revision-create" end) - start_supervised( - {EventStreams, - adapter: HTTPoisonMock, - endpoint: "https://stream.test/v2/stream/", - streams: "revision-create"} - ) + start_stream(streams: "revision-create") EventStreams.stream() |> Stream.take(1) diff --git a/test/ores_test.exs b/test/ores_test.exs index 565bcca..f8e29e0 100644 --- a/test/ores_test.exs +++ b/test/ores_test.exs @@ -10,6 +10,11 @@ defmodule OresTest do setup :verify_on_exit! + defp create_session(opts \\ []) do + opts = Keyword.put(opts, :adapter, TeslaAdapterMock) + Ores.new("testwiki", opts) + end + test "requests singular resource" do TeslaAdapterMock |> expect(:call, fn env, _opts -> @@ -45,7 +50,7 @@ defmodule OresTest do end) session = - Ores.new("testwiki", adapter: TeslaAdapterMock, endpoint: "https://ores.test/v3/scores/") + create_session(endpoint: "https://ores.test/v3/scores/") |> Ores.request!(%{ models: "damaging", revids: 12_345 @@ -122,7 +127,7 @@ defmodule OresTest do end) session = - Ores.new("testwiki", adapter: TeslaAdapterMock) + create_session() |> Ores.request!(%{ models: ~w(damaging wp10), revids: [12_345, 67_890] @@ -138,7 +143,7 @@ defmodule OresTest do end) assert_raise Error, ":econnrefused", fn -> - Ores.new("testwiki", adapter: TeslaAdapterMock) + create_session() |> Ores.request!(%{}) end end @@ -150,7 +155,7 @@ defmodule OresTest do end) assert_raise Error, "Empty response", fn -> - Ores.new("testwiki", adapter: TeslaAdapterMock) + create_session() |> Ores.request!(%{}) end end @@ -164,7 +169,7 @@ defmodule OresTest do # FIXME: should report malformed, not empty assert_raise Error, "Empty response", fn -> - Ores.new("testwiki", adapter: TeslaAdapterMock) + create_session() |> Ores.request!(%{}) end end @@ -176,7 +181,7 @@ defmodule OresTest do end) assert_raise Error, "Error received with HTTP status 500", fn -> - Ores.new("testwiki", adapter: TeslaAdapterMock) + create_session() |> Ores.request!(%{}) end end @@ -195,7 +200,7 @@ defmodule OresTest do end) assert_raise Error, "No scorers available for zenwiki", fn -> - Ores.new("zenwiki", adapter: TeslaAdapterMock) + create_session() |> Ores.request!(%{}) end end -- GitLab From 721136cb5e3956f7dde03494f961d63e2b54bdc5 Mon Sep 17 00:00:00 2001 From: Adam Wight <adam.wight@wikimedia.de> Date: Wed, 21 Sep 2022 00:11:24 +0200 Subject: [PATCH 21/40] Update docs mentioning git hooks --- README.md | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-) diff --git a/README.md b/README.md index d508a8f..7e3069a 100644 --- a/README.md +++ b/README.md @@ -81,7 +81,10 @@ the `Wiki.Action.get!()` method raises an error directly. Find the [project homepage](https://gitlab.com/adamwight/mediawiki_client_ex) on GitLab. To contribute, feel free to write an issue, push a merge request, or contact the author. -Several linters are configured, these are called on `git push`. To skip tests call `git push --no-verify`, but please don't do this on the main branch. +To run all tests, +```shell script +mix test.all +``` To generate a test coverage report, ```shell script -- GitLab From 4b3a329cf8d298b647942339d4d7b9da5c96368f Mon Sep 17 00:00:00 2001 From: Adam Wight <adam.wight@wikimedia.de> Date: Wed, 21 Sep 2022 00:31:53 +0200 Subject: [PATCH 22/40] Minor genserver cleanup Still much I don't understand --- lib/event_streams.ex | 14 ++++---------- test/event_streams_test.exs | 2 +- 2 files changed, 5 insertions(+), 11 deletions(-) diff --git a/lib/event_streams.ex b/lib/event_streams.ex index 6fed5bf..960794f 100644 --- a/lib/event_streams.ex +++ b/lib/event_streams.ex @@ -151,8 +151,6 @@ defmodule Wiki.EventStreams do defmodule RelaySupervisor do @moduledoc false - use Supervisor, restart: :permanent - # TODO: Should this logic be moved to init/1? @doc """ ## Arguments @@ -167,12 +165,6 @@ defmodule Wiki.EventStreams do {:ok, _} = Supervisor.start_link(children, strategy: :one_for_one) end - - @impl true - def init(args) do - # TODO: I don't know why this is necessary or what it's doing. - {:ok, args} - end end @type options :: [option] @@ -183,12 +175,14 @@ defmodule Wiki.EventStreams do | {:send_to, GenServer.server()} | {:streams, atom | [atom]} + # TODO: Is only run as a genserver from tests, so this should go away or be + # replaced by a supervisor. use GenServer @impl true @spec init(term) :: {:ok, any} - def init(args) do - {:ok, args} + def init(_) do + {:ok, {}} end @doc """ diff --git a/test/event_streams_test.exs b/test/event_streams_test.exs index e4f027d..520796f 100644 --- a/test/event_streams_test.exs +++ b/test/event_streams_test.exs @@ -17,7 +17,7 @@ defmodule EventStreamsTest do endpoint: "https://stream.test/v2/stream/" ) - start_supervised({EventStreams, opts}) + start_supervised!({EventStreams, opts}) end test "follows events" do -- GitLab From 47344ba2ae9b14804bec3d5d06be0230d14bc969 Mon Sep 17 00:00:00 2001 From: Adam Wight <adam.wight@wikimedia.de> Date: Wed, 21 Sep 2022 00:42:29 +0200 Subject: [PATCH 23/40] Redundant mix task was preventing coverage report --- mix.exs | 1 - 1 file changed, 1 deletion(-) diff --git a/mix.exs b/mix.exs index e7827af..114681f 100644 --- a/mix.exs +++ b/mix.exs @@ -36,7 +36,6 @@ defmodule Elixir.MixProject do "clean", "compile --warnings-as-errors", "credo --strict", - "test", "coveralls.html", "dialyzer", "doctor" -- GitLab From e07074c21d74940df57f5768eaa090838ac19318 Mon Sep 17 00:00:00 2001 From: Adam Wight <adam.wight@wikimedia.de> Date: Wed, 21 Sep 2022 09:00:17 +0200 Subject: [PATCH 24/40] Start 0.4.1 --- mix.exs | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/mix.exs b/mix.exs index 114681f..53f5f64 100644 --- a/mix.exs +++ b/mix.exs @@ -4,7 +4,7 @@ defmodule Elixir.MixProject do def project do [ app: :mediawiki_client, - version: "0.4.0", + version: "0.4.1", elixir: "~> 1.9", elixirc_paths: ~w(lib), start_permanent: Mix.env() == :prod, -- GitLab From b7a6c95f30a107296e77fccc83d4dac26534624b Mon Sep 17 00:00:00 2001 From: Adam Wight <adam.wight@wikimedia.de> Date: Wed, 21 Sep 2022 08:26:48 +0200 Subject: [PATCH 25/40] Fix doc code block syntax name --- README.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/README.md b/README.md index 7e3069a..33689f6 100644 --- a/README.md +++ b/README.md @@ -82,11 +82,11 @@ Find the [project homepage](https://gitlab.com/adamwight/mediawiki_client_ex) on To contribute, feel free to write an issue, push a merge request, or contact the author. To run all tests, -```shell script +```shell mix test.all ``` To generate a test coverage report, -```shell script +```shell mix coveralls.html ``` -- GitLab From bb097e894d62e999b24a5e98b7a8af9fb11147ab Mon Sep 17 00:00:00 2001 From: Adam Wight <adam.wight@wikimedia.de> Date: Wed, 21 Sep 2022 08:58:47 +0200 Subject: [PATCH 26/40] Explicit types for client options, document all keywords --- lib/action.ex | 25 +++++++++++++++++++------ lib/ores.ex | 42 ++++++++++++++++++++++++++---------------- 2 files changed, 45 insertions(+), 22 deletions(-) diff --git a/lib/action.ex b/lib/action.ex index ae04153..22846f9 100644 --- a/lib/action.ex +++ b/lib/action.ex @@ -188,6 +188,21 @@ defmodule Wiki.Action do alias Wiki.{Action.Session, Error, SiteMatrix, Util} + @type client_option :: + {:accumulate, true} + | {:adapter, module()} + | {:debug, true} + | {:user_agent, binary()} + + @typedoc """ + - `:accumulate` - Merge results from each step of a pipeline, rather than + overwriting with the latest response. + - `:adapter` - Override the HTTP adapter + - `:debug` - Turn on verbose logging by setting to `true` + - `:user_agent` - Override the user-agent header string + """ + @type client_options :: [client_option()] + @default_adapter Tesla.Adapter.Hackney @doc """ @@ -197,10 +212,9 @@ defmodule Wiki.Action do - `site` - SiteMatrix.Spec or raw `api.php` endpoint for the wiki you will connect to. For example, "https://en.wikipedia.org/w/api.php". - - `opts` - - `:accumulate` - Merge results from each step of a pipeline, rather than overwriting with the latest response. + - `opts` - configuration options which modify client behavior """ - @spec new(String.t() | SiteMatrix.Spec.t(), keyword) :: Session.t() + @spec new(String.t() | SiteMatrix.Spec.t(), client_options()) :: Session.t() def new(site, opts \\ []) def new(%SiteMatrix.Spec{} = site, opts) do @@ -494,9 +508,8 @@ defmodule Wiki.Action do ]}, Tesla.Middleware.FollowRedirects, Tesla.Middleware.JSON - # Debugging only: - # Tesla.Middleware.Logger - ]) + ] ++ + if(opts[:debug], do: [Tesla.Middleware.Logger], else: [])) |> Tesla.client(adapter) end end diff --git a/lib/ores.ex b/lib/ores.ex index a97114f..5e7f780 100644 --- a/lib/ores.ex +++ b/lib/ores.ex @@ -36,6 +36,18 @@ defmodule Wiki.Ores do alias Wiki.{Error, Util} + @type client_option :: + {:adapter, module()} + | {:debug, true} + | {:endpoint, binary()} + | {:user_agent, binary()} + @typedoc """ + - `:adapter` - Override the HTTP adapter + - `:debug` - Turn on verbose logging by setting to `true` + - `:endpoint` - Override the base URL to query + - `:user_agent` - Override the user-agent header string + """ + @type client_options :: [client_option()] @default_adapter Tesla.Adapter.Hackney @default_endpoint "https://ores.wikimedia.org/v3/scores/" @@ -50,14 +62,13 @@ defmodule Wiki.Ores do ## Arguments - `project` - Short code for the wiki where your articles appear. For example, "enwiki" for English Wikipedia. - - `opts` - Keyword options - - `:endpoint` - Override the base URL to query + - `opts` - Configuration options that can change client behavior ## Return value Returns an opaque client object, which should be passed to `request/2`. """ - @spec new(String.t(), keyword()) :: Tesla.Client.t() + @spec new(String.t(), client_options()) :: Tesla.Client.t() def new(project, opts \\ []) do endpoint = opts[:endpoint] || @default_endpoint url = endpoint <> project <> "/" @@ -150,23 +161,22 @@ defmodule Wiki.Ores do "unknown" end - @spec client(binary(), keyword()) :: Tesla.Client.t() + @spec client(binary(), client_options()) :: Tesla.Client.t() defp client(url, opts) do adapter = opts[:adapter] || @default_adapter user_agent = opts[:user_agent] || Util.default_user_agent() - [ - {Tesla.Middleware.BaseUrl, url}, - {Tesla.Middleware.Compression, format: "gzip"}, - {Tesla.Middleware.Headers, - [ - {"user-agent", user_agent} - ]}, - Tesla.Middleware.FollowRedirects, - Tesla.Middleware.JSON - # Debugging only: - # Tesla.Middleware.Logger - ] + ([ + {Tesla.Middleware.BaseUrl, url}, + {Tesla.Middleware.Compression, format: "gzip"}, + {Tesla.Middleware.Headers, + [ + {"user-agent", user_agent} + ]}, + Tesla.Middleware.FollowRedirects, + Tesla.Middleware.JSON + ] ++ + if(opts[:debug], do: [Tesla.Middleware.Logger], else: [])) |> Tesla.client(adapter) end end -- GitLab From 06a843d6cc7228871f53a0ba3e7e059f95e8b0a0 Mon Sep 17 00:00:00 2001 From: Adam Wight <adam.wight@wikimedia.de> Date: Wed, 21 Sep 2022 08:59:15 +0200 Subject: [PATCH 27/40] Drop unused keyword options to action methods Whatever this was about can probably be handled better by passing options to the factory. --- lib/action.ex | 30 ++++++++++++++---------------- 1 file changed, 14 insertions(+), 16 deletions(-) diff --git a/lib/action.ex b/lib/action.ex index 22846f9..b47b113 100644 --- a/lib/action.ex +++ b/lib/action.ex @@ -279,22 +279,21 @@ defmodule Wiki.Action do - `session` - `Wiki.Action.Session` object. - `params` - Keyword list of query parameters as atoms or strings. - - `opts` - Options to pass to the adapter. ## Return value Session object with its `.result` populated. """ - @spec get(Session.t(), keyword, keyword) :: Session.result() - def get(session, params, opts \\ []), - do: request(session, :get, opts ++ [query: normalize_params(params)]) + @spec get(Session.t(), keyword) :: Session.result() + def get(session, params), + do: request(session, :get, query: normalize_params(params)) @doc """ Assertive variant of `get`. """ - @spec get!(Session.t(), keyword, keyword) :: Session.t() - def get!(session, params, opts \\ []) do - case get(session, params, opts) do + @spec get!(Session.t(), keyword) :: Session.t() + def get!(session, params) do + case get(session, params) do {:ok, result} -> result {:error, error} -> raise error end @@ -308,22 +307,21 @@ defmodule Wiki.Action do - `session` - `Wiki.Action.Session` object. If credentials are required for this action, you should have created this object with the `authenticate/3` function. - `params` - Keyword list of query parameters as atoms or strings. - - `opts` - Options to pass to the adapter. ## Return value Session object with a populated `:result` attribute. """ - @spec post(Session.t(), keyword, keyword) :: Session.result() - def post(session, params, opts \\ []), - do: request(session, :post, opts ++ [body: normalize_params(params)]) + @spec post(Session.t(), keyword) :: Session.result() + def post(session, params), + do: request(session, :post, body: normalize_params(params)) @doc """ Assertive variant of `post`. """ - @spec post!(Session.t(), keyword, keyword) :: Session.t() - def post!(session, params, opts \\ []) do - case post(session, params, opts) do + @spec post!(Session.t(), keyword) :: Session.t() + def post!(session, params) do + case post(session, params) do {:ok, result} -> result {:error, error} -> raise error end @@ -385,9 +383,9 @@ defmodule Wiki.Action do end @spec request(Session.t(), :get | :post, keyword) :: Session.result() - defp request(session, method, opts) do + defp request(session, method, params) do # TODO: This can be extracted into a generic StatefulAdapter now. - opts = [opts: session.state] ++ opts ++ [method: method] + opts = [opts: session.state] ++ params ++ [method: method] with {:ok, result} <- Tesla.request(session.__client__, opts), {:ok, result} <- validate(result) do -- GitLab From 50189eb83ad0958fff6e98e261b7cb0607fe8606 Mon Sep 17 00:00:00 2001 From: Adam Wight <adam.wight@wikimedia.de> Date: Wed, 21 Sep 2022 09:00:32 +0200 Subject: [PATCH 28/40] Make "last 0.x" version more obvious --- CHANGELOG.md | 4 +--- 1 file changed, 1 insertion(+), 3 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index bb06edc..aee8d3e 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -14,14 +14,12 @@ and 7zip bindings for Erlang or Elixir. * meter how many calls are made, when, and bandwidth used. * Wikidata Query Service -## 0.6.0-TODO +## 0.99.0-TODO Holding place for everything needed to finish the 0.x series. The focus is on a few robust, core apis. What it should already include: * Detect server and network errors, fail fast. Show helpful API debugging in dev environment. Demonstrate how to call with error handling. * Longer, configurable default timeouts to match servers. (#18) -* Use atoms for selecting known server-side event streams. Similarly, for -some of the action API? * Convenient logging—a global setting to inject the logging middleware into all client plugs. * ... -- GitLab From 1beaef6adec28f8b50cc755c72e7431aa5b000ee Mon Sep 17 00:00:00 2001 From: Adam Wight <adam.wight@wikimedia.de> Date: Wed, 21 Sep 2022 09:01:24 +0200 Subject: [PATCH 29/40] Coverage knob --- coveralls.json | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/coveralls.json b/coveralls.json index 4f9e7da..a33b0a5 100644 --- a/coveralls.json +++ b/coveralls.json @@ -1,5 +1,5 @@ { "coverage_options": { - "minimum_coverage": 93 + "minimum_coverage": 94 } } -- GitLab From 41f11d9fa703692c03b01871a033bb2ac83dcb4e Mon Sep 17 00:00:00 2001 From: Adam Wight <adam.wight@wikimedia.de> Date: Wed, 21 Sep 2022 09:20:46 +0200 Subject: [PATCH 30/40] Document more keyword options --- CHANGELOG.md | 1 + lib/event_streams.ex | 36 +++++++++++++++++++++--------------- lib/site_matrix.ex | 8 +++++--- 3 files changed, 27 insertions(+), 18 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index aee8d3e..27f0732 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -36,6 +36,7 @@ Small changes in progress... ## 0.4.0 (Sep 2022) Breaking changes: * Config variables are deprecated, all customization is done via keyword options. +* Wiki.EventStreams `:send_to` option renamed to `:stream_to`. Other: * `mix test.all` task replaces git_hooks for development. diff --git a/lib/event_streams.ex b/lib/event_streams.ex index 960794f..ecad3b6 100644 --- a/lib/event_streams.ex +++ b/lib/event_streams.ex @@ -46,9 +46,9 @@ defmodule Wiki.EventStreams do @doc """ ## Arguments - * {:producer, state} + * Client configuration. `:streams` is required. """ - @spec start_link(keyword) :: GenServer.on_start() + @spec start_link(Wiki.EventStreams.client_options()) :: GenServer.on_start() def start_link(opts) do {:ok, relay_pid} = GenStage.start_link(__MODULE__, opts, name: __MODULE__) # FIXME: Rationalize the supervision tree. @@ -108,12 +108,12 @@ defmodule Wiki.EventStreams do defmodule Source do @moduledoc false - alias Wiki.Util + alias Wiki.{EventStreams, Util} @default_endpoint "https://stream.wikimedia.org/v2/stream/" @doc false - @spec child_spec(Keyword.t()) :: map + @spec child_spec(EventStreams.client_options()) :: map def child_spec(opts \\ []) do adapter = opts[:adapter] endpoint = opts[:endpoint] || @default_endpoint @@ -155,11 +155,9 @@ defmodule Wiki.EventStreams do @doc """ ## Arguments - * `endpoint` - Optionally override the default endpoint URL. - * `streams` - One or more atoms with the stream names to subscribe to. - * `send_to` - Optional application which will receive the events. + * `opts` - Client keyword options """ - @spec start_link(keyword) :: GenServer.on_start() + @spec start_link(Wiki.EventStreams.client_options()) :: GenServer.on_start() def start_link(opts) do children = [{Relay, opts}] @@ -167,13 +165,21 @@ defmodule Wiki.EventStreams do end end - @type options :: [option] - - @type option :: + @type client_option :: {:adapter, module()} - | {:endpoint, String.t()} - | {:send_to, GenServer.server()} + | {:endpoint, binary()} + | {:stream_to, GenServer.server()} | {:streams, atom | [atom]} + | {:user_agent, binary()} + @typedoc """ + * `:adapter` - Override the HTTP adapter. + * `:endpoint` - Override the default endpoint URL. + * `:stream_to` - Optional application which will receive the events, otherwise + they go to the process starting the EventStream. + * `:streams` - One or more atoms with the stream names to subscribe to. + * `:user_agent` - Custom user-agent header string + """ + @type client_options :: [client_option()] # TODO: Is only run as a genserver from tests, so this should go away or be # replaced by a supervisor. @@ -198,7 +204,7 @@ defmodule Wiki.EventStreams do - `{:streams, atom | [atom]}` - Select which streams to listen to. An updated list can be [found here](https://stream.wikimedia.org/?doc#/Streams). Required. """ - @spec start_link(options) :: GenServer.on_start() + @spec start_link(client_options()) :: GenServer.on_start() def start_link(args) do RelaySupervisor.start_link(args) end @@ -206,7 +212,7 @@ defmodule Wiki.EventStreams do @doc """ Capture subscribed events and relay them as a `Stream`. """ - @spec stream(keyword) :: Enumerable.t() + @spec stream(client_options()) :: Enumerable.t() def stream(options \\ []) do GenStage.stream([Relay], options) end diff --git a/lib/site_matrix.ex b/lib/site_matrix.ex index 9b87d62..e3894f2 100644 --- a/lib/site_matrix.ex +++ b/lib/site_matrix.ex @@ -34,6 +34,8 @@ defmodule Wiki.SiteMatrix do ] end + @type client_options :: [Wiki.Action.client_option() | {:api, binary()}] + @doc """ ## Options @@ -45,7 +47,7 @@ defmodule Wiki.SiteMatrix do sites: %{binary() => Spec.t()} } - @spec new(keyword()) :: sitematrix_state() + @spec new(client_options()) :: sitematrix_state() def new(opts \\ []) do api = opts[:api] || @metawiki_api @@ -71,13 +73,13 @@ defmodule Wiki.SiteMatrix do {:ok, Map.values(sitematrix.sites)} end - @spec action_client(binary(), keyword()) :: Wiki.Action.Session.t() + @spec action_client(binary(), client_options()) :: Wiki.Action.Session.t() defp action_client(api, opts) do api |> Wiki.Action.new(opts) end - @spec do_get_all(binary(), keyword()) :: map() + @spec do_get_all(binary(), client_options()) :: map() defp do_get_all(api, opts) do action_client(api, opts) |> fetch_sitematrix() -- GitLab From 048ff30be1ecd41b7622dcf9e9f11df5311ef876 Mon Sep 17 00:00:00 2001 From: Adam Wight <adam.wight@wikimedia.de> Date: Wed, 21 Sep 2022 22:25:16 +0200 Subject: [PATCH 31/40] Doc fixups --- README.md | 10 ++++++---- lib/action.ex | 23 ++--------------------- lib/event_streams.ex | 2 +- 3 files changed, 9 insertions(+), 26 deletions(-) diff --git a/README.md b/README.md index 33689f6..39699e3 100644 --- a/README.md +++ b/README.md @@ -68,7 +68,8 @@ Wiki.SiteMatrix.new() # } ``` -See the module documentation for more examples. +See the [source documentation](https://hexdocs.pm/mediawiki_client/api-reference.html) +for more examples. ### Error handling @@ -78,15 +79,16 @@ the `Wiki.Action.get!()` method raises an error directly. ## Development -Find the [project homepage](https://gitlab.com/adamwight/mediawiki_client_ex) on GitLab. -To contribute, feel free to write an issue, push a merge request, or contact the author. +The [project homepage](https://gitlab.com/adamwight/mediawiki_client_ex) is on GitLab. +To contribute, feel free to start with an issue, push a merge request, or +contact maintainers. To run all tests, ```shell mix test.all ``` -To generate a test coverage report, +To generate a test [coverage report](../cover/excoveralls.html), ```shell mix coveralls.html ``` diff --git a/lib/action.ex b/lib/action.ex index b47b113..1b352f0 100644 --- a/lib/action.ex +++ b/lib/action.ex @@ -41,20 +41,7 @@ defmodule Wiki.Action do siprop: :statistics ) # %Wiki.Action.Session{ - # __client__: %Tesla.Client{ - # adapter: {Tesla.Adapter.Hackney, :call, [[]]}, - # fun: nil, - # post: [], - # pre: [ - # {Tesla.Middleware.BaseUrl, :call, ["https://de.wikipedia.org/w/api.php"]}, - # {Tesla.Middleware.Compression, :call, [[format: "gzip"]]}, - # {Wiki.StatefulClient.CookieJar, :call, [[]]}, - # {Tesla.Middleware.FormUrlencoded, :call, [[]]}, - # {Tesla.Middleware.Headers, :call, - # [[{"user-agent", "mediawiki_client_ex/0.2.6 (spam@ludd.net)"}]]}, - # {Tesla.Middleware.JSON, :call, [[]]} - # ] - # }, + # ... # result: %{ # "batchcomplete" => true, # "query" => %{ @@ -71,13 +58,7 @@ defmodule Wiki.Action do # } # } # }, - # state: [ - # cookies: %{ - # "GeoIP" => "DE:BE:Berlin:52.57:13.42:v4", - # "WMF-Last-Access" => "04-Jun-2021", - # "WMF-Last-Access-Global" => "04-Jun-2021" - # } - # ] + # ... # } ``` diff --git a/lib/event_streams.ex b/lib/event_streams.ex index ecad3b6..3d4fcf4 100644 --- a/lib/event_streams.ex +++ b/lib/event_streams.ex @@ -210,7 +210,7 @@ defmodule Wiki.EventStreams do end @doc """ - Capture subscribed events and relay them as a `Stream`. + Indefinitely capture subscribed events and relay them as a `Stream`. """ @spec stream(client_options()) :: Enumerable.t() def stream(options \\ []) do -- GitLab From c6e5fa3f02f23f077cb5e033135eeaf086713699 Mon Sep 17 00:00:00 2001 From: Adam Wight <adam.wight@wikimedia.de> Date: Wed, 21 Sep 2022 22:26:03 +0200 Subject: [PATCH 32/40] Cleanup, inlining --- lib/event_streams.ex | 6 ++---- 1 file changed, 2 insertions(+), 4 deletions(-) diff --git a/lib/event_streams.ex b/lib/event_streams.ex index 3d4fcf4..cf606b7 100644 --- a/lib/event_streams.ex +++ b/lib/event_streams.ex @@ -145,7 +145,7 @@ defmodule Wiki.EventStreams do defp normalize_streams(streams) when is_list(streams), do: Enum.join(streams, ",") - defp normalize_streams(streams), do: streams + defp normalize_streams(streams) when is_atom(streams) or is_binary(streams), do: streams end defmodule RelaySupervisor do @@ -159,9 +159,7 @@ defmodule Wiki.EventStreams do """ @spec start_link(Wiki.EventStreams.client_options()) :: GenServer.on_start() def start_link(opts) do - children = [{Relay, opts}] - - {:ok, _} = Supervisor.start_link(children, strategy: :one_for_one) + {:ok, _} = Supervisor.start_link([{Relay, opts}], strategy: :one_for_one) end end -- GitLab From 0f79618377a1bc807f53869a248bf89ea5c3a80b Mon Sep 17 00:00:00 2001 From: Adam Wight <adam.wight@wikimedia.de> Date: Wed, 21 Sep 2022 22:25:42 +0200 Subject: [PATCH 33/40] helper functions to illuminate cookie logic --- lib/action.ex | 54 +++++++++++++++++++-------------------------------- 1 file changed, 20 insertions(+), 34 deletions(-) diff --git a/lib/action.ex b/lib/action.ex index 1b352f0..c98cd31 100644 --- a/lib/action.ex +++ b/lib/action.ex @@ -500,51 +500,37 @@ defmodule Wiki.StatefulClient.CookieJar do @impl true def call(env, next, _opts) do - cookie_header = - case env.opts[:cookies] do - nil -> [] - cookies -> [{"cookie", serialize_cookies(cookies)}] - end - - env = - env - |> Tesla.put_headers(cookie_header) + env = set_cookie_header(env, env.opts[:cookies]) with {:ok, env} <- Tesla.run(env, next) do - cookies = - env - |> Tesla.get_headers("set-cookie") - |> extract_cookies() - |> update_cookies(env.opts[:cookies]) + {:ok, merge_cookies(env, env.opts[:cookies])} + end + end - env = - env - |> Tesla.put_opt(:cookies, cookies) + @spec set_cookie_header(Tesla.Env.t(), nil | map) :: Tesla.Env.t() + defp set_cookie_header(env, cookies_opt) + defp set_cookie_header(env, nil), do: env - {:ok, env} - end + defp set_cookie_header(env, cookies) do + serialized = Enum.map_join(cookies, "; ", fn {key, value} -> key <> "=" <> value end) + Tesla.put_headers(env, [{"cookie", serialized}]) end - @spec update_cookies(map, map) :: map - defp update_cookies(new_cookies, old_cookies) do - case old_cookies do - nil -> new_cookies - _ -> Map.merge(old_cookies, new_cookies) - end + @spec merge_cookies(Tesla.Env.t(), nil | map) :: Tesla.Env.t() + defp merge_cookies(env, old_cookies) + defp merge_cookies(env, nil), do: env + + defp merge_cookies(env, old_cookies) do + merged_cookies = Map.merge(old_cookies, extract_cookie_headers(env)) + Tesla.put_opt(env, :cookies, merged_cookies) end - @spec extract_cookies(Keyword.t()) :: map - defp extract_cookies(headers) do - headers + @spec extract_cookie_headers(Tesla.Env.t()) :: map + defp extract_cookie_headers(env) do + Tesla.get_headers(env, "set-cookie") |> Enum.map(&SetCookie.parse/1) |> Enum.into(%{}, fn %{key: k, value: v} -> {k, v} end) end - - @spec serialize_cookies(map) :: String.t() - defp serialize_cookies(cookies) do - cookies - |> Enum.map_join("; ", fn {key, value} -> key <> "=" <> value end) - end end defmodule Wiki.StatefulClient.CumulativeResult do -- GitLab From 4eea683bc8b9fb694ba9dd101831c5a185490aa4 Mon Sep 17 00:00:00 2001 From: Adam Wight <adam.wight@wikimedia.de> Date: Tue, 20 Dec 2022 13:14:26 +0100 Subject: [PATCH 34/40] Stay compatible with tesla 1.5 Compression belonged at the end and a library update begs the question. --- lib/action.ex | 4 ++-- lib/ores.ex | 4 ++-- 2 files changed, 4 insertions(+), 4 deletions(-) diff --git a/lib/action.ex b/lib/action.ex index c98cd31..c839625 100644 --- a/lib/action.ex +++ b/lib/action.ex @@ -478,7 +478,6 @@ defmodule Wiki.Action do (if(opts[:accumulate], do: [Wiki.StatefulClient.CumulativeResult], else: []) ++ [ {Tesla.Middleware.BaseUrl, url}, - {Tesla.Middleware.Compression, format: "gzip"}, Wiki.StatefulClient.CookieJar, Tesla.Middleware.FormUrlencoded, {Tesla.Middleware.Headers, @@ -486,7 +485,8 @@ defmodule Wiki.Action do {"user-agent", user_agent} ]}, Tesla.Middleware.FollowRedirects, - Tesla.Middleware.JSON + Tesla.Middleware.JSON, + Tesla.Middleware.Compression ] ++ if(opts[:debug], do: [Tesla.Middleware.Logger], else: [])) |> Tesla.client(adapter) diff --git a/lib/ores.ex b/lib/ores.ex index 5e7f780..def3f44 100644 --- a/lib/ores.ex +++ b/lib/ores.ex @@ -168,13 +168,13 @@ defmodule Wiki.Ores do ([ {Tesla.Middleware.BaseUrl, url}, - {Tesla.Middleware.Compression, format: "gzip"}, {Tesla.Middleware.Headers, [ {"user-agent", user_agent} ]}, Tesla.Middleware.FollowRedirects, - Tesla.Middleware.JSON + Tesla.Middleware.JSON, + Tesla.Middleware.Compression ] ++ if(opts[:debug], do: [Tesla.Middleware.Logger], else: [])) |> Tesla.client(adapter) -- GitLab From e77e6fa1058862dbc6297ad6db48e499d42c897e Mon Sep 17 00:00:00 2001 From: Adam Wight <adam.wight@wikimedia.de> Date: Tue, 20 Dec 2022 13:14:26 +0100 Subject: [PATCH 35/40] Actually compress Compression belonged at the end and a library update begged the question by adding necessary headers to enable compression. --- test/action_test.exs | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/test/action_test.exs b/test/action_test.exs index 6c0bc42..c468a64 100644 --- a/test/action_test.exs +++ b/test/action_test.exs @@ -183,7 +183,7 @@ defmodule ActionTest do assert env.method == :post assert env.query == [] - assert env.body == + assert :zlib.gunzip(env.body) == "action=login&format=json&formatversion=2&lgname=TestUser%40bot&lgpassword=botpass&lgtoken=5c31497c51b4b28f2d6c19f3349070d25eccae52%2B%5C" assert List.keyfind(env.headers, "cookie", 0) == -- GitLab From a1978cc123f066feab001b77c4198508ba72fb33 Mon Sep 17 00:00:00 2001 From: Adam Wight <adam.wight@wikimedia.de> Date: Tue, 20 Dec 2022 14:37:01 +0100 Subject: [PATCH 36/40] Update changelog --- CHANGELOG.md | 8 ++++++-- README.md | 2 +- 2 files changed, 7 insertions(+), 3 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 27f0732..2ac826d 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -29,8 +29,12 @@ This is the development target and next steps. * Built-in Mediawiki [REST API](https://www.mediawiki.org/wiki/API:REST_API) * ... -## 0.4.1-TODO -Small changes in progress... +## 0.4.2-TODO +* ... + +## 0.4.1 (Dec 2022) +* Now compatible with tesla ~>1.5 +* Fix response compression * ... ## 0.4.0 (Sep 2022) diff --git a/README.md b/README.md index 39699e3..df84bba 100644 --- a/README.md +++ b/README.md @@ -21,7 +21,7 @@ Install this package by adding `mediawiki_client` to your dependencies in `mix.e ```elixir def deps do [ - {:mediawiki_client, "~> 0.4"} + {:mediawiki_client, "~> 0.4.0"} ] end ``` -- GitLab From 8193e749b73b8d7cc61fb93e16a366ff6c1e88d2 Mon Sep 17 00:00:00 2001 From: Adam Wight <adam.wight@wikimedia.de> Date: Tue, 20 Dec 2022 14:37:53 +0100 Subject: [PATCH 37/40] Bump version to 0.4.2-dev --- mix.exs | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/mix.exs b/mix.exs index 53f5f64..1776040 100644 --- a/mix.exs +++ b/mix.exs @@ -4,7 +4,7 @@ defmodule Elixir.MixProject do def project do [ app: :mediawiki_client, - version: "0.4.1", + version: "0.4.2", elixir: "~> 1.9", elixirc_paths: ~w(lib), start_permanent: Mix.env() == :prod, -- GitLab From afe9aebf6c2f375fded7c4847c64620732ed12a0 Mon Sep 17 00:00:00 2001 From: Adam Wight <adam.wight@wikimedia.de> Date: Wed, 21 Dec 2022 11:33:09 +0100 Subject: [PATCH 38/40] Fix tests for tesla 1.5 --- CHANGELOG.md | 3 +-- mix.lock | 14 +++++++------- test/action_test.exs | 6 ++---- test/ores_test.exs | 3 +-- 4 files changed, 11 insertions(+), 15 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 2ac826d..1e42239 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -34,8 +34,7 @@ This is the development target and next steps. ## 0.4.1 (Dec 2022) * Now compatible with tesla ~>1.5 -* Fix response compression -* ... +* Fix compression for Action and ORES APIs ## 0.4.0 (Sep 2022) Breaking changes: diff --git a/mix.lock b/mix.lock index ae5c393..0b06488 100644 --- a/mix.lock +++ b/mix.lock @@ -5,17 +5,17 @@ "credo": {:hex, :credo, "1.6.7", "323f5734350fd23a456f2688b9430e7d517afb313fbd38671b8a4449798a7854", [:mix], [{:bunt, "~> 0.2.1", [hex: :bunt, repo: "hexpm", optional: false]}, {:file_system, "~> 0.2.8", [hex: :file_system, repo: "hexpm", optional: false]}, {:jason, "~> 1.0", [hex: :jason, repo: "hexpm", optional: false]}], "hexpm", "41e110bfb007f7eda7f897c10bf019ceab9a0b269ce79f015d54b0dcf4fc7dd3"}, "decimal": {:hex, :decimal, "2.0.0", "a78296e617b0f5dd4c6caf57c714431347912ffb1d0842e998e9792b5642d697", [:mix], [], "hexpm", "34666e9c55dea81013e77d9d87370fe6cb6291d1ef32f46a1600230b1d44f577"}, "dialyxir": {:hex, :dialyxir, "1.2.0", "58344b3e87c2e7095304c81a9ae65cb68b613e28340690dfe1a5597fd08dec37", [:mix], [{:erlex, ">= 0.2.6", [hex: :erlex, repo: "hexpm", optional: false]}], "hexpm", "61072136427a851674cab81762be4dbeae7679f85b1272b6d25c3a839aff8463"}, - "doctor": {:hex, :doctor, "0.19.0", "f7974836bc85756b38b99de46cc2c6ba36741f21d8eabcbef78f6806ca6769ed", [:mix], [{:decimal, "~> 2.0", [hex: :decimal, repo: "hexpm", optional: false]}], "hexpm", "504f17473dc6b39618e693c5198d85e274b056b73eb4a4605431aec0f42f0023"}, - "earmark_parser": {:hex, :earmark_parser, "1.4.26", "f4291134583f373c7d8755566122908eb9662df4c4b63caa66a0eabe06569b0a", [:mix], [], "hexpm", "48d460899f8a0c52c5470676611c01f64f3337bad0b26ddab43648428d94aabc"}, + "doctor": {:hex, :doctor, "0.21.0", "20ef89355c67778e206225fe74913e96141c4d001cb04efdeba1a2a9704f1ab5", [:mix], [{:decimal, "~> 2.0", [hex: :decimal, repo: "hexpm", optional: false]}], "hexpm", "a227831daa79784eb24cdeedfa403c46a4cb7d0eab0e31232ec654314447e4e0"}, + "earmark_parser": {:hex, :earmark_parser, "1.4.29", "149d50dcb3a93d9f3d6f3ecf18c918fb5a2d3c001b5d3305c926cddfbd33355b", [:mix], [], "hexpm", "4902af1b3eb139016aed210888748db8070b8125c2342ce3dcae4f38dcc63503"}, "erlex": {:hex, :erlex, "0.2.6", "c7987d15e899c7a2f34f5420d2a2ea0d659682c06ac607572df55a43753aa12e", [:mix], [], "hexpm", "2ed2e25711feb44d52b17d2780eabf998452f6efda104877a3881c2f8c0c0c75"}, "eventsource_ex": {:hex, :eventsource_ex, "1.1.0", "41ec298216bb6bd48d92ca8256bd730393fe9534c7c8400282a6abf795bad103", [:mix], [{:httpoison, "~> 1.5", [hex: :httpoison, repo: "hexpm", optional: false]}], "hexpm", "d11d13ab4d5845426f90c055983e89e8d70fab6b57dcc2125743be195c6953a3"}, - "ex_doc": {:hex, :ex_doc, "0.28.5", "3e52a6d2130ce74d096859e477b97080c156d0926701c13870a4e1f752363279", [:mix], [{:earmark_parser, "~> 1.4.19", [hex: :earmark_parser, repo: "hexpm", optional: false]}, {:makeup_elixir, "~> 0.14", [hex: :makeup_elixir, repo: "hexpm", optional: false]}, {:makeup_erlang, "~> 0.1", [hex: :makeup_erlang, repo: "hexpm", optional: false]}], "hexpm", "d2c4b07133113e9aa3e9ba27efb9088ba900e9e51caa383919676afdf09ab181"}, - "excoveralls": {:hex, :excoveralls, "0.14.6", "610e921e25b180a8538229ef547957f7e04bd3d3e9a55c7c5b7d24354abbba70", [:mix], [{:hackney, "~> 1.16", [hex: :hackney, repo: "hexpm", optional: false]}, {:jason, "~> 1.0", [hex: :jason, repo: "hexpm", optional: false]}], "hexpm", "0eceddaa9785cfcefbf3cd37812705f9d8ad34a758e513bb975b081dce4eb11e"}, + "ex_doc": {:hex, :ex_doc, "0.29.1", "b1c652fa5f92ee9cf15c75271168027f92039b3877094290a75abcaac82a9f77", [:mix], [{:earmark_parser, "~> 1.4.19", [hex: :earmark_parser, repo: "hexpm", optional: false]}, {:makeup_elixir, "~> 0.14", [hex: :makeup_elixir, repo: "hexpm", optional: false]}, {:makeup_erlang, "~> 0.1", [hex: :makeup_erlang, repo: "hexpm", optional: false]}], "hexpm", "b7745fa6374a36daf484e2a2012274950e084815b936b1319aeebcf7809574f6"}, + "excoveralls": {:hex, :excoveralls, "0.15.1", "83c8cf7973dd9d1d853dce37a2fb98aaf29b564bf7d01866e409abf59dac2c0e", [:mix], [{:hackney, "~> 1.16", [hex: :hackney, repo: "hexpm", optional: false]}, {:jason, "~> 1.0", [hex: :jason, repo: "hexpm", optional: false]}], "hexpm", "f8416bd90c0082d56a2178cf46c837595a06575f70a5624f164a1ffe37de07e7"}, "file_system": {:hex, :file_system, "0.2.10", "fb082005a9cd1711c05b5248710f8826b02d7d1784e7c3451f9c1231d4fc162d", [:mix], [], "hexpm", "41195edbfb562a593726eda3b3e8b103a309b733ad25f3d642ba49696bf715dc"}, "gen_stage": {:hex, :gen_stage, "1.1.2", "b1656cd4ba431ed02c5656fe10cb5423820847113a07218da68eae5d6a260c23", [:mix], [], "hexpm", "9e39af23140f704e2b07a3e29d8f05fd21c2aaf4088ff43cb82be4b9e3148d02"}, - "hackney": {:hex, :hackney, "1.18.1", "f48bf88f521f2a229fc7bae88cf4f85adc9cd9bcf23b5dc8eb6a1788c662c4f6", [:rebar3], [{:certifi, "~>2.9.0", [hex: :certifi, repo: "hexpm", optional: false]}, {:idna, "~>6.1.0", [hex: :idna, repo: "hexpm", optional: false]}, {:metrics, "~>1.0.0", [hex: :metrics, repo: "hexpm", optional: false]}, {:mimerl, "~>1.1", [hex: :mimerl, repo: "hexpm", optional: false]}, {:parse_trans, "3.3.1", [hex: :parse_trans, repo: "hexpm", optional: false]}, {:ssl_verify_fun, "~>1.1.0", [hex: :ssl_verify_fun, repo: "hexpm", optional: false]}, {:unicode_util_compat, "~>0.7.0", [hex: :unicode_util_compat, repo: "hexpm", optional: false]}], "hexpm", "a4ecdaff44297e9b5894ae499e9a070ea1888c84afdd1fd9b7b2bc384950128e"}, + "hackney": {:hex, :hackney, "1.18.1", "f48bf88f521f2a229fc7bae88cf4f85adc9cd9bcf23b5dc8eb6a1788c662c4f6", [:rebar3], [{:certifi, "~> 2.9.0", [hex: :certifi, repo: "hexpm", optional: false]}, {:idna, "~> 6.1.0", [hex: :idna, repo: "hexpm", optional: false]}, {:metrics, "~> 1.0.0", [hex: :metrics, repo: "hexpm", optional: false]}, {:mimerl, "~> 1.1", [hex: :mimerl, repo: "hexpm", optional: false]}, {:parse_trans, "3.3.1", [hex: :parse_trans, repo: "hexpm", optional: false]}, {:ssl_verify_fun, "~> 1.1.0", [hex: :ssl_verify_fun, repo: "hexpm", optional: false]}, {:unicode_util_compat, "~> 0.7.0", [hex: :unicode_util_compat, repo: "hexpm", optional: false]}], "hexpm", "a4ecdaff44297e9b5894ae499e9a070ea1888c84afdd1fd9b7b2bc384950128e"}, "httpoison": {:hex, :httpoison, "1.8.2", "9eb9c63ae289296a544842ef816a85d881d4a31f518a0fec089aaa744beae290", [:mix], [{:hackney, "~> 1.17", [hex: :hackney, repo: "hexpm", optional: false]}], "hexpm", "2bb350d26972e30c96e2ca74a1aaf8293d61d0742ff17f01e0279fef11599921"}, - "idna": {:hex, :idna, "6.1.1", "8a63070e9f7d0c62eb9d9fcb360a7de382448200fbbd1b106cc96d3d8099df8d", [:rebar3], [{:unicode_util_compat, "~>0.7.0", [hex: :unicode_util_compat, repo: "hexpm", optional: false]}], "hexpm", "92376eb7894412ed19ac475e4a86f7b413c1b9fbb5bd16dccd57934157944cea"}, + "idna": {:hex, :idna, "6.1.1", "8a63070e9f7d0c62eb9d9fcb360a7de382448200fbbd1b106cc96d3d8099df8d", [:rebar3], [{:unicode_util_compat, "~> 0.7.0", [hex: :unicode_util_compat, repo: "hexpm", optional: false]}], "hexpm", "92376eb7894412ed19ac475e4a86f7b413c1b9fbb5bd16dccd57934157944cea"}, "jason": {:hex, :jason, "1.4.0", "e855647bc964a44e2f67df589ccf49105ae039d4179db7f6271dfd3843dc27e6", [:mix], [{:decimal, "~> 1.0 or ~> 2.0", [hex: :decimal, repo: "hexpm", optional: true]}], "hexpm", "79a3791085b2a0f743ca04cec0f7be26443738779d09302e01318f97bdb82121"}, "makeup": {:hex, :makeup, "1.1.0", "6b67c8bc2882a6b6a445859952a602afc1a41c2e08379ca057c0f525366fc3ca", [:mix], [{:nimble_parsec, "~> 1.2.2 or ~> 1.3", [hex: :nimble_parsec, repo: "hexpm", optional: false]}], "hexpm", "0a45ed501f4a8897f580eabf99a2e5234ea3e75a4373c8a52824f6e873be57a6"}, "makeup_elixir": {:hex, :makeup_elixir, "0.16.0", "f8c570a0d33f8039513fbccaf7108c5d750f47d8defd44088371191b76492b0b", [:mix], [{:makeup, "~> 1.0", [hex: :makeup, repo: "hexpm", optional: false]}, {:nimble_parsec, "~> 1.2.3", [hex: :nimble_parsec, repo: "hexpm", optional: false]}], "hexpm", "28b2cbdc13960a46ae9a8858c4bebdec3c9a6d7b4b9e7f4ed1502f8159f338e7"}, @@ -27,6 +27,6 @@ "nimble_parsec": {:hex, :nimble_parsec, "1.2.3", "244836e6e3f1200c7f30cb56733fd808744eca61fd182f731eac4af635cc6d0b", [:mix], [], "hexpm", "c8d789e39b9131acf7b99291e93dae60ab48ef14a7ee9d58c6964f59efb570b0"}, "parse_trans": {:hex, :parse_trans, "3.3.1", "16328ab840cc09919bd10dab29e431da3af9e9e7e7e6f0089dd5a2d2820011d8", [:rebar3], [], "hexpm", "07cd9577885f56362d414e8c4c4e6bdf10d43a8767abb92d24cbe8b24c54888b"}, "ssl_verify_fun": {:hex, :ssl_verify_fun, "1.1.6", "cf344f5692c82d2cd7554f5ec8fd961548d4fd09e7d22f5b62482e5aeaebd4b0", [:make, :mix, :rebar3], [], "hexpm", "bdb0d2471f453c88ff3908e7686f86f9be327d065cc1ec16fa4540197ea04680"}, - "tesla": {:hex, :tesla, "1.4.4", "bb89aa0c9745190930366f6a2ac612cdf2d0e4d7fff449861baa7875afd797b2", [:mix], [{:castore, "~> 0.1", [hex: :castore, repo: "hexpm", optional: true]}, {:exjsx, ">= 3.0.0", [hex: :exjsx, repo: "hexpm", optional: true]}, {:finch, "~> 0.3", [hex: :finch, repo: "hexpm", optional: true]}, {:fuse, "~> 2.4", [hex: :fuse, repo: "hexpm", optional: true]}, {:gun, "~> 1.3", [hex: :gun, repo: "hexpm", optional: true]}, {:hackney, "~> 1.6", [hex: :hackney, repo: "hexpm", optional: true]}, {:ibrowse, "4.4.0", [hex: :ibrowse, repo: "hexpm", optional: true]}, {:jason, ">= 1.0.0", [hex: :jason, repo: "hexpm", optional: true]}, {:mime, "~> 1.0 or ~> 2.0", [hex: :mime, repo: "hexpm", optional: false]}, {:mint, "~> 1.0", [hex: :mint, repo: "hexpm", optional: true]}, {:poison, ">= 1.0.0", [hex: :poison, repo: "hexpm", optional: true]}, {:telemetry, "~> 0.4 or ~> 1.0", [hex: :telemetry, repo: "hexpm", optional: true]}], "hexpm", "d5503a49f9dec1b287567ea8712d085947e247cb11b06bc54adb05bfde466457"}, + "tesla": {:hex, :tesla, "1.5.0", "7ee3616be87024a2b7231ae14474310c9b999c3abb1f4f8dbc70f86bd9678eef", [:mix], [{:castore, "~> 0.1", [hex: :castore, repo: "hexpm", optional: true]}, {:exjsx, ">= 3.0.0", [hex: :exjsx, repo: "hexpm", optional: true]}, {:finch, "~> 0.13", [hex: :finch, repo: "hexpm", optional: true]}, {:fuse, "~> 2.4", [hex: :fuse, repo: "hexpm", optional: true]}, {:gun, "~> 1.3", [hex: :gun, repo: "hexpm", optional: true]}, {:hackney, "~> 1.6", [hex: :hackney, repo: "hexpm", optional: true]}, {:ibrowse, "4.4.0", [hex: :ibrowse, repo: "hexpm", optional: true]}, {:jason, ">= 1.0.0", [hex: :jason, repo: "hexpm", optional: true]}, {:mime, "~> 1.0 or ~> 2.0", [hex: :mime, repo: "hexpm", optional: false]}, {:mint, "~> 1.0", [hex: :mint, repo: "hexpm", optional: true]}, {:msgpax, "~> 2.3", [hex: :msgpax, repo: "hexpm", optional: true]}, {:poison, ">= 1.0.0", [hex: :poison, repo: "hexpm", optional: true]}, {:telemetry, "~> 0.4 or ~> 1.0", [hex: :telemetry, repo: "hexpm", optional: true]}], "hexpm", "1d0385e41fbd76af3961809088aef15dec4c2fdaab97b1c93c6484cb3695a122"}, "unicode_util_compat": {:hex, :unicode_util_compat, "0.7.0", "bc84380c9ab48177092f43ac89e4dfa2c6d62b40b8bd132b1059ecc7232f9a78", [:rebar3], [], "hexpm", "25eee6d67df61960cf6a794239566599b09e17e668d3700247bc498638152521"}, } diff --git a/test/action_test.exs b/test/action_test.exs index c468a64..276b4d5 100644 --- a/test/action_test.exs +++ b/test/action_test.exs @@ -57,8 +57,7 @@ defmodule ActionTest do TeslaAdapterMock |> expect(:call, fn env, _opts -> - [{"user-agent", user_agent}] = env.headers - + {_, user_agent} = List.keyfind!(env.headers, "user-agent", 0) assert String.match?(user_agent, ~r/mediawiki_client_ex.*\d.*/) assert env.method == :get @@ -374,8 +373,7 @@ defmodule ActionTest do test "can override user agent" do TeslaAdapterMock |> expect(:call, fn env, _opts -> - [{"user-agent", user_agent}] = env.headers - + {_, user_agent} = List.keyfind!(env.headers, "user-agent", 0) assert user_agent == "foo bar" {:ok, %Env{env | body: %{bar: :baz}, status: 200}} diff --git a/test/ores_test.exs b/test/ores_test.exs index f8e29e0..82fccc2 100644 --- a/test/ores_test.exs +++ b/test/ores_test.exs @@ -18,8 +18,7 @@ defmodule OresTest do test "requests singular resource" do TeslaAdapterMock |> expect(:call, fn env, _opts -> - [{"user-agent", user_agent}] = env.headers - + {_, user_agent} = List.keyfind!(env.headers, "user-agent", 0) assert String.match?(user_agent, ~r/mediawiki_client_ex.*\d.*/) assert env.query == [models: "damaging", revids: 12_345] -- GitLab From d076b4471714b39fab77d35f2053706e2ac3f2c3 Mon Sep 17 00:00:00 2001 From: Adam Wight <adam.wight@wikimedia.de> Date: Wed, 21 Dec 2022 11:53:47 +0100 Subject: [PATCH 39/40] Test fixes for elixir < 1.13 --- test/action_test.exs | 4 ++-- test/ores_test.exs | 2 +- 2 files changed, 3 insertions(+), 3 deletions(-) diff --git a/test/action_test.exs b/test/action_test.exs index 276b4d5..86d91f9 100644 --- a/test/action_test.exs +++ b/test/action_test.exs @@ -57,7 +57,7 @@ defmodule ActionTest do TeslaAdapterMock |> expect(:call, fn env, _opts -> - {_, user_agent} = List.keyfind!(env.headers, "user-agent", 0) + {{_, user_agent}, _} = List.keytake(env.headers, "user-agent", 0) assert String.match?(user_agent, ~r/mediawiki_client_ex.*\d.*/) assert env.method == :get @@ -373,7 +373,7 @@ defmodule ActionTest do test "can override user agent" do TeslaAdapterMock |> expect(:call, fn env, _opts -> - {_, user_agent} = List.keyfind!(env.headers, "user-agent", 0) + {{_, user_agent}, _} = List.keytake(env.headers, "user-agent", 0) assert user_agent == "foo bar" {:ok, %Env{env | body: %{bar: :baz}, status: 200}} diff --git a/test/ores_test.exs b/test/ores_test.exs index 82fccc2..40bf63d 100644 --- a/test/ores_test.exs +++ b/test/ores_test.exs @@ -18,7 +18,7 @@ defmodule OresTest do test "requests singular resource" do TeslaAdapterMock |> expect(:call, fn env, _opts -> - {_, user_agent} = List.keyfind!(env.headers, "user-agent", 0) + {{_, user_agent}, _} = List.keytake(env.headers, "user-agent", 0) assert String.match?(user_agent, ~r/mediawiki_client_ex.*\d.*/) assert env.query == [models: "damaging", revids: 12_345] -- GitLab From 3b78e471eec8e1e5abd765c191d28c67918a664e Mon Sep 17 00:00:00 2001 From: Adam Wight <adam.wight@wikimedia.de> Date: Wed, 21 Dec 2022 12:01:34 +0100 Subject: [PATCH 40/40] Require supported elixir Drop releases no longer receiving security updates. --- .gitlab-ci.yml | 2 +- mix.exs | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml index 9e0064f..2c90f9f 100644 --- a/.gitlab-ci.yml +++ b/.gitlab-ci.yml @@ -18,7 +18,7 @@ variables: stable: extends: .common - image: elixir:1.9-alpine + image: elixir:1.11-alpine before_script: # FIXME: can be removed if forked deps are merged diff --git a/mix.exs b/mix.exs index 1776040..c65a03b 100644 --- a/mix.exs +++ b/mix.exs @@ -5,7 +5,7 @@ defmodule Elixir.MixProject do [ app: :mediawiki_client, version: "0.4.2", - elixir: "~> 1.9", + elixir: "~> 1.11", elixirc_paths: ~w(lib), start_permanent: Mix.env() == :prod, description: description(), -- GitLab