Unable to send http request to API in docker Service
Summary
I am unable to send http requests to a an API service running during a CI job
Steps to reproduce
I am trying to send an http request (using python Requests) to an api running as a docker service in a single job, (during a pytest session, but I don't suspect that is the problem).
I've used this for URLs in SQLAlchemy connection strings, so it seems odd that a hostname is failing
.gitlab-ci.yml
test api in service:
stage: test
services:
- name: tutum/hello-world:latest
alias: some_host
before_script:
- pip install -r requirements.txt
script:
- curl some_host
- pytest tests.pymain.py
import requests
def main():
response = requests.get("http:some_host:80")
response.raise_for_status()
return response.text
if __name__ == "__main__":
result = main()
print(result)tests.py
from main import main
def test_test_discovery():
return True
def test_http_request():
main()Actual behavior
Requests is unable to resolve the hostname for the service called.
Expected behavior
The http request should be passed through to the api in the container
Relevant logs and/or screenshots
job log
Running with gitlab-runner 13.2.0-rc2 (45f2b4ec)
on docker-auto-scale ed2dce3a
Preparing the "docker+machine" executor
00:44
Using Docker executor with image python:3.7 ...
Starting service tutum/hello-world:latest ...
Pulling docker image tutum/hello-world:latest ...
Using docker image sha256:31e17b0746e48958b27f1d3dd4fe179fbba7e8efe14ad7a51e964181a92847a6 for tutum/hello-world:latest ...
Waiting for services to be up and running...
Pulling docker image python:3.7 ...
Using docker image sha256:22c70bba8283cb87951743be5fa043bec56ade01ceee079e6c2357a10b44159a for python:3.7 ...
Preparing environment
00:01
Running on runner-ed2dce3a-project-20123946-concurrent-0 via runner-ed2dce3a-srm-1595543123-8b2782d8...
Getting source from Git repository
00:02
$ eval "$CI_PRE_CLONE_SCRIPT"
Fetching changes with git depth set to 50...
Initialized empty Git repository in /builds/BenDavidAaron/minimal-http-gitlab-service/.git/
Created fresh repository.
Checking out 0160a60d as master...
Skipping Git submodules setup
Executing "step_script" stage of the job script
00:06
$ pip install -r requirements.txt
Collecting attrs==19.3.0
Downloading attrs-19.3.0-py2.py3-none-any.whl (39 kB)
Collecting certifi==2020.6.20
Downloading certifi-2020.6.20-py2.py3-none-any.whl (156 kB)
Collecting chardet==3.0.4
Downloading chardet-3.0.4-py2.py3-none-any.whl (133 kB)
Collecting idna==2.10
Downloading idna-2.10-py2.py3-none-any.whl (58 kB)
Collecting importlib-metadata==1.7.0
Downloading importlib_metadata-1.7.0-py2.py3-none-any.whl (31 kB)
Collecting more-itertools==8.4.0
Downloading more_itertools-8.4.0-py3-none-any.whl (43 kB)
Collecting packaging==20.4
Downloading packaging-20.4-py2.py3-none-any.whl (37 kB)
Collecting pluggy==0.13.1
Downloading pluggy-0.13.1-py2.py3-none-any.whl (18 kB)
Collecting py==1.9.0
Downloading py-1.9.0-py2.py3-none-any.whl (99 kB)
Collecting pyparsing==2.4.7
Downloading pyparsing-2.4.7-py2.py3-none-any.whl (67 kB)
Collecting pytest==5.4.3
Downloading pytest-5.4.3-py3-none-any.whl (248 kB)
Collecting requests==2.24.0
Downloading requests-2.24.0-py2.py3-none-any.whl (61 kB)
Collecting six==1.15.0
Downloading six-1.15.0-py2.py3-none-any.whl (10 kB)
Collecting urllib3==1.25.10
Downloading urllib3-1.25.10-py2.py3-none-any.whl (127 kB)
Collecting wcwidth==0.2.5
Downloading wcwidth-0.2.5-py2.py3-none-any.whl (30 kB)
Collecting zipp==3.1.0
Downloading zipp-3.1.0-py3-none-any.whl (4.9 kB)
Installing collected packages: attrs, certifi, chardet, idna, zipp, importlib-metadata, more-itertools, pyparsing, six, packaging, pluggy, py, wcwidth, pytest, urllib3, requests
Successfully installed attrs-19.3.0 certifi-2020.6.20 chardet-3.0.4 idna-2.10 importlib-metadata-1.7.0 more-itertools-8.4.0 packaging-20.4 pluggy-0.13.1 py-1.9.0 pyparsing-2.4.7 pytest-5.4.3 requests-2.24.0 six-1.15.0 urllib3-1.25.10 wcwidth-0.2.5 zipp-3.1.0
$ curl some_host
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 478 0 478 0 0 95600 0 --:--:-- --:--:-- --:--:-- 95600
<html>
<head>
<title>Hello world!</title>
<link href='http://fonts.googleapis.com/css?family=Open+Sans:400,700' rel='stylesheet' type='text/css'>
<style>
body {
background-color: white;
text-align: center;
padding: 50px;
font-family: "Open Sans","Helvetica Neue",Helvetica,Arial,sans-serif;
}
#logo {
margin-bottom: 40px;
}
</style>
</head>
<body>
<img id="logo" src="logo.png" />
<h1>Hello world!</h1>
<h3>My hostname is 1807c7b442f0</h3> </body>
</html>
$ pytest tests.py
============================= test session starts ==============================
platform linux -- Python 3.7.8, pytest-5.4.3, py-1.9.0, pluggy-0.13.1
rootdir: /builds/BenDavidAaron/minimal-http-gitlab-service
collected 2 items
tests.py .F [100%]
=================================== FAILURES ===================================
______________________________ test_http_request _______________________________
def test_http_request():
> main()
tests.py:7:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
main.py:4: in main
response = requests.get("http:some_host:80")
/usr/local/lib/python3.7/site-packages/requests/api.py:76: in get
return request('get', url, params=params, **kwargs)
/usr/local/lib/python3.7/site-packages/requests/api.py:61: in request
return session.request(method=method, url=url, **kwargs)
/usr/local/lib/python3.7/site-packages/requests/sessions.py:516: in request
prep = self.prepare_request(req)
/usr/local/lib/python3.7/site-packages/requests/sessions.py:459: in prepare_request
hooks=merge_hooks(request.hooks, self.hooks),
/usr/local/lib/python3.7/site-packages/requests/models.py:314: in prepare
self.prepare_url(url, params)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
self = <PreparedRequest [GET]>, url = 'http:some_host:80'
params = OrderedDict()
def prepare_url(self, url, params):
"""Prepares the given HTTP URL."""
#: Accept objects that have string representations.
#: We're unable to blindly call unicode/str functions
#: as this will include the bytestring indicator (b'')
#: on python 3.x.
#: https://github.com/psf/requests/pull/2238
if isinstance(url, bytes):
url = url.decode('utf8')
else:
url = unicode(url) if is_py2 else str(url)
# Remove leading whitespaces from url
url = url.lstrip()
# Don't do any URL preparation for non-HTTP schemes like `mailto`,
# `data` etc to work around exceptions from `url_parse`, which
# handles RFC 3986 only.
if ':' in url and not url.lower().startswith('http'):
self.url = url
return
# Support for unicode domain names and paths.
try:
scheme, auth, host, port, path, query, fragment = parse_url(url)
except LocationParseError as e:
raise InvalidURL(*e.args)
if not scheme:
error = ("Invalid URL {0!r}: No schema supplied. Perhaps you meant http://{0}?")
error = error.format(to_native_string(url, 'utf8'))
raise MissingSchema(error)
if not host:
> raise InvalidURL("Invalid URL %r: No host supplied" % url)
E requests.exceptions.InvalidURL: Invalid URL 'http:some_host:80': No host supplied
/usr/local/lib/python3.7/site-packages/requests/models.py:391: InvalidURL
=========================== short test summary info ============================
FAILED tests.py::test_http_request - requests.exceptions.InvalidURL: Invalid ...
========================= 1 failed, 1 passed in 0.15s ==========================
ERROR: Job failed: exit code 1Environment description
This is using the public runners with default settings and no tags.
Possible fixes
I wish I knew, I'm going to comb the runner codebase to find a workaround.