Deploying the Resource Type#

To properly “deploy” the resource type, your repo should look something like this:

.
|-- Dockerfile
|-- README.md
|-- assets
|   |-- check
|   |-- in
|   |-- out
|-- requirements.txt
|-- concourse.py
|-- tests.py

Resource Module#

The concourse.py module contains your Version and ConcourseResource subclasses. It can be called whatever you like, but concourse.py is the most clear, and is the default for the CLI.

Note

It is possible to split your code into multiple modules, or even packages. This is sometimes useful if you have written a lot of code, or if it naturally splits. However, make sure that this code is also put in the correct place in your Docker image.

Requirements#

The requirements.txt file should contain a complete list of requirements within your virtual environment (version pinned) sufficient to reproduce with

$ pip install -r requirements.txt --no-deps

If this is not possible, then rewrite your requirements file. If you have any dependencies which are not public, then you will have to make some adjustments to your Dockerfile.

Assets#

The contents of each of the files in assets follow the same pattern. For example, here is the contents of assets/check:

#!/usr/bin/env python3
"""
Check for new versions of the resource.
"""
from concourse import MyResource


if __name__ == "__main__":
    MyResource.check_main()

It is these files which get called by Concourse as part of the resource. Note that none of the asset files have any extensions, but specific their executable at the top of the file. MyResource should correspond to your ConcourseResource subclass, and resource to the module in which you have placed it. Replace check_main() with in_main() and out_main() for assets/in and assets/out respectively.

Important

Every file in assets need to be executable. This can be done with

$ chmod +x assets/*

Tip

Because this pattern is suitable for almost every resource type, you can automate the creation of the assets folder by running python3 -m concoursetools assets. You can also pass the following options:

  • -e, --executable: The python executable to place at the top of the file. Defaults to /usr/bin/env python3.

  • -r, --resource-file: The path to your resource module. Defaults to concourse.py.

  • -c, --class-name: The name of the resource class to import from the module. If there is only one subclass of ConcourseResource, then this will be automatically selected. If there are multiple subclasses and this option is not set then an error will be raised.

This corresponds to the create_asset_scripts() function.

Dockerfile#

The Dockerfile should look something like:

FROM python:3.8-alpine

COPY requirements.txt requirements.txt

RUN python3 -m pip install --upgrade pip && \
    pip install -r requirements.txt --no-deps

WORKDIR /opt/resource/
COPY concourse.py ./concourse.py
RUN python3 -m concoursetools . -r concourse.py

ENTRYPOINT ["python3"]

You should adjust the base image according to your requirements. If you cannot use the CLI to create your assets folder, then you should manually copy your asset files across to /opt/resource.

Tip

You can automate the creation of this file with

$ python3 -m concoursetools --docker .

The same arguments from the previous section can also be passed here, and will be used by the inner call to the CLI.

This corresponds to the create_dockerfile() function.

Including Certs in your Docker Build#

If any of your requirements are private (Concourse Tools is not a public project, and so you are probably putting a git clone path in your Requirements file), then you should build your Docker image in two stages, keeping your keys out of the final image which gets pushed to your Docker registry:

FROM python:3.8 as builder

RUN apk update --no-progress && apk add openssh-client && apk add --no-cache --no-progress git

ARG ssh_known_hosts
ARG ssh_private_key

RUN mkdir -p /root/.ssh && chmod 0700 /root/.ssh
RUN echo "$ssh_known_hosts" > /root/.ssh/known_hosts && chmod 600 /root/.ssh/known_hosts
RUN echo "$ssh_private_key" > /root/.ssh/id_rsa && chmod 600 /root/.ssh/id_rsa

COPY requirements.txt requirements.txt

RUN python3 -m venv /opt/venv
# Activate venv
ENV PATH="/opt/venv/bin:$PATH"

RUN python3 -m pip install --upgrade pip && \
    pip install -r requirements.txt --no-deps


FROM python:3.8-alpine as runner
COPY --from=builder /opt/venv /opt/venv
# Activate venv
ENV PATH="/opt/venv/bin:$PATH"

WORKDIR /opt/resource/
COPY concourse.py ./concourse.py
RUN python3 -m concoursetools . -r concourse.py

ENTRYPOINT ["python3"]

You can then pass the contents of your SSH private key and known_hosts file to the builder image, but not the final image:

$ docker build --build-arg ssh_private_key="$(cat ~/.ssh/id_rsa)" --build-arg ssh_known_hosts="$(cat ~/.ssh/known_hosts)" . -t <repo>:<tag>

Note

The reason we don’t just mount .ssh or similar is to make this pattern as reusable as possible, since you may need to pass arbitrary certs.

The Python venv module is necessary to easily copy the installed requirements to the new image.

Tip

You can easily generate this skeleton with

$ python3 -m concoursetools --docker . --include-rsa

Dockertools Reference#

Functions for creating the Dockerfile or asset files.

concoursetools.dockertools.create_dockerfile(args, encoding=None)[source]#

Create a skeleton dockerfile.

Parameters:

  • args (Namespace) – The CLI args.

  • encoding (Optional[str]) – The encoding of the file as passed to write_text(). Setting to None (default) will use the user’s default encoding.

Return type:

None

concoursetools.dockertools.create_asset_scripts(assets_folder, resource_class, executable='/usr/bin/env python3')[source]#

Create the scripts in a given folder.

Parameters:

  • assets_folder (Path) – The location to which the assets folder will be written. The folder will be created if it doesn’t yet exist.

  • resource_class (Type[ConcourseResource]) – A ConcourseResource subclass whose methods will be passed to create_script_file().

  • executable (str) – The executable to use for the script (at the top).

Return type:

None

concoursetools.dockertools.create_script_file(path, method, executable='/usr/bin/env python3', permissions=493, encoding=None)[source]#

Create a script file at a given path.

Parameters:

  • path (Path) – The path at which the file will be created.

  • method (Callable[[], None]) – The method of the ConcourseResource to be exported.

  • executable (str) – The executable to use for the script (at the top).

  • permissions (int) – The (Linux) permissions the file should have. Defaults to rwxr-xr-x.

  • encoding (Optional[str]) – The encoding of the file as passed to write_text(). Setting to None (default) will use the user’s default encoding.

Return type:

None

concoursetools.dockertools.file_path_to_import_path(file_path)[source]#

Convert a file path to an import path.

Parameters:

file_path (Path) – The path to a Python file.

Raises:

ValueError – If the path doesn’t end in a ‘.py’ extension.

Example:
>>> file_path_to_import_path(pathlib.Path("module.py"))
'module'
>>> file_path_to_import_path(pathlib.Path("path/to/module.py"))
'path.to.module'
Return type:

str

concoursetools.dockertools.import_resource_class_from_module(file_path, class_name=None, parent_class=<class 'concoursetools.resource.ConcourseResource'>)[source]#

Import the resource class from the module.

Similar to import_resource_classes_from_module(), but ensures only one class is returned.

Parameters:

  • file_path (Path) – The location of the module as a file path.

  • class_name (Optional[str]) – The name of the class to extract. Required if multiple are returned.

  • parent_class (Type[Any]) – All subclasses of this class defined within the module (not imported from elsewhere) will be extracted.

Return type:

Type[Any]

Returns:

The extracted class.

Raises:

RuntimeError – If too many or too few classes are available in the module, unless the class name is specified.

concoursetools.dockertools.import_resource_classes_from_module(file_path, parent_class=<class 'concoursetools.resource.ConcourseResource'>)[source]#

Import all available resource classes from the module.

Parameters:

  • file_path (Path) – The location of the module as a file path.

  • parent_class (Type[Any]) – All subclasses of this class defined within the module (not imported from elsewhere) will be extracted.

Return type:

Dict[str, Type[Any]]

Returns:

A mapping of class name to class.

class concoursetools.dockertools.Namespace(path, executable='/usr/bin/env python3', resource_file='concourse.py', class_name=None, docker=False, include_rsa=False)[source]#

Represents the parsed args for typing purposes.

Parameters:

  • path (str) – The location at which to place the scripts.

  • executable (str) – The python executable to place at the top of the file.

  • resource_file (str) – The path to the module containing the resource class.

  • class_name (Optional[str]) – The name of the resource class in the module, if there are multiple.

  • docker (bool) – Pass to create a skeleton Dockerfile at the path instead.

  • include_rsa (bool) – Enable the Dockerfile to (securely) use your RSA private key during building.

property resource_path: Path#

Return a path object pointing to the resource file.