Additional Patterns

Concourse Tools comes with some additional resource type “patterns” to cover some common requirements.

class concoursetools.additional.OutOnlyConcourseResource(version_class)

A version-less Concourse resource which can only run “out” code.

This resource implements check and in as “no-ops”. It can not be used in a get step, or to trigger builds. If you only need your resource to run code on a put step, then it is useful to inherit from this class instead to avoid needing to think about the relevant methods.

Parameters:

version_class (type[Version]) – The resource parses all inputs with this version class.

abstractmethod publish_new_version(sources_dir, build_metadata)

Update a resource by publishing a new version.

This method is called on a put step, and the step parameters are passed as additional keyword arguments. The method should return the new, and a dictionary of metadata (see Step Metadata).

Warning

The sources_dir argument does not behave the same as the destination_dir argument passed to the download_version() method. This is to more easily enable the resource to interact with other resources (and task outputs), but makes it difficult to “track down” the files relating to this resource. This is a deliberate design decision by Concourse, and you should expect users to explicitly pass the path to those files should they be needed in this method.

Example:

If the resource code looks like this:

class MyResource(ConcourseResource):

    def publish_new_version(self, sources_dir, build_metadata,
                            file_path, overwrite=False):
        ...
        metadata = {
            "HTTP Status": 200,
        }
        return version, metadata

then the resource user would invoke it in the pipeline like this:

- put: my-resource
  params:
    file_path: path/to/file.txt
    overwrite: true

Parameters:

• sources_dir (Path) – A path to folder containing all resources, not just this resource.

• build_metadata (BuildMetadata) – Metadata associated with this build.

Return type:

tuple[Version, dict[str, str]]

Returns:

The new version, and a dictionary of metadata.

class concoursetools.additional.InOnlyConcourseResource

A version-less Concourse resource which acts like an external function.

The common use-case of a OutOnlyConcourseResource in a Concourse pipeline is to run a “function”. The only way that this can “return” anything is if the result of the run is stored externally in your resource. This resource allows the user to “fetch” something from an external resource when this is not the case.

Note

The user should overload download_data() instead of download_version().

The correct use case of this resource is to execute a put step, and then place parameters in the get_params section:

- put: app-image
  get_params:
    skip_download: true

Note

This resource used a pre-defined version consisting of a single datetime object to ensure unique, non-empty versions.

abstractmethod download_data(destination_dir, build_metadata)

Download resource data and place files within the resource directory in your pipeline.

Note

This method is deliberately not passed a Version instance.

Example:

Consider a resource to download an image from an unversioned URL.

import requests

class MyResource(InOnlyConcourseResource):

    def __init__(self, image_url: str):
        super().__init__()
        self.image_url = image_url

    def download_data(self, destination_dir, build_metadata,
                      name: str = "image", chunk_size: int = 1024):
        response = requests.get(self.image_url, stream=True)
        response.raise_for_status()

        image_path = destination_dir / name
        with open(image_path, "wb") as wf:
            for chunk in response.iter_content(chunk_size):
                wf.write(chunk)

        metadata = {
            "HTTP Status": response.status_code,
        }
        return metadata

then the resource user would invoke it in the pipeline like this:

- put: my-resource
  get_params:
    name: my_image.png

Parameters:

• destination_dir (Path) – A path to a folder into which resource files should be placed.

• build_metadata (BuildMetadata) – Metadata associated with this build.

Return type:

dict[str, str]

Returns:

A dictionary of metadata.

class concoursetools.additional.TriggerOnChangeConcourseResource(version_class)

A Concourse resource which emits a new version whenever something has changed.

Only use this resource if you will never have any intermittent versions, and if linear versioning is not valid in this scenario. You should ensure that two instances of the version class can be checked for equality. If the latest version and the previous version match, then this resource will not emit a version, even if changes technically occurred between checks.

Note

The user should overload fetch_latest_version() instead of fetch_new_versions().

Parameters:

version_class (type[Version]) – The resource parses all inputs with this version class.

abstractmethod fetch_latest_version()

Fetch the latest version of the resource.

Return type:

Version

Returns:

The latest version of the resource.

class concoursetools.additional.MultiVersionConcourseResource(key, sub_version_class)

Bases: TriggerOnChangeConcourseResource[MultiVersion[SortableVersionT]]

A Concourse resource type designed to trigger to a change in available versions.

Sometimes a resource is designed to track a set of available items at any given time, and to emit a new version when that set changes. This resource treats each item as a “sub-version”, and tracks the set of these sub-versions at the source through an implicit “multi-version” class.

The default behaviour for download_version() is to create a JSON file containing a list of the flattened versions, which can be used within the pipeline. By default this resource does not publish new versions.

Parameters:

• key (str) – When the multi-version class is flattened, the sub-versions are cast to a sorted list and encoded as a JSON string. The final version consists of a single key/value pair, with the value being the JSON string and the key being this one.

• sub_version_class (type[Version]) – A subclass of Version to be used for each sub-version. Equality of these classes is what checks that the total set has changed. This class must be sortable.

Warning

The sub-version class must be sortable. Sub-versions are sorted prior to flattening to ensure consistency in the final flattened payload.

Tip

This resource class is best suited to resources used in conjunction with the set pipeline step.

abstractmethod fetch_latest_sub_versions()

Fetch the latest sub versions from the resource.

Return type:

set[Version]

Returns:

A set of the latest subversions from the resource.

download_version(version, destination_dir, build_metadata, file_name=None, indent=None)

Download a JSON file containing the sub-version data.

Parameters:

• version (MultiVersion[Version]) – The version to be downloaded.

• destination_dir (Path) – A path to a folder into which resource files should be placed.

• build_metadata (BuildMetadata) – Metadata associated with this build.

• file_name (str | None) – The name of the file. Defaults to the key parameter passed to the resource class.

• indent (int | None) – An optional indent for the JSON file. Only useful if you care about formatting.

Return type:

tuple[MultiVersion[Version], dict[str, str]]

Returns:

The unchanged version and an empty dictionary of metadata.

class concoursetools.additional.SelfOrganisingConcourseResource(version_class)

A Concourse resource which orders and filters versions on your behalf.

Users should rely on the resource to deduce the new versions from a list of all versions, and the order in which they should be presented. This is useful for simplifying logic in certain scenarios, but requires that versions can be ordered.

Note

The user should overload fetch_all_versions() instead of fetch_new_versions().

Caution

This is not always easy to do when chronology is determined by the ordering of a web response, for example, instead of a value within the version itself. However, it does avoid the need to remember the order in which new versions should be returned.

Parameters:

version_class (type[Version]) – The resource parses all inputs with this version class.

abstractmethod fetch_all_versions()

Fetch every available version of the resource.

Note

As usual, If there are no new versions, the list of new versions should only include the previous version.

Return type:

set[Version]

Returns:

A list of every resource version.

Combining Resource Types

Occasionally you may wish to implement multiple resource types with the same set of dependencies. Although it is often cleaner to treat these separately and build separate Docker images, there are times where it is easier to build a single image containing all of your resource types, and to select one of them via the resource config. To do this, you can use the combine_resource_types() function:

concoursetools.additional.combine_resource_types(resources, param_key='resource')

Return a pseudo-resource which will delegate to other resources depending on a flag.

Returns a pseudo-ConcourseResource class which delegates to other resources depending on the flag passed in the resource config.

Warning

This pseudo-resource cannot be instantiated as normal, and can only be run via the Main Scripts.

Parameters:

• resources (dict[str, type[ConcourseResource[Version]]]) – A mapping of key to ConcourseResource subclass. The delegated user is selected with the key.

• param_key (str) – The key in the resource config used to select the resource from resources. The value is popped from the config before it is passed to the delegate.

Example:

>>> resources = {
...     "A": ResourceA,
...     "B": ResourceB,
... }
>>> CombinedResource = combine_resource_types(resources)

This is then instantiated like so:

resources:
  - name: my-resource
    type: multi-resource-type
    source:
      resource: A
      ...

Return type:

type[_PseudoConcourseResource[Version]]