Pipelines Resources

Resource Scope and Visibility

Pipeline resources have the same scope as the pipeline source where they are defined.

When defining resources, the recommended approach is to define them in a pipeline source that is shared across environments in the same Project. This ensures that the resources are available across environments in a project. For more information, seeCreating a Project - Pipelines Resources.

Resource Types

Resources play different roles based on where and how they are used:

• Triggering Resource: Whenever there is a change in this type of resource, it triggers the dependent step. Examples:CronTrigger,GitRepo,IncomingWebhook.
• Generated Resource: These are resources that are generated by a step and they can trigger successive downstream steps. Examples:Aql,BuildInfo,FileSpec,Image.
• Webhook Resource:IncomingWebhookandOutgoingWebhookare resources that can be used to integrate a pipeline with third-party services.

Pipelines supports several types of resources, with each resource capable of serving a specific activity:

Resource Versions

One of the key features of Pipelines resource is versioning.Resource versions are useful for controlling the flow of pipelines and tracking the changes that a resource undergoes over time. You can trigger runs usingspecific versionsorskip stepsin the same run when input resources are not updated.

Every resource starts with an initial version, which is updated every time the resource changes. For example, the version of anImageresource used in a pipeline is updated whenever a new tag is pushed to a Docker image. Pipelines tracks these changes by updating the resource version, which is based on the metadata received for that particular resource.

A new resource version is created when:

• The resource definition is updated in the pipelines YAML file.
• An output resource is updated during a run.
• There is an external event, such as pushing a commit to a git repository.

A resource returns the same result every time a specific version is used for a run. By default, steps always run using the latest version of an input resource. However, because a resource is versioned and holds the entire history of all the available versions, a run can be customized to use a specific version of a resource. For moreinformation, seeTriggering a Run with Custom Parameters.

Example - Resource Versions

This is a simple, single-step pipeline, withGitRepoas the input resource for the step.

In this example:

• First version: When the pipeline syncs and loads for the first time, the latest commit on the Git repository is seeded as the very first version for the Gitrepo resource.
  When the first run is triggered, this resource version is used for the run.
• Second version: When a new commit is pushed to the Git repository, the Gitrepo resource is updated with the new version, which becomes the latest version of the resource.
  The Git commit automatically triggers the run and the second version of the resource, which is now the latest, is used for the run.

Creating Resources

All resourcesare defined in apipeline YAMLunder the2022世界杯阿根廷预选赛赛程tag, as shown below. After a resource is defined and committed to a source control, it is consumed within a pipeline, based on the scope defined for the pipeline source.

While each resource has its own specific configuration, they all require anameand atype.

resources: - name:  type:  configuration: 

resources: - name: myPropertyBag type: PropertyBag configuration: storeVersionHistory: true myEnv: 2

Examples - Resource Definition

These examples show the YAML definition forGitRepoandImageresources:

• Example 1 - GitRepo Resource
  

  resources: - name: gitrepo_trigger type: GitRepo configuration: gitProvider: my_github path: myuser/repo-name branches: include: master

• Example 2 - Image Resource
  

  resources: - name: Image_1 type: Image configuration: registry: PSSG_DockerRegistry imageName: docker/jfreq_win imageTag: latest autoPull: true

Modifying Resources

在管道、tig资源及其版本2022世界杯阿根廷预选赛赛程htly coupled. Therefore, when a resource is deleted, itshistorical data is permanently affected. This can mess up your DevOps Assembly Lines as it is a connected inter-dependent workflow.

The following rules apply when editing resources:

• If you modify a resource's name, it is treated as new resource.
• A resource'stypecannot be modified.

Using Resources

In a pipeline, steps can use resources as:

• Inputs: When a resource is an input for a step, it is called an input resource.The input resource of one step can be the output resource of other steps.
• Outputs: When a resource is an output of a step, it is called an output resource. The output resource of one step can be the input resource of other steps.

Input and output resources can originate from the same pipeline source as the pipeline or another pipeline source in the same project and environment.

Input Resources

Input resources enable you to create dependencies between steps and pipelines. Steps that have input resources from other pipelines trigger a run of the pipeline when the resource is updated. By default, input resources that were an output resource of another step in the same run will run whether or not the resource is updated. A resource can also be referred to by its name as an argument in the shell commands that the step executes.

Input Resource Definiton

A resource can be specified as an input for a step by adding it to theinputResourcessection of a step.

steps: - name:  type:  configuration: inputResources: - name:  trigger:  # default true newVersionOnly:  # default false branch:  # see description of defaults below

steps: - name: step_1 type: Bash configuration: inputResources: - name: my_app_repo trigger: false # optional; default true newVersionOnly: true # optional; default false branch: master # optional

Using Input Resources

This section provides information about the various ways in which input resources can be used to manipulate pipeline runs.

Skip Automatic Trigger for All Commits

By default, changes to an input resource triggers the execution of the dependent steps.For example, when a step specifies aGitReporesource, any new code committed to that Git repository automatically causes that step to execute. However, this behavior can be changed by declaringtriggerasfalse(see below). Now, even when the resource is updated, the dependent step is not triggered. This is especially useful for a production pipeline, when you do not want to deploy every new build.

pipelines: - name: java_pipeline steps: - name: step_1 type: Bash configuration: inputResources: - name: my_app_repo trigger: false - name: cron_trigger trigger: false execution: onExecute: - pushd $res_my_app_repo_resourcePath - ./execute.sh - popd

Whentriggeris set asfalse, the line linking the input resource and the step appears as a dashed line.

Trigger Automatically on New Version Only

Whenever a resource undergoes a change, its version is updated and the dependent step is triggered. This is the default behaviour for all input resources.To skip steps in a run when input resources are not updated, add thenewVersionOnlytag and set it astrue. During a run, the step is triggered only when the resource is updated. If the resource is not updated, the step is skipped and all the downstream steps are skipped as well.

pipelines: - name: java_pipeline steps: - name: step_1 type: Bash configuration: inputResources: - name: my_app_repo - newVersionOnly: true execution: onExecute: - pushd $res_my_app_repo_resourcePath - ./execute.sh - popd

resources: - name: S_WF_012_resource type: PropertyBag configuration: runNumber: 0 pipelines: - name: pipeline_S_WF_012_001 steps: - name: S_WF_012_input1 type: Bash configuration: outputResources: - name: S_WF_012_resource execution: onExecute: #- write_output S_WF_012_resource runNumber=${run_number} - echo "test" - name: step1 type: Bash configuration: inputResources: - name: S_WF_012_resource newVersionOnly: true execution: onExecute: - echo "test" - name: step2 type: Bash configuration: inputResources: - name: S_WF_012_resource execution: onExecute: - echo "test"

Trigger Manually using Specific Versions

A run can be customized by selecting a specific version for an input resource. For more information, seeTriggering a Run with Custom Parameters.

Pinning Resource Versions

By default, Pipelines uses the most recent or latest version of aninput resource when running a job. However, there could be cases where you want to use a specific version of an input resource for a run. This is called pinning and input versions can be pinned using the YAML configuration. When a resource version is unpinned, it switches to using the latest version for all subsequent runs.

Resource version Ids have a global sequence, which can be found on theResourcetab. For more information, seeViewing Resources.

Pinning Resource Versions in YAML

You can use thepintag to pin a specific input version as shown below below:

resources: - name:  type: DistributionRule configuration: pin: versionId: 

The following resources support version pinning:

• BuildInfo
• GitRepo
• HelmChart
• Image
• PropertyBag
• ReleaseBundle

Output Resources

Output resources are resources that are either generated or changed by a step.When specified as the output of a step, the resource receives the output of the step. If required, this output resource can then be used as an input resource in a subsequent step in the same pipeline or another pipeline.The output resourcecan also be referred to by its name as an argument in the shell commands that the step executes.

Output Resource Definiton

A resource can be specified as an output for a step by adding it in theoutputResourcessection of a step.

steps: - name:  type:  configuration: outputResources: - name:  branch:  # see description of defaults below

steps: - name: step_2 type: Bash configuration: outputResources: - name: my_repo branch: master

Viewing Resources in the UI

After a pipeline's YAML fileis committed to a repository, add the repository to Pipelinesthrough the UI. The Pipelines platform then watches for changes (job additions, edits or deletes) through source control webhooks. YAML changes are automatically synced and are reflected in the UI immediately.

After the pipeline source successfully syncs the YAML file, selectApplications|My Pipelinesto view the the pipeline.

In the Pipelines view:

• Each resource is shown asa circular icon
• Clicking a resource displays information specific to that resource
• Clicking the YAMLicon →2022世界杯阿根廷预选赛赛程tab displays the2022世界杯阿根廷预选赛赛程definition for the pipeline

Resetting Resource Versions

If a resource is in an invalid state, you have the option to reset it, which delete all previous versions of the resource. For example, when a source repository invalidates the SHA that is used by a GitRepo resource.

To reset a resource:

1. In the Pipelines view, click the resource and then click theResetbutton.
2. ClickConfirm.

Advanced Usage of Resources

In a pipeline, the role of a resource goes beyond inputs and outputs. Here are several other ways in which you can use resources in your pipeline.

Using Resources Values in Environment Variables

A step that specifies a resource can access the resource and its attributes through environment variables.

• These environment variables have a standard naming convention:res__
• This can be extended further to access an integration that is specified in the resource:res___

Example

• If the definition of aGitReporesource namedapp_gitrepois as follows:

  GitRepo Resource Definition

  resources: - name: app_gitRepo type: GitRepo configuration: path: user1/repo1 gitProvider: myGitProvider

• The environment variable definition would be as follows:

  Environment Variable	Result	Description
  res_app_gitRepo_path	user1/repo1	Returns thepathattribute of theapp_gitReporesource
  res_app_gitRepo_gitProvider_url		Returnsurlof the Git Provider of theapp_gitReporesource

• Then the following environment variables will be available to a step that uses theapp_gitReporesource as an input:
  

  Environment Variables

  steps: - name: build_app type: MvnBuild configuration: sourceLocation: . mvnCommand: clean install configFileLocation: . inputResources: - name: app_gitRepo # Use the app_gitRepo resource execution: onSuccess: - send_notification notifySlack --text "Maven build completed for $res_app_gitRepo_path at $res_app_gitRepo_gitProvider_url" onFailure: - send_notification notifySlack --text "Maven build FAILED for $res_app_gitRepo_path at $res_app_gitRepo_gitProvider_url"

Using Arrays

Certain resources, such asAql,DistributionRule, andVmClusterhave arrays in their configuration.

• For arrays within resources (VmCluster):

res___len (this tells you how many entries in array)
res___0
res___1
res___2

• For object arrays within resources (DistributionRule):

res___len (this tells you how many entries in array)
res___0_
res___1_

Using Stateful Resources

Resources are stateful entities and persist across pipelines, enabling passing of information between pipelines. This is especially useful when creating a pipeline of pipelines.

Steps can store any key-value pair data in a resource using thewrite_outpututility function. These values can then be referencedas environment variables by any subsequent cuting step that uses that resource as an input. Therefore, a step can pass information to another step in the run of the pipeline.

The environment variable for the stored value is of the formres__.

Example

The following example creates three properties in the resourcemyImage.

write_output myImage sport="baseball" equipment="bat" field="diamond"

When the resource is specified in a step'sinputResources, these properties can be accessed as the following environment variables:

$ printenv res_myImage_baseball baseball $ printenv res_myImage_equipment bat $ printenv res_myImage_field diamond

For more information, seeCreating Stateful Pipelines.

Using Extension Resources

Extension resources enable Pipelines users to extend the Pipelines DSL by specifying their own resource types.后一个扩展资源加载,可以used by any step in a pipeline. For more information, seePipelines Extension Resource Model.