Release - September 22
Search…
⌃K

Solution Overview

In order to configure [email protected] with GitLab, there are a number of key features that need to be setup on the platform before executing the pipeline. Whether you are new to GitLab or have prior experience developing CI/CD for other software stacks on GitLab, this guide is meant to summarize the key configuration steps on the platform and use the templates provided in our sample repository.
For a deeper dive on the platform, documentation is available on GitLab Docs.

GitLab Overview

GitLab Architecture
Feature
Summary
Organization
Top level hierarchy in GitLab used to configure your users and access to projects.
Project
Projects are used to host your codebase, track issues, plan work, collaborate on code, and continuously build, test, and use built-in CI/CD to deploy your app.
Repository
Store your code in metadata source format and make changes to it. Your changes are tracked with version control.
Issues
Collaborate on ideas, solve problems, and plan work.
CI/CD
Tooling built into GitLab for software development through the continuous methodologies:
  • Continuous Integration (CI)
  • Continuous Delivery (CD)
  • Continuous Deployment (CD)
Package & Repository
Acts as a private or public registry for a variety of common package managers and allows publishing and sharing packages. Project Level scoped npm packages are used for the solution.
Settings
Update general project settings, merge request approvals, default and protected branches, access tokens, CI/CD variables.
Others
Additional menu options in GitLab such as Project Information, Security & Compliance, Deployments, Monitor, Infrastructure, Analytics, Wiki, and
Snippets.
Member
Users and groups who have access to your project. Each member gets a role, which determines what they can do in the project.
Group
A group is used to manage one or more related projects at the same time.
Subgroup
Nested groups or hierarchical groups.
User
Each GitLab account has a user profile, which contains information about you and your GitLab activity. Set up of Personal Access Tokens and SSH Keys will enable users to to securely access the GitLab API from their computers.
Roles
Users have different abilities depending on the role they have in a particular group or project. The roles available in GitLab are Owner, Maintainer, Developer, Reporter and Guest.

GitLab Terminology

Area
Description
Anchors
Used to duplicate or inherit properties. The template leverages anchors for reusable rules based on merge requests, merges, and manual executions of the pipeline.
Artifacts
List of files and directories to attach to a job when it succeeds, fails, or always. Artifacts are use to output pmd reports, scratch org prepare logs, and quick build artifacts from the runner that are not published to artifact registry.
Override a set of commands that are executed before job scripts. The before_script commands are concatenated with any scripts you specify in the main script. The template uses the before_script to authenticate to the DevHub, configure access to the npm package registry, sets git repository url to allow for pushing git tags and committing change log files.
Custom variable in the .gitlab-ci.yml file can be defined in the top level or the job level.
Restrict which artifacts are passed to a specific job by providing a list of jobs to fetch artifacts from. Dependencies are used in the template to pass artifacts from the quickbuild stage to the deploy stage to push code to the Shared Developer Sandbox during code merges.
Name of an environment to which the job deploys. The template defines environments for the deploy-env, build-env, ST, SIT, UAT, and PROD environments.
Extends
Used for reuse configuration sections defined above in the Anchors for merge requests, merge, and manual execution rules in the pipeline.
Jobs
Pipeline configuration begins with jobs. Jobs are picked up by runners and executed in the environment of the runner. Jobs defined in the template follow the [email protected] best practices from validate to release.
Needs
Execute jobs earlier than the stage ordering and ensures previous jobs in the pipeline are completed before moving to the next stage and job.
.npmrc
The npm config file that is configured to allow publishing and sharing packages.
Pipeline
Pipelines are the top-level component of continuous integration, delivery, and deployment. Pipeline comprise of jobs, which define what to do and stages, which define when to run the jobs.
Predefined CI/CD variables are available in every GitLab CI/CD pipeline.
Similar to personal access tokens but are used by projects to authenticate with the GitLab API and are supported on GitLab SaaS Premium and above as well as self-managed instances on Free tier and above. API access grants complete read/write access to the scoped project API, including the Package Registry. This is required for the template to push git tags and changelog to the repository.
Used for concurrency control in the pipeline. The keyword is added to the deploy, build, and environment release jobs to ensure runners are prevented from executing jobs concurrently during execution.
Roles
Users have different abilities depending on the role they have in a particular group or project. If a user is both in a project’s group and the project itself, the highest role is used.
Rules
List of conditions to evaluate and determine selected attributes of a job, and whether or not it’s created.
Runners
GitLab Runner is an application that works with GitLab CI/CD to run jobs in a pipeline.
Script
Specify commands for the runner to execute. Most scripts in the template leverage sfpowerscripts:orchestrator commands during execution.
SSH Keys
GitLab uses the SSH protocol to securely communicate with Git. When you use SSH keys to authenticate to the GitLab remote server, you don’t need to supply your username and password each time.
Stage
Define which stage a job runs in. Jobs in the same stage can execute in parallel.
Variables
CI/CD variables are a type of environment variable

Security Design

There are a number of security considerations that need to factored into the setup of your GitLab project and pipeline. The following list will walk you through some considerations as you use the template and customize your pipelines for deployments.
  • Project and group visibility - GitLab allows Owners to set a project’s or group’s visibility as Public, Internal, Private. For most implementations, Private should be the preferred option.
  • Merge Request Approvals - Approval rules can be defined for merge requests to limit who can merge to certain branches
  • Protected Branches - Permissions are fundamentally defined around the idea of having read or write permission to the repository and branches. To impose further restrictions on certain branches, they can be protected. At a minimum, the main branch should be protected and certain releases branches created from main should be candidates for protection.
  • Permission and Roles - Users have different abilities depending on the role they have in a particular group or project. Review the structure of your team and confirm who to designate as Maintainers and who are primary Developers.
  • CI/CD Variable Types - Project, group and instance CI/CD variables can be Variable or File type. File type CI/CD variables are used in the template for storing sfdxurl file as input for authentication to Salesforce instances. Protect variable and Mask variable should be reviewed for each variable defined in the project to ensure that variables masked in job logs and can only run on protected branches or tags.
  • Using external secrets in CI - Secrets represent sensitive information your CI job needs to complete work. This sensitive information can be items like API tokens, database credentials, or private keys. Secrets are sourced from your secrets provider.
  • Integrations - Integrations allow you to integrate GitLab with other applications. Review current active integrations and determine which applications make sense to integrate to GitLab. Webhooks can be used in GitLabs but Integrations are preferred.
  • Project Access Tokens - Creation and rotation of project access tokens should be reviewed for configuring access to the GitLab API and the project.
  • SSH Keys and Personal Access Tokens - Depending on security guidelines, each contributor to your project repository should create these to streamline secure access.

Configuration File

The .gitlab-ci.yml template file is the primary configuration file used for executing continuous integration, delivery and deployment on the platform. This CI/CD configuration file exists by default in the root directory of your repository and controls pipeline execution of stages and jobs triggered from updates to the repository via merge requests/merges, scheduled executions, or manual execution of the pipeline.
In the template file provided, the structure of the YAML file follows the following structure:
Template Code Structure
Section
Description
Comments
General header for [email protected] Template for GitLab and reference links to review.
Project CI/CD Variables
Initial project Project CI/CD Variables required to be created in the project to authenticate and setup DevHub, Production and Sandbox Environment aliases, NPM Scope for artifacts, Project Access Tokens, and optional dashboard connection details for DataDog or New Relic.
Image
Docker image to use for the job, defaulted to the docker-sfpowerscripts image running on Ubuntu.
Variables
Custom and Predefined variables are used in the .gitlab-ci.yml during pipeline execution.
  • DEVHUB_ALIAS
  • DEVHUB_SFDX_AUTH_URL
  • SHAREDDEV_ALIAS
  • SHAREDDEV_SFDX_AUTH_URL
  • ST_ALIAS
  • ST_SFDX_AUTH_URL
  • UAT_ALIAS
  • UAT_SFDX_AUTH_URL
  • PROD_ALIAS
  • PROD_SFDX_AUTH_URL
  • NPM_SCOPE
  • PROJECT_ACCESS_TOKEN
  • SFPOWERSCRIPTS_DATADOG_HOST (Optional)
  • SFPOWERSCRIPTS_DATADOG_API_KEY (Optional)
  • SFPOWERSCRIPTS_NEWRELIC (Optional)
  • SFPOWERSCRIPTS_NEWRELIC_API_KEY (Optional)
  • CI_PROJECT_ID
  • CI_SERVER_HOST
  • CI_JOB_TOKEN
  • CI_PROJECT_PATH
  • CI_PIPELINE_SOURCE
  • CI_COMMIT_BRANCH
  • CI_PIPELINE_ID
  • BUILD_BRANCH
  • SCRATCH_ORG_USERNAME
  • TARGETTASKNAME
DevHub Authentication
SFDX auth URL is used to authorize the DevHub. Subsequent scripts in the pipeline jobs will use the DevHub alias during the deployment lifecycle. The account used in this authentication should be a dedicated CI/CD service account.
NPM Configuration
Setup of the npm config file used to allow publishing and sharing packages in the Project Package Registry is generated here using the NPM Scope defined in the Project Variables and predefined variables.
Git Repo Configuration
In order to push git tags and change logs to the remote repository for the project from the Runners, the project access token is used here to initialize the remote url.
Stages
Predefined stages for the deployment lifecycle for [email protected] is baseline here from validate to release and preparing and cleaning scratch org pools.
Anchors
Reusable rules are created here based on merge requests, merges, and manual executions of the pipeline to be later referenced in jobs for execution.
Trigger Job(s)
Each job in the pipeline is controlled by rules defined earlier by anchors to be executed in each stage and leveraging sfpowerscripts orchestrator commands.
Schedule Jobs(s)
Scheduled jobs are filtered in the configuration file using the TARGETTASKNAME variable and only executed during their defined interval pattern.
Manual Jobs(s)
Manually triggered jobs from the pipeline will be filtered. Deployments to environments once the artifacts have been successfully published are controlled by manual triggers by the user.

Pipeline Design

Triggered Jobs

The diagram below depicts the various stages and jobs configured in the GitLab CI/CD configuration file .gitlab-ci.yml which incorporates the sfpowerscripts orchestrator and sfpowerkit package commands to manage your CI/CD process. The grouping of stages and jobs are split into merge requests, merges, and manual triggers of the pipeline.
Type
Stage
Job
Command
Description
Merge Request
Analyze
analyze-pmd
Static code analysis PMD
Merge Request
Validate
validate-package
Validates against metadata coverage
Merge Request
Validate
validate-source
Validate deployment against scratch org
Merge
Quickbuild
quickbuild
Parallel build of packages without dependencies and code coverage
Merge
Deploy
deploy
Install package dependencies and deploy packages
Merge
Build
build
Build all packages and publish to artifact registry
Manual
Release
ST, SIT, UAT, PROD
Release to Salesforce Org based on YAML file

Scheduled and Manual Jobs

The diagram below highlights the dxatscale-template recommended scheduled jobs and manual job for Scratch Org Pool Management. The stages and jobs configured in the GitLab CI/CD configuration file .gitlab-ci.yml uses the sfpowerscripts prepare commands to build scratch org pools for Developer and CI Scratch Orgs. Additional schedule job for publishing metrics to your dashboard platform is provided as well.
Type
Stage
Job
Command
Description
Scheduled
Prepare
prepare-ci-pool
Prepares pool of CI Scratch Orgs
Scheduled
Prepare
prepare-dev-pool
Prepares pool of Developer Scratch Orgs
Scheduled
Clean
clean-ci-pool
Deletes specified tagged CI Scratch Org Pool
Scheduled
Clean
clean-dev-pool
Deletes specified tagged Developer Scratch Org Pool
Scheduled
Report
report-so-count
pool:metrics:publish
Publish metrics to Dashboard
Manual
Delete
delete-fetched-so
Deletes specified scratch org from provided user name

Dashboard Metrics

Metrics should be a key part of your DevOps process. It is through these metrics, one can drive continuous improvement of your delivery process.
The dxatscale-template includes sample cicd-dashboard.json file for NewRelic enables you to push StatsD metrics to the platform and monitor key metrics. Integration and setup of this is available in the Getting Started section.
The following variables are required to be setup in your project variables to push the metrics to New Relic.
  • SFPOWERSCRIPTS_NEWRELIC (Optional)
  • SFPOWERSCRIPTS_NEWRELIC_API_KEY (Optional)
Read more about Dashboards and Metrics here.

Configuration Options

There are additional configurations for the GitLab Project that should be reviewed and configured to ensure it aligns with your internal IT policies.
  • Merge requests - Merge method, merge options, merge checks, merge suggestions should be reviewed to determine preferred approaches such as merge commits, deleting source branches by default option, and allowing squash commits when merging.
[email protected] recommends squash commits when merging to provide succinct release notes during change log creation.
  • Merge request approvals **- Merge request approval rules prevent users from overriding certain settings on the project level. Define the number of approvals** required for merging into specific branches and specify users or groups that are allowed to approve them in the approval rules.
  • Repository Settings - A number of configurations for the repository can be reviewed and change such as default branches, push rules, protected branches and protected tags. Limit the restrictions where possible to not create bottlenecks for the pipeline and developers.
  • CI/CD Settings - Runners can be configured to be shared or specified. Environments can be setup for protection in the CI/CD.
  • Clean Scratch Org Pools - The clean-pool stage in the .gitlab-ci.yml file can be split into separate stages to hand the deletion of unused developer scratch orgs and CI scratch orgs.
Releases in GitLab are not used by the dxatscale-template as we leverage our own release management process through the sfpowerscripts orchestrator and package registry. We will revisit the potential use of this feature in GitLab in the future if there is a need.

Design Considerations

The following are additional design configurations to consider when using this template:
  1. 11.
    Merge Request Rules (eg. Squash commits when merging)
  2. 13.
    Security Requirements - Access Tokens and SSH Keys
  3. 14.
    Security Design - Permissions and Roles Assignment to Users
  4. 16.
    Naming Conventions for Repositories and Environments
  5. 17.
    Automated Testing Integration
  6. 18.
    Static Code Analysis Integration
  7. 19.
    Dynamic Code Testing Integration

FAQs

Why is the Access Tokens menu not available to configure my Project Access Tokens?

For GitLab SaaS hosting, Project Access Tokens are only available for Premium and above licenses and self-managed instances on Free tier and above.

References

Tutorials