When building frontend projects with npm, it’s common for dependency downloads to take a long time, and reusing artifacts or caches across different jobs is also challenging. Whether using artifacts or cache, we ultimately need persistent reuse of files. Here, we’ll use cache as an example.

Note: The GitLab Runner is deployed to the Kubernetes cluster via Helm chart (deployment details are omitted). You must prepare a Ceph S3 key pair in advance for configuring accesskey and secretkey.

Create Kubernetes Secret

Used by the GitLab Runner to connect to Ceph S3.

1
2
3
4
5
6
7
8
9

apiVersion: v1
data:
  accesskey: N1NMT0hIRzYxddfsgxVzVssddfsdY=
  secretkey: d25Uc0NDQVdsfsdUkssCQ1VsdEwxeUsdsNwb2R4TnRzZDliTG1DTUN6cQ==
kind: Secret
metadata:
  name: gitlab-runner-s3
  namespace: gitlab-managed-apps
type: Opaque

Deploy GitLab Runner via Helm

Refer to the official GitLab Helm chart: https://gitlab.com/gitlab-org/charts/gitlab-runner.git. Modify the values.yaml file in the Helm chart directory as follows:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12

cache:
  ## General settings
  cacheType: s3
  cachePath: "devops" # Specify the Ceph S3 cache path; here we use department names for separation
  cacheShared: true

  ## S3 settings
  s3ServerAddress: "ops-rgw.test.cn"
  s3BucketName: "runners-cache"
  s3BucketLocation:
  s3CacheInsecure: true
  secretName: "gitlab-runner-s3"

Update the Helm configuration:

1
2

cd gitlab-runner
helm upgrade runner-devops -f values.yaml -n gitlab-runner .

Test GitLab CI Job

Configure GitLab CI by modifying .gitlab-ci.yaml. Here’s an example for a frontend project build:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19

stages:
  - Build

build-and-deploy:
  image: registry.test.cn/devops/node:latest
  stage: Build
  cache:
    key: devops-vue
    paths:
      - node_modules/
      - .yarn
  tags:
    - devopstest
  script:
    - yarn config set registry https://r.cnpmjs.org
    - yarn config set @test:registry https://npm.test.cn/
    - yarn --pure-lockfile --cache-folder .yarn --network-timeout 600000
    - yarn build
  when: always

After triggering the GitLab CI job, observe the real-time job execution. At this point, cache files have been uploaded to Ceph S3. Subsequent builds will be significantly faster!

1
2
3
4
5
6
7
8

64 Creating cache devops-vue-4...
65 node_modules/: found 30710 matching files and directories
66 .yarn: found 34390 matching files and directories 
67 Uploading cache.zip to https://ops-rgw.test.cn/runners-cache/devops/project/4187/devops-vue-js-starter-4
68 Created cache
70 Cleaning up file based variables
00:00
72 Job succeeded