Dockerfile Multi-stage Builds

wanzi

Understanding Docker Multi-stage Builds:

  • Building an image requires a base image; all subsequent operations are based on this base image.
  • Docker image files have a layered structure. Each RUN instruction adds a new layer, so reducing the number of layers helps minimize image size.
  • When multiple FROM instructions are used, only the last FROM image becomes the root image of the final build.

Example of multi-stage build in my own project deployment: Here, we compile a binary using the Golang base image and then directly copy it into a minimal Alpine-based image:

Read more

Detailed Dockerfile syntax

wanzi

FROM

Specifies the base image used to build the Docker image. FROM must be the first non-comment instruction in a Dockerfile. If the specified image is not available locally, Docker will automatically pull it from the public registry.

Example:

1

FROM ubuntu:14.04  # Inherits from ubuntu:14.04

MAINTAINER

Specifies the maintainer information.

1

MAINTAINER wanzi "iwz2099@163.com"

ENV

Sets environment variables that are available for subsequent RUN instructions and persist at runtime within the container.

Read more

Docker Basic Commands

wanzi

Common Commands

1
2
3
4
5
6
7
8

docker info     # View local Docker information
docker search openresty   # Search remote image repository
docker images      # View images in the local image repository
docker ps          # View currently running containers
docker pull centos  # Pull an image from the remote repository (default tag is 'latest' if not specified)
docker container run -p 8000:80 --rm -t -i centos:latest /bin/bash  # Map container port 80 to host port 8000; --rm removes the container after termination (useful for temporary debugging); -t allocates a pseudo-terminal and binds it to the container's stdin; -i keeps the container's stdin open
docker container exec -i -t b5d7bad57561 /bin/bash # Enter a running container
docker rmi $(docker images -f "dangling=true" -q) # Batch remove dangling (unnamed) images

Importing and Exporting Images

1
2
3

docker save -o centos7.tar centos  # Export image to tar file
docker load < centos7.tar   # Import image (via standard input, including original metadata such as tags)
docker load --input centos7.tar   # Load image from tar file (non-standard input method)

Importing and Exporting Containers

1
2
3
4
5
6

docker ps   # Default: show currently running containers
docker ps -a   # Show all containers, including stopped ones
docker ps -l   # Show the most recently created container (including stopped ones)

docker export 5a80afa126ba > centos7.5a80afa126ba.tar  # Export container snapshot
docker import 5a80afa126ba > centos7.5a80afa126ba.tar  # Import container snapshot into image repository

Note: Users can use docker load to import image files into the local image library, or docker import to import container snapshots. The key difference is that container snapshot files discard all historical records and metadata (preserving only the state at the time of the snapshot), while image storage files preserve full history and are larger in size. Additionally, when importing from a container snapshot, you can reassign metadata like tags.

Read more

Issuing Certificates Using OpenSSL

wanzi

Generate Client Private Key:

1

openssl genrsa -out server.key 2048

Generate Client Certificate:

1

openssl req -new -sha256 -x509 -days 3650 -key server.key -out server.crt

Certificate Signing Request (CSR):

1

openssl req -new -key server.key -out server.csr

Generate CA Private Key

Encrypted with des3, requires a password of more than 4 characters:

1

openssl genrsa -des3 -out ca.key 4096

Generate CA Certificate

1

openssl req -new -x509 -days 3650 -key ca.key -out ca.crt

Add the following configuration to the CA config file /private/etc/ssl/openssl.cnf (macOS system):

Read more

SaltStack's SaltAPI

wanzi

Installation

1

yum -y install salt-api

Configuration

cat /etc/salt/master.d/api.conf # Configure certificate and port

1
2
3
4
5

rest_cherrypy:
  port: 8888
  debug: True
  ssl_crt: /etc/pki/tls/certs/localhost.crt
  ssl_key: /etc/pki/tls/private/localhost_nopass.key

cat /etc/salt/master.d/eauth.conf # Set permissions

1
2
3
4
5
6

external_auth:
  pam:
    saltapi:
      - .*
      - '@wheel'
      - '@runner'

Add User

1
2

useradd -M -s /sbin/nologin saltapi
echo "saltapi_xxxxxx" | passwd saltapi --stdin

Start Service

1
2

systemctl enable salt-api
systemctl start salt-api

Test

Test using curl to obtain token information:

Read more

A Practical Guide to Salt Commands in SaltStack

wanzi

Common Commands

1
2
3
4
5
6
7
8
9

salt -N 'ceph' test.ping                     # Test connectivity
salt -E '^server10*' test.ping                # Match by regex for connectivity
salt -S 192.168.150.101 test.ping             # Match by agent IP address
salt -S 192.168.150.0/24 test.ping            # Match by IP subnet
salt -N 'ceph' cmd.run 'df -Th'              # Check disk usage by group
salt -N 'ceph' cmd.exec_code python 'import os; print os.system("df -Th")'  # Execute Python code
salt -N 'ceph' cmd.exec_code perl 'print scalar localtime'  # Execute Perl code
salt -G 'osrelease:6.3' cmd.run 'python -V'   # Filter by grains and execute
salt '*' smbios.get system-serial-number      # Get server hardware serial number

Remote Script Execution:

1

salt -N 'ceph' salt://scripts/runme.sh  # Execute script remotely

Remote File Copy:

1
2
3
4
5
6
7
8

salt -N 'ceph' cp.get_file salt://files/opencdn/a.txt /tmp/a.txt     # Pull file from master to agent
salt -N 'ceph' cp.get_file salt://files/opencdn/dir1/ /tmp           # Pull directory from master to agent
salt -N 'ceph' cp.get_url http://www.baidu.com /tmp/index.html       # Download URL
salt -N 'ceph' cp.push /etc/fstab                                   # Push file from minion to master (default: /var/cache/salt/master/minions/minion-id/files)
salt -N 'ceph' cp.push_dir /etc/modprobe.d/ glob='*.conf'          # Push specific config files from directory
salt-cp '*' fstab /etc/fstab                                         # Copy local fstab file to target node
salt-cp -E 'gpu0[1-2][1-5]' fstab /etc/fstab                        # Copy via regex match
salt-cp -G 'os:CentOS*' fstab /etc/fstab                             # Copy via grains match

Disk Information

1
2
3

salt -N 'ceph' disk.percent /data01        # Get disk usage percentage
salt -N 'ceph' disk.nodeusage /data01      # Get inode usage
salt -N 'ceph' xfs.info /data01            # Get XFS partition info

File Operations

1
2
3

salt -N 'ceph' file.chown /etc/passwd root root    # Change file ownership
salt -N 'ceph' file.copy /etc/passwd /tmp/passwd   # Copy file
salt -N 'ceph' file.directory_exists /etc/passwd   # Check if file exists

User Management

1
2
3

salt -N 'ceph' user.list_users         # List all system users
salt -N 'ceph' user.info lianghaiqiang  # Show user details
salt -N 'ceph' user.delete lianghaiqiang remove=True force=True  # Force delete user

Timezone, Reboot, and Cron

1
2
3
4
5

salt -N 'ceph' timezone.get_zone        # Get timezone
salt -N 'ceph' timezone.set_zone        # Set timezone
salt -N 'ceph' system.reboot            # Reboot system
salt -N 'ceph' cron.list_tab root       # List root's cron jobs
salt -N 'ceph' cron.raw_cron root       # List root's cron jobs in raw text format

File Extraction & Compression

1
2
3

salt -N 'ceph' archive.cmd_unzip template=jinja /tmp/zipfile.zip /tmp/{{grains.id}}/ excludes=file_1,file_2
salt -N 'ceph' archive.cmd_unzip /tmp/zipfile.zip /home/strongbad/ excludes=file_1,file_2
salt -N 'ceph' archive.cmd_zip template=jinja /tmp/zipfile.zip /tmp/sourcefile1,/tmp/{{grains.id}}.txt

Networking

1
2
3
4
5
6

salt -N 'ceph' network.dig www.baidu.com     # DNS lookup
salt -N 'ceph' network.get_hostname          # Get hostname
salt -N 'ceph' network.hw_addr eth0          # Get MAC address
salt -N 'ceph' network.ip_addrs              # Get IP addresses
salt -N 'ceph' network.ping www.baidu.com timeout=3  # Ping with timeout
salt -N 'ceph' pkg.install zlib               # Install package

Get Help on Objects:

e.g., grains/disk/pillar/pip/pkg

Read more

Python Coding Standards

wanzi

Encoding Declaration

Default encoding is not ASCII. You can specify it using:

1

# coding=<encoding name>

or

1
2

#!/usr/bin/python
# -*- coding: <encoding name> -*-

or

1
2

#!/usr/bin/python
# coding: utf-8

Code Layout

  • Use four spaces for indentation; avoid using tabs, and never mix tabs with spaces.
  • Limit line length to 79 characters. Use backslashes for line continuation, but prefer parentheses instead.
  • Leave two blank lines between classes and top-level functions; leave one blank line between methods within a class.
  • Module structure order: module description and docstring, import statements, global variables, constants, others. Within import, arrange in order: standard library, third-party, local modules. Avoid writing multiple imports on one line.
  • Add one space on both sides of operators. Omit spaces around assignment operators in default function arguments. Do not add spaces before closing brackets (], }, )).
  • Comments must be in English, preferably complete sentences. Start with a capital letter, end with a period (or other punctuation), followed by two spaces before the next sentence. For phrases, punctuation may be omitted.
  • Always write docstrings for public modules, functions, classes, and methods. Non-public ones don’t require docstrings, but comments are acceptable (placed on the line immediately after def).

Naming Conventions

  • Avoid using single lowercase letter l or uppercase O as they can easily be confused.
  • Module names should be short and use all lowercase letters, optionally separated by underscores.
  • Package names should be short and use all lowercase letters without underscores.
  • Class names use CapWords. Internal classes use _CapWords.
  • Exception names use CapWords with an Error suffix.
  • Global variables should ideally be limited to module scope (like static in C). Two ways to achieve this: use __all__ mechanism or prefix with an underscore.
  • Function names use all lowercase letters, optionally separated by underscores.
  • Constant names use all uppercase letters, optionally separated by underscores.
  • Class attributes (methods and variables) use all lowercase letters, optionally separated by underscores.
  • Class attributes have three scopes: public, non-public, and subclass API—similar to public, private, protected in C++. Prefix non-public attributes with a single underscore.
  • If a class attribute conflicts with a keyword, append an underscore. Avoid abbreviations or other workarounds.
  • To prevent naming conflicts with subclasses, prefix certain class attributes with double underscores. For example, in class Foo, define __a; access via Foo._Foo__a to avoid ambiguity. If a subclass also uses Foo, this protection fails.
  • The first parameter of instance methods must be self. The first parameter of static methods must be cls.

Code Checking Tools

1

pip install flake8

After installing flake8 successfully, open VSCode, go to File → Preferences → Settings, and add "python.linting.flake8Enabled": true in settings.json.

Read more