harbor配置外部数据库

​ harbor默认安装会使用官方打包的PostgreSQL docker镜像goharbor/harbor-db，harbor启动之后的数据均存放在改数据库上，后续管理可能存在不便，故使用harbor配置外部数据库。

一、搭建PostgreSQL数据库

​ harbor从1.6版本之后仅支持PostgreSQL数据库作为外部数据库，故需要搭建PostgreSQL数据库使用。这里目前作为测试，故仅安装了单节点，未配置主从，搭建步骤如下：

1. 安装

- 系统版本：CentOS7

## 1. 安装依赖包
yum install -y cmake gcc gcc-c++ perl readline readline-devel openssl openssl-devel zlib zlib-devel ncurses-devel readline readline-devel zlib zlib-devel

## 2. 源码安装PostgreSQL
[postgres@xxxx dba]$ wget https://ftp.postgresql.org/pub/source/v12.2/postgresql-12.2.tar.gz
[postgres@xxxx dba]$ tar zxf postgresql-12.2.tar.gz 
[postgres@xxxx dba]$ cd postgresql-12.2
[postgres@xxxx postgresql-12.2]$ ./configure --prefix=/usr/local/postgresql
[postgres@xxxx postgresql-12.2]$ make && make install
### 这里configure的时候制定了安装目录，故需要将该目录下的bin目录写入到环境变量中去，以方便后面直接使用，也可不指定，安装到默认路径下

2. 配置

# 1. 创建数据目录
[postgres@xxxx data]$ mkdir -p /work/harbor-db/data
# 2. 创建日志目录
[postgres@xxxx data]$ mkdir -p /work/harbor-db/log
# 3. 创建socket目录
[postgres@xxxx data]$ mkdir -p /work/harbor-db/tmp
# 4. 授权
[postgres@xxxx data]$ chown -R postgres.postgres /work/harbor-db/
# 5. 初始化pg实例
[postgres@xxxx data]$ initdb --username=postgres -D /work/harbor-db/data/
 
## 这里PostgreSQL数据库与harbor并未在同一台主机上，故除了修改配置文件postgresql.conf外还需要修改客户端认证配置pg_hba.conf文件，若在同一台主机上没有网络以及认证需求的话，可以不修改
# 6. 根据需要修改初始化的配置文件，修改位置如下：
[postgres@xxxx data]$ vim /work/harbor-db/data/postgresql.conf
 # 数据目录指定
data_directory = '/work/harbor-db/data'
 # 客户端可连接ip，默认为localhost，若不需要可不修改，*为所有
listen_addresses = '*'
 # 端口设置
port = 7002
 # 允许最大连接数
max_connections = 100
 # socket目录及权限设置
unix_socket_directories = '/work/harbor-db/tmp'
unix_socket_group = ''
unix_socket_permissions = 0777
 # 内存大小
shared_buffers = 128MB
 # 时区修改
timezone = 'Asia/Shanghai'
 
 # 日志：
 ## 是否开启日志
logging_collector = on
 ## 日志存放目录
log_directory = '/work/harbor-db/log'
 ## 每个日志最大size
log_rotation_size = 1GB
 ## 日志时区
log_timezone = 'Asia/Shanghai'
 ## 记录执行时间大于100ms的sql及执行时间，相当于慢SQL日志
log_min_duration_statement = 100

## 由于这里需要远程可以连接，所以需要添加认证配置pg_hba.conf，根据自己需求配置，若不需要的话可不配置该文件
[postgres@xxxx data]$ vim pg_hba.conf 
# 在文件末尾添加，以下配置表示，允许ADDRESS对应的主机，通过harbor用户访问该库的所有数据库
# TYPE  DATABASE        USER            ADDRESS                 METHOD
host    all             harbor             x.x.x.x/x           trust

3. 启动

[postgres@xxxx data]$ su - postgres
## 启动方式使用以下1种即可
[postgres@xxxx data]$ pg_ctl -D /work/harbor-db/data/ -l /work/harbor-db/log/start.log start

或使用 postgres -D /work/harbor-db/data > /work/harbor-db/log/start.log 2>&1 & 命令启动

4. 登陆测试

## 1. 本地测试
# 安装完成后会有postgres用户，相当于MySQL的root用户，默认没有密码
[postgres@xxxx data]$ psql -h 127.0.0.1 -p 7002 -U postgres
psql (12.2)
Type "help" for help.
### 修改postgres用户的密码
postgres=# \password postgres
Enter new password: 
Enter it again: 
### 创建harbor用户，并创建harbor所涉及数据库及进行授权
postgres=# create user harbor with password 'harbor123';
CREATE ROLE
postgres=# CREATE DATABASE harbor;
CREATE DATABASE
postgres=# create database harbor_clair;
CREATE DATABASE
postgres=# create database harbor_notary_server;
CREATE DATABASE
postgres=# create database harbor_notary_signer; 
CREATE DATABASE
postgres=# GRANT ALL PRIVILEGES ON DATABASE harbor to harbor;           
GRANT
postgres=# GRANT ALL PRIVILEGES ON DATABASE harbor_clair to harbor;           
GRANT
postgres=# GRANT ALL PRIVILEGES ON DATABASE harbor_notary_server to harbor;            
GRANT
postgres=# GRANT ALL PRIVILEGES ON DATABASE harbor_notary_signer to harbor;                    
GRANT

## 2. 远程主机harbor用户测试
[root@remote harbor]# psql -h x.x.x.x -p 7002 -U harbor -W 
Password: 
psql (12.2)
Type "help" for help.

harbor=>

​ 至此，PostgreSQL数据库及基础配置设置完毕。

二、Harbor配置

1. 配置文件修改

[root@remote harbor]# vim harbor.yml
# Uncomment external_database if using external database.
external_database:
  harbor:
    host: x.x.x.x
    port: 7002
    db_name: harbor
    username: harbor
    password: xxxxxxxx
    ssl_mode: disable
    max_idle_conns: 50
    max_open_conns: 100
  clair:
    host: x.x.x.x
    port: 7002
    db_name: harbor_clair
    username: harbor
    password: xxxxxxxx
    ssl_mode: disable
  notary_signer:
    host: x.x.x.x
    port: 7002
    db_name: harbor_notary_signer
    username: harbor
    password: xxxxxxxx
    ssl_mode: disable
  notary_server:
    host: x.x.x.x
    port: 7002
    db_name: harbor_notary_server
    username: harbor
    password: xxxxxxxx

2. docker-compose文件修改

​ 设置了外部数据库之后，便不再需要harbor本身的harbor-db镜像来支持，由安装重启文件install.sh可看出最终的安装等操作都由docker-compose.yml文件来完成，故需要在docker-compose文件中删除或注释掉harbor-db相关，修改完成后执行sh install.sh文件重启harbor服务即可。

三、测试验证

1. 数据库验证

​ 当harbor服务重启完成后，进入外部数据库中会发现刚才配置的库里面有了harbor的一些相关表。

[root@remote harbor]# psql -h x.x.x.x -p 7002 -U harbor -W 
Password: 
psql (12.2)
Type "help" for help.
# 查看有哪些库
harbor=> \l
                                       List of databases
         Name         |  Owner   | Encoding |   Collate   |    Ctype    |   Access privileges   
----------------------+----------+----------+-------------+-------------+-----------------------
 harbor               | postgres | UTF8     | en_US.UTF-8 | en_US.UTF-8 | =Tc/postgres         +
                      |          |          |             |             | postgres=CTc/postgres+
                      |          |          |             |             | harbor=CTc/postgres
 harbor_clair         | postgres | UTF8     | en_US.UTF-8 | en_US.UTF-8 | =Tc/postgres         +
                      |          |          |             |             | postgres=CTc/postgres+
                      |          |          |             |             | harbor=CTc/postgres
 harbor_notary_server | postgres | UTF8     | en_US.UTF-8 | en_US.UTF-8 | =Tc/postgres         +
                      |          |          |             |             | postgres=CTc/postgres+
                      |          |          |             |             | harbor=CTc/postgres
 harbor_notary_signer | postgres | UTF8     | en_US.UTF-8 | en_US.UTF-8 | =Tc/postgres         +
                      |          |          |             |             | postgres=CTc/postgres+
                      |          |          |             |             | harbor=CTc/postgres
 postgres             | postgres | UTF8     | en_US.UTF-8 | en_US.UTF-8 | 
 template0            | postgres | UTF8     | en_US.UTF-8 | en_US.UTF-8 | =c/postgres          +
                      |          |          |             |             | postgres=CTc/postgres
 template1            | postgres | UTF8     | en_US.UTF-8 | en_US.UTF-8 | =c/postgres          +
                      |          |          |             |             | postgres=CTc/postgres
(7 rows)
# 进入harbor库中
harbor=> \c harbor
Password for user harbor: 
You are now connected to database "harbor" as user "harbor".
# 查看该库有哪些表
harbor=> \dt
                 List of relations
 Schema |           Name           | Type  | Owner  
--------+--------------------------+-------+--------
 public | access                   | table | harbor
 public | access_log               | table | harbor
 public | admin_job                | table | harbor
 public | alembic_version          | table | harbor
 public | artifact                 | table | harbor
 public | artifact_blob            | table | harbor
 public | blob                     | table | harbor
 public | cve_whitelist            | table | harbor
 public | harbor_label             | table | harbor
 public | harbor_resource_label    | table | harbor
 public | harbor_user              | table | harbor
 public | immutable_tag_rule       | table | harbor
 public | job_log                  | table | harbor
 public | notification_job         | table | harbor
 public | notification_policy      | table | harbor
 public | oidc_user                | table | harbor
 public | project                  | table | harbor
 public | project_blob             | table | harbor
 public | project_member           | table | harbor
 public | project_metadata         | table | harbor
 public | properties               | table | harbor
 public | quota                    | table | harbor
 public | quota_usage              | table | harbor
 public | registry                 | table | harbor
 public | replication_execution    | table | harbor
 public | replication_policy       | table | harbor
 public | replication_schedule_job | table | harbor
 public | replication_task         | table | harbor
 public | repository               | table | harbor
 public | retention_execution      | table | harbor
 public | retention_policy         | table | harbor
 public | retention_task           | table | harbor
 public | robot                    | table | harbor
 public | role                     | table | harbor
 public | scan_report              | table | harbor
 public | scanner_registration     | table | harbor
 public | schedule                 | table | harbor
 public | schema_migrations        | table | harbor
 public | user_group               | table | harbor
(39 rows)

## 由于并没有开启其他三个组件的功能，所以其他三个库里面没有表，当harbor库中有表存在后，则外部数据库配置成功

2. Web页面测试

​ 根据之前的harbor搭建中最后的web页面创建镜像仓库的演示，可新创建一个镜像仓库，并上传一个镜像，完成后，在数据库中可看到记录

harbor_notary_signer=> \c harbor
Password for user harbor: 
You are now connected to database "harbor" as user "harbor".
# 查看操作日志，创建了一个pingcap仓库，并上传了一个tikv:v3.0.12的镜像到pingcap仓库中，所有的操作均为admin用户执行
harbor=> select * from access_log;
 log_id | username | project_id |  repo_name   | repo_tag | guid | operation |          op_time           
--------+----------+------------+--------------+----------+------+-----------+----------------------------
      1 | admin    |          2 | pingcap/     | N/A      |      | create    | 2020-04-08 18:02:50.369493
      2 | admin    |          2 | pingcap/tikv | v3.0.12  |      | push      | 2020-04-08 18:03:48.824079
(2 rows)
# 查看目前有哪些仓库，即project
harbor=> select * from project;   
 project_id | owner_id |  name   |       creation_time        |        update_time         | deleted 
------------+----------+---------+----------------------------+----------------------------+---------
          1 |        1 | library | 2020-04-08 17:48:10.024358 | 2020-04-08 17:48:10.024358 | f
          2 |        1 | pingcap | 2020-04-08 18:02:50        | 2020-04-08 18:02:50        | f
(2 rows)
# 查看目前有哪些镜像
harbor=> select * from repository;
 repository_id |     name     | project_id | description | pull_count | star_count |       creation_time        |        update_t
ime         
---------------+--------------+------------+-------------+------------+------------+----------------------------+----------------
------------
             1 | pingcap/tikv |          2 |             |          0 |          0 | 2020-04-08 18:03:48.824717 | 2020-04-08 18:0
3:48.824717
(1 row)

​ 可以看到，所有的结果均符合预期，harbor配置外部数据库及测试完成。^_^