nginx反向代理部署本地docker registry

1. docker registry部署

通过拉取官方镜像部署,以下命令中的podman需要提前安装,或者使用docker cli

  • 镜像拉取

    podman pull registry
  • 启动镜像

    podman run -d --restart always --name registry \
    -v /data/local-registry:/var/lib/registry \
    -e REGISTRY_HTTP_HOST=https://docker.test.com \
    -p 5000:5000 \
    registry:latest
    • -v 定义本地存储镜像位置,格式为{本地路径}:{容器路径},容器内默认的镜像存储位置是/var/lib/registry
    • -e 设置环境变量,这里的REGISTRY_HTTP_HOST,主要用于设置registry的配置文件中的host属性,具体可以参考官方文档https://distribution.github.io/distribution/about/configuration/
      文档中所有的配置选项均可通过REGISTRY_XXX_XXX的方式来定义
    • -p 定义registry映射端口,默认为5000

启动后通过netstat命令确认5000端口是否正常监听

2. nginx反向代理部署

配置中的证书可以通过let's encrypt获取,对应参数自行调整。其中client_max_body_size及chunked_transfer_encoding参数需要关注一下,具体官方文档上也有相应的说明

upstream docker-registry {
  server 192.168.1.1:5000;
}

server {
    listen       443 ssl;
    server_name docker.test.com;

    ssl_certificate     /etc/letsencrypt/live/test.com/fullchain.pem;
    ssl_certificate_key /etc/letsencrypt/live/test.com/privkey.pem;

    ssl_protocols TLSv1.1 TLSv1.2;
    ssl_ciphers 'EECDH+AESGCM:EDH+AESGCM:AES256+EECDH:AES256+EDH';
    ssl_prefer_server_ciphers on;
    ssl_session_cache shared:SSL:10m;

    access_log /var/log/registry/access.log;
    error_log /var/log/registry/error.log warn;
    
    #关闭传输大小限制,防止传输大镜像时遇到http 413的错误,也可根据具体需要调整大小
    client_max_body_size 0;
    #防止http 411错误,具体参考 https://github.com/moby/moby/issues/1486
    chunked_transfer_encoding on;

    location /v2/ {
        add_header 'Docker-Distribution-Api-Version' 'registry/2.0' always;

        proxy_pass http://docker-registry;
        proxy_set_header  Host           $http_host;   # 防止出现blob unkown http 404错误
        proxy_set_header  X-Real-IP      $remote_addr;
        proxy_read_timeout               900;
        proxy_buffering off;
        proxy_request_buffering off;
    }
}

3. 常见问题处理

blob unkown 问题

  1. 推送镜像遇到以下错误

    podman push docker.test.com/defaultbackend-amd64:1.5
    Getting image source signatures
    Copying blob b108d4968233 [============================>---------] 3.8MiB / 4.9MiB
    Error: writing blob: uploading layer chunked: StatusCode: 404, <html>
    ...ad><title>404 Not Found</title></head>
  2. docker registry 日志提示 err.code="blob unknown" err

    time="2024-01-15T05:26:59.657630023Z" level=info msg="response completed" go.version=go1.20.8 http.request.host=docker.test.com http.request.id=882b97be-3422-44ef-a0d6-d0577cd6d8ec http.request.method=GET http.request.remoteaddr=192.168.1.1 http.request.uri="/v2/" http.request.useragent="containers/5.23.1 (github.com/containers/image)" http.response.contenttype="application/json; charset=utf-8" http.response.duration=2.690795ms http.response.status=200 http.response.written=2 
    10.0.2.100 - - [15/Jan/2024:05:26:59 +0000] "GET /v2/ HTTP/1.0" 200 2 "" "containers/5.23.1 (github.com/containers/image)"
    time="2024-01-15T05:26:59.678145536Z" level=error msg="response completed with error" err.code="blob unknown" err.
  3. 处理方法
    在启动容器是设置 -e REGISTRY_HTTP_HOST=https://docker.test.com,nginx配置文件中设置 proxy_set_header Host $http_host;

使用@digest拉取镜像http 500报错

从本地registry使用不带签名拉取镜像时正常,带上签名会得到http 500的错误,其实是验签失败了。google上查了一圈找到个issue提到docker计算签名会有不一致的情况,具体没有深入研究,目前这个issue还没有close。

https://github.com/docker/hub-feedback/issues/2043

Error response from daemon: received unexpected HTTP status: 500 Internal Server Error

可以使用google提供的gcrane来复制镜像到本地,项目地址 https://github.com/google/go-containerregistry

 gcrane copy registry.k8s.io/ingress-nginx/controller:v1.9.5 docker.test.com/ingress-nginx/controller:v1.9.5

这样也比pull、tag、push复制镜像到本地registry要来的方便

评论

暂无评论