Bon, un pense bête plus qu'un post.

J'ai mon serveur Nextcloud et Collabora online qui sont derriere un reverse proxy Nginx. Ces 2 bidules necessite 2 ou 3 paramètres en plus sur la config de celui-ci et forcément, Nextcloud est moins pointilleux , voici un exemple:

server {
        server_name nextcloud.tonputinde.dns;
                location / {
                proxy_pass         https://192.168.0.XX/;
                proxy_set_header    Host $host;
        }
        location = /.well-known/carddav {
         return 301 $scheme://$host:$server_port/remote.php/dav;
        }
        location = /.well-known/caldav {
         return 301 $scheme://$host:$server_port/remote.php/dav;
        }

    listen [::]:443 ssl;
    listen 443 ssl;
    ssl_certificate /etc/tonputindecertificat.pem;
    ssl_certificate_key /etc/tonputindecertificatprivkey.pem;
    ssl_dhparam /etc/tonputindecertificat-dhparams.pem;

}

Voila pour Nextcloud, rien de bien compliqué. Pour Collabora c'est un peu plus long :

server {
        listen 9980 ssl;
        listen [::]:9980 ssl;
        server_name _;
        #location / {
        #proxy_pass     https://192.168.0.xx:9980/;
        #proxy_set_header    Host $host;
        #}

    ssl_certificate /etc/tonputindecertificat.pem;
    ssl_certificate_key /etc/tonputindecertificatprivkey.pem;
    ssl_dhparam /etc/tonputindecertificat-dhparams.pem;

#config reverse collabora
    # static files
    location ^~ /loleaflet {
        proxy_pass https://192.168.0.xx:9980;
        proxy_set_header Host $http_host;
    }

    # WOPI discovery URL
    location ^~ /hosting/discovery {
        proxy_pass https://192.168.0.xx:9980;
        proxy_set_header Host $http_host;
    }

    # Capabilities
    location ^~ /hosting/capabilities {
        proxy_pass https://192.168.0.xx:9980;
        proxy_set_header Host $http_host;
    }

    # main websocket
    location ~ ^/lool/(.*)/ws$ {
        proxy_pass https://192.168.0.xx:9980;
        proxy_set_header Upgrade $http_upgrade;
        proxy_set_header Connection "Upgrade";
        proxy_set_header Host $http_host;
        proxy_read_timeout 36000s;
    }

    # download, presentation and image upload
    location ~ ^/lool {
        proxy_pass https://192.168.0.xx:9980;
        proxy_set_header Host $http_host;
    }

    # Admin Console websocket
    location ^~ /lool/adminws {
        proxy_pass https://192.168.0.xx:9980;
        proxy_set_header Upgrade $http_upgrade;
        proxy_set_header Connection "Upgrade";
        proxy_set_header Host $http_host;
        proxy_read_timeout 36000s;
    }
#-------------------------------
}

Voila, en principe ça fonctionne.
La doc officiel :
https://www.collaboraoffice.com/code/nginx-reverse-proxy/
https://docs.nextcloud.com/server/latest/admin_manual/configuration_server/reverse_proxy_configuration.html