[OSGeo] #3370: gitea disable tarball download

#3370: gitea disable tarball download
----------------------------+----------------------
Reporter: robe | Owner: robe
     Type: task | Status: assigned
Priority: normal | Milestone:
Component: SysAdmin/Gitea | Keywords:
----------------------------+----------------------
Been discussion this on matrix, that we should disable tarball download
from gitea.

When gitea generates tarballs for repos, it generally does this real time
unless there is an archive. This takes resources and I think sometimes
causes a DDOS when some crawler like those recent AI crawlers attack.

It also takes up disk space and snapshot space if these aren't purged
quick enough.

We discussed there is not much need for people to download tarballs from
gitea, since all the relevant ones can be found on download.osgeo.org,
gitlab or github.

I'll leave this open for a bit in case anyone wants to comment otherwise.

It's an easy change to undo.
--
Ticket URL: <https://trac.osgeo.org/osgeo/ticket/3370&gt;
OSGeo <Gter - OSGeo;
OSGeo committee and general foundation issue tracker.

#3370: gitea disable tarball download
----------------------------+-----------------------
Reporter: robe | Owner: robe
     Type: task | Status: assigned
Priority: normal | Milestone:
Component: SysAdmin/Gitea | Resolution:
Keywords: |
----------------------------+-----------------------
Comment (by strk):

Isn't that a job for robots.txt ?
--
Ticket URL: <#3370 (gitea disable tarball download) – OSGeo;
OSGeo <Gter - OSGeo;
OSGeo committee and general foundation issue tracker.

#3370: gitea disable tarball download
----------------------------+-----------------------
Reporter: robe | Owner: robe
     Type: task | Status: assigned
Priority: normal | Milestone:
Component: SysAdmin/Gitea | Resolution:
Keywords: |
----------------------------+-----------------------
Comment (by lnicola):

Not just space, but also a lot of CPU usage IIRC. I just tried to download
one for PostGIS and it took about 30 seconds, and I didn't even get the
ZIP for some reason (GET request finished with a 15 MB payload, but no
download).
--
Ticket URL: <#3370 (gitea disable tarball download) – OSGeo;
OSGeo <Gter - OSGeo;
OSGeo committee and general foundation issue tracker.

#3370: gitea disable tarball download
----------------------------+-----------------------
Reporter: robe | Owner: robe
     Type: task | Status: assigned
Priority: normal | Milestone:
Component: SysAdmin/Gitea | Resolution:
Keywords: |
----------------------------+-----------------------
Comment (by lnicola):

No, very few robots are nice enough to respect that. Looks like the
download is a POST, but bots were still calling it, which is also not very
nice.
--
Ticket URL: <#3370 (gitea disable tarball download) – OSGeo;
OSGeo <Gter - OSGeo;
OSGeo committee and general foundation issue tracker.

#3370: gitea disable tarball download
----------------------------+-----------------------
Reporter: robe | Owner: robe
     Type: task | Status: assigned
Priority: normal | Milestone:
Component: SysAdmin/Gitea | Resolution:
Keywords: |
----------------------------+-----------------------
Comment (by lnicola):

I mentioned this on IRC, but to download an archive you need to get a CSRF
token from a `<script>` block in the page, do a POST request with it, then
come back later (not sure when) and do a GET to actually download the
file.

I don't think anyone is using this on CI.
--
Ticket URL: <#3370 (gitea disable tarball download) – OSGeo;
OSGeo <Gter - OSGeo;
OSGeo committee and general foundation issue tracker.

#3370: gitea disable tarball download
----------------------------+---------------------
Reporter: robe | Owner: robe
     Type: task | Status: closed
Priority: normal | Milestone:
Component: SysAdmin/Gitea | Resolution: fixed
Keywords: |
----------------------------+---------------------
Changes (by robe):

* resolution: => fixed
* status: assigned => closed

Comment:

I applied @lnicola pull request and ran

on both dev and production
{{{
make check-gitea
make deploy-gitea
}}}

On production it made changes to osgeo7-nginx as well it seems

I put in a diff clause in the check and deploy so I could see what changes
were being made
I assume the change was just to move stuff into a snippet file

{{{
@@ -20,6 +20,7 @@
  # Global maximum creation limit of repository per user
  MAX_CREATION_LIMIT = 40
  PREFERRED_LICENSES = AGPL-3.0,GPL-3.0,LGPL-3.0
+DISABLE_DOWNLOAD_SOURCE_ARCHIVES = true

  [server]
  DOMAIN = git.osgeo.org

changed: [osgeo7_tracsvn]

TASK [gitea : unset secrets]
****************************************************************************************************************************************************************************************************
ok: [osgeo7_tracsvn]

TASK [gitea : Install snippets/gitea.conf file on osgeo7_nginx]
*****************************************************************************************************************************************************************
--- before
+++ after: /home/robe/.ansible/tmp/ansible-local-
139987zz3fe5fx/tmpn09hgd8r/nginx-snippet-gitea.j2
@@ -0,0 +1,15 @@
+# WARNING: This file is managed by ansible,
+# manual changes may be reverted, see
+# https://git.osgeo.org/gitea/sac/ansible-deployment
+
+# First attempt to serve request as file, then
+# as directory, then fall back to displaying a 404.
+#try_files $uri $uri/ =404;
+if (-f $document_root/under_maintenance_gitea.html) {
+ return 503;
+}
+client_max_body_size 0;
+include /etc/nginx/proxy_protocol_params;
+proxy_pass http://tracsvn.lxd:3001/;
+proxy_redirect off;
+

changed: [osgeo7_tracsvn -> osgeo7_nginx(osgeo7.osgeo.osuosl.org)]

TASK [gitea : Install _git.osgeo.org.ansible file on osgeo7_nginx]
**************************************************************************************************************************************************************
--- before: /etc/nginx/sites-available/_git.osgeo.org.ansible
+++ after: /home/robe/.ansible/tmp/ansible-local-
139987zz3fe5fx/tmprymlc1zo/nginx-site-git.j2
@@ -11,16 +11,7 @@
  error_log /var/log/nginx/git.osgeo.org.error_log info;

  location /gitea/ {
- # First attempt to serve request as file, then
- # as directory, then fall back to displaying a 404.
- #try_files $uri $uri/ =404;
- if (-f $document_root/under_maintenance_gitea.html) {
- return 503;
- }
- client_max_body_size 0;
- include /etc/nginx/proxy_protocol_params;
- proxy_pass http://tracsvn.lxd:3001/;
- proxy_redirect off;
+ include snippets/gitea.conf;
  }

  location = /robots.txt {

changed: [osgeo7_tracsvn -> osgeo7_nginx(osgeo7.osgeo.osuosl.org)]

TASK [gitea : Install _gitea.osgeo.org.ansible file on osgeo7_nginx]
************************************************************************************************************************************************************
--- before: /etc/nginx/sites-available/_gitea.osgeo.org.ansible
+++ after: /home/robe/.ansible/tmp/ansible-local-
139987zz3fe5fx/tmpgvxaeh_v/nginx-site-gitea.j2
@@ -8,21 +8,10 @@

  access_log /var/log/nginx/gitea.osgeo.org.access_log pcombined;
  error_log /var/log/nginx/gitea.osgeo.org.error_log info;
-rewrite_log on;
-#rewrite ^(.*)$ /gitea/$1? break;

  location /gitea/ {
- # First attempt to serve request as file, then
- # as directory, then fall back to displaying a 404.
- #try_files $uri $uri/ =404;
- if (-f $document_root/under_maintenance_gitea.html) {
- return 503;
- }
- client_max_body_size 0;
- include /etc/nginx/proxy_protocol_params;
- proxy_pass http://tracsvn.lxd:3001/;
- proxy_redirect off;
+ include snippets/gitea.conf;
  }
}}}
--
Ticket URL: <#3370 (gitea disable tarball download) – OSGeo;
OSGeo <Gter - OSGeo;
OSGeo committee and general foundation issue tracker.