#2499: svn http 502 bad gateway errors on MapGuide/FDO repos
---------------------------+-----------------------
Reporter: jng | Owner: sac@…
Type: defect | Status: new
Priority: normal | Milestone: Unplanned
Component: Systems Admin | Keywords:
---------------------------+-----------------------
I've been experiencing http 502 bad gateway errors on the MapGuide/FDO svn
repos.
Some examples:
1. Using the [https://tortoisesvn.net/ TortoiseSVN] windows client, trying
to bring up the log of any directory of a svn checkout within brings up
the bad gateway error (see attached image)
2. I've been experimenting with some git mirroring/migration scripts and
the MapGuide repo fails with bad gateway when I try to run
[https://github.com/nirvdrum/svn2git svn2git]
Example `svn2git` command:
{{{
svn2git https://svn.osgeo.org/mapguide --authors ~/svn_authors.txt
--metadata --no-minimize-url
}}}
It fails trying to pull out commit r1 with
{{{
RA layer request failed: Unexpected HTTP status 502 'Bad Gateway' on
'/mapguide/!svn/rvr/29/trunk/MgDev/UnitTest/TestData/FeatureService/FeatureServiceTest.dump'
at /usr/share/perl5/Git/SVN/Ra.pm line 312
}}}
For 1. I can somewhat workaround this by using the SVN command-line
client, where the bad gateway errors aren't happening, though I much
prefer to do merges through the TortoiseSVN GUI, so this problem hampers
by productivity somewhat.
For 2. this is a more pressing problem as I would one day like to see
MapGuide/FDO formally be migrated to git, but if I can't pull out the full
history of these repos because of bad gateway errors, that idea of git
mirroring/migration with full history is dead in the water.
I theorize this started happening when `http://svn.osgeo.org` was taken
down and everything redirected to `https://svn.osgeo.org`. My git
mirroring scripts worked then and I don't recall encountering bad gateway
errors with TortoiseSVN.
--
Ticket URL: <https://trac.osgeo.org/osgeo/ticket/2499>
OSGeo <https://osgeo.org/>
OSGeo committee and general foundation issue tracker.
#2499: svn http 502 bad gateway errors on MapGuide/FDO repos
---------------------------+------------------------
Reporter: jng | Owner: sac@…
Type: defect | Status: new
Priority: normal | Milestone: Unplanned
Component: Systems Admin | Resolution:
Keywords: |
---------------------------+------------------------
Changes (by jng):
* Attachment "2020-08-19 01_01_00-D__fdo-trunk - Log Messages -
TortoiseSVN.png" added.
TortoiseSVN bad gateway error
--
Ticket URL: <https://trac.osgeo.org/osgeo/ticket/2499>
OSGeo <https://osgeo.org/>
OSGeo committee and general foundation issue tracker.
#2499: svn http 502 bad gateway errors on MapGuide/FDO repos
---------------------------+---------------------------------------
Reporter: jng | Owner: sac@…
Type: defect | Status: new
Priority: normal | Milestone: Sysadmin Contract 2020-I
Component: Systems Admin | Resolution:
Keywords: |
---------------------------+---------------------------------------
Changes (by robe):
* milestone: Unplanned => Sysadmin Contract 2020-I
Comment:
I'll check this out later today.
--
Ticket URL: <https://trac.osgeo.org/osgeo/ticket/2499#comment:1>
OSGeo <https://osgeo.org/>
OSGeo committee and general foundation issue tracker.
#2499: svn http 502 bad gateway errors on MapGuide/FDO repos
---------------------------+---------------------------------------
Reporter: jng | Owner: sac@…
Type: defect | Status: new
Priority: normal | Milestone: Sysadmin Contract 2020-I
Component: Systems Admin | Resolution:
Keywords: |
---------------------------+---------------------------------------
Changes (by robe):
* cc: strk (added)
Comment:
Okay I just tried with my Windows Tortoise SVN
doing checkout of this
https://svn.osgeo.org/mapguide/
and was able and was going well pulled a ton of file but then I did
finally get a Bad Gateway:
Error: Unexpected HTTP status 502 'Bad Gateway' on
'/mapguide/!svn/rvr/8981/trunk/MgDev/UnitTest/TestData/FeatureService/FeatureServiceTest.dump'
Doing ShowLog from Windows explorer tortoise svn brings up the show log
fine though:
I see last revision 9720 - jng - August 18th, 2020
This might have to do with some measures strk put in a while back as I
think our svn was being hit a lot and suffering dos attacks. I'll check
with him to see if he remembers what he changed.
--
Ticket URL: <https://trac.osgeo.org/osgeo/ticket/2499#comment:2>
OSGeo <https://osgeo.org/>
OSGeo committee and general foundation issue tracker.
#2499: svn http 502 bad gateway errors on MapGuide/FDO repos
---------------------------+---------------------------------------
Reporter: jng | Owner: sac@…
Type: defect | Status: new
Priority: normal | Milestone: Sysadmin Contract 2020-I
Component: Systems Admin | Resolution:
Keywords: |
---------------------------+---------------------------------------
Comment (by strk):
There was an apache module, I think mod_evasive
or similar. Are we still using Apache ?
--
Ticket URL: <https://trac.osgeo.org/osgeo/ticket/2499#comment:3>
OSGeo <https://osgeo.org/>
OSGeo committee and general foundation issue tracker.
#2499: svn http 502 bad gateway errors on MapGuide/FDO repos
---------------------------+---------------------------------------
Reporter: jng | Owner: sac@…
Type: defect | Status: new
Priority: normal | Milestone: Sysadmin Contract 2020-I
Component: Systems Admin | Resolution:
Keywords: |
---------------------------+---------------------------------------
Comment (by robe):
Yes we still use apache for trac and svn though the apache traffic proxies
thru nginx. So don't discount nginx at fault.
--
Ticket URL: <https://trac.osgeo.org/osgeo/ticket/2499#comment:4>
OSGeo <https://osgeo.org/>
OSGeo committee and general foundation issue tracker.
#2499: svn http 502 bad gateway errors on MapGuide/FDO repos
---------------------------+---------------------------------------
Reporter: jng | Owner: sac@…
Type: defect | Status: new
Priority: normal | Milestone: Sysadmin Contract 2020-I
Component: Systems Admin | Resolution:
Keywords: |
---------------------------+---------------------------------------
Comment (by jng):
Just to add. The gateway error on TortoiseSVN log problem happens on the
FDO repo.
It doesn't happen for me on the MapGuide one, that gateway error happens
in a way described in example 2.
--
Ticket URL: <https://trac.osgeo.org/osgeo/ticket/2499#comment:5>
OSGeo <https://osgeo.org/>
OSGeo committee and general foundation issue tracker.
#2499: svn http 502 bad gateway errors on MapGuide/FDO repos
---------------------------+---------------------------------------
Reporter: jng | Owner: sac@…
Type: defect | Status: new
Priority: normal | Milestone: Sysadmin Contract 2020-I
Component: Systems Admin | Resolution:
Keywords: |
---------------------------+---------------------------------------
Comment (by robe):
hmm I get the same behavior with FDO and MapGuide using TortoiseSVN.
e.g. I can checkout and it goes for a while until it gives a bad gateway.
I can always see the log using the TortoiseSVN show log button. I'll try
to look thru this mod_evasive module strk mentioned. I'm guessing it has
to be some denial of service protection thing getting in the way and
perhaps windows GUI is just more aggressive than the commandline and thus
is more likely to trigger it.
--
Ticket URL: <https://trac.osgeo.org/osgeo/ticket/2499#comment:6>
OSGeo <https://osgeo.org/>
OSGeo committee and general foundation issue tracker.
#2499: svn http 502 bad gateway errors on MapGuide/FDO repos
---------------------------+-----------------------------------------
Reporter: jng | Owner: sac@…
Type: defect | Status: new
Priority: normal | Milestone: Sysadmin Contract 2020-III
Component: Systems Admin | Resolution:
Keywords: |
---------------------------+-----------------------------------------
Comment (by jng):
It seems it is no longer possible to do a clean SVN checkout of MapGuide.
It will inevitably fail with the 502 bad gateway on
`/mapguide/!svn/rvr/8981/trunk/MgDev/UnitTest/TestData/FeatureService/FeatureServiceTest.dump`
and no matter how many retries of `svn cleanup` and `svn update` to resume
it, it will never finish the incomplete working copy.
For FDO, I still get 502 bad gateway on fresh checkout, but I can retry
`svn cleanup` and `svn update` to repair the incomplete working copy to
completion.
--
Ticket URL: <https://trac.osgeo.org/osgeo/ticket/2499#comment:9>
OSGeo <https://osgeo.org/>
OSGeo committee and general foundation issue tracker.
#2499: svn http 502 bad gateway errors on MapGuide/FDO repos
---------------------------+-----------------------------------------
Reporter: jng | Owner: sac@…
Type: defect | Status: new
Priority: normal | Milestone: Sysadmin Contract 2020-III
Component: Systems Admin | Resolution:
Keywords: |
---------------------------+-----------------------------------------
Comment (by robe):
jng sorry unfortunately I haven't figured out why this is happening. I
did confirm I have the same issue too trying to do the pulls.
Not sure if it's the size of the pull or what.
Any chance you'd be amenable to moving to git. Seems most other OSGeo
users have moved on from svn. We can at very least do a mirror to see if
it's an issue with the server the way we have timeout configured or SVN
itself.
Both git.osgeo.org and SVN are housed on tracsvn container.
--
Ticket URL: <https://trac.osgeo.org/osgeo/ticket/2499#comment:10>
OSGeo <https://osgeo.org/>
OSGeo committee and general foundation issue tracker.
#2499: svn http 502 bad gateway errors on MapGuide/FDO repos
---------------------------+-----------------------------------------
Reporter: jng | Owner: sac@…
Type: defect | Status: new
Priority: normal | Milestone: Sysadmin Contract 2020-III
Component: Systems Admin | Resolution:
Keywords: |
---------------------------+-----------------------------------------
Comment (by robe):
Disregard my comment about git. I forgot your last bit about trying to
mirror to git.
I did find some misconfigurations in the nginx proxy which I fixed but
that didn't fix the issue.
I also tried to pull using the svn commandline that comes with Visual SVN
Server and it crapped out at the same spot
{{{
svn: E175002: Unexpected HTTP status 502 'Bad Gateway' on
'/mapguide/!svn/rvr/8981/trunk/MgDev/UnitTest/TestData/FeatureService/FeatureServiceTest.dump'
}}}
And svn update on fdo commandline fails on:
{{{
svn: E175002: Unexpected HTTP status 502 'Bad Gateway' on
'/fdo/!svn/rvr/7016/branches/3.9/Thirdparty/boost/boost/typeof/vector200.hpp'
}}}
I'm going to do an local checkout to see if I have same issue on server to
rule out corruption and then look at nginx config to see if I'm missing a
size set.
The vector200.hpp is 2.4 MB when I look thru the TortoiseSVN repo browser.
The mapguide when I got thru repo browser just crashes on me (that might
be my machine though)
--
Ticket URL: <https://trac.osgeo.org/osgeo/ticket/2499#comment:11>
OSGeo <https://osgeo.org/>
OSGeo committee and general foundation issue tracker.
#2499: svn http 502 bad gateway errors on MapGuide/FDO repos
---------------------------+-----------------------------------------
Reporter: jng | Owner: sac@…
Type: defect | Status: new
Priority: normal | Milestone: Sysadmin Contract 2020-III
Component: Systems Admin | Resolution:
Keywords: |
---------------------------+-----------------------------------------
Comment (by robe):
Sorry took so long to figure this out.
The nginx logs were showing the error so the issue was in nginx.
https://ma.ttias.be/nginx-proxy-upstream-sent-big-header-reading-response-
header-upstream/
I changed just the svn.osgeo.org conf for now but plan to change the main
one included in all the sites once I confirm this will not cause any
issues.
Please give another try and close this out if this fixed your issue.
--
Ticket URL: <https://trac.osgeo.org/osgeo/ticket/2499#comment:12>
OSGeo <https://osgeo.org/>
OSGeo committee and general foundation issue tracker.
#2499: svn http 502 bad gateway errors on MapGuide/FDO repos
---------------------------+-----------------------------------------
Reporter: jng | Owner: sac@…
Type: defect | Status: new
Priority: normal | Milestone: Sysadmin Contract 2020-III
Component: Systems Admin | Resolution:
Keywords: |
---------------------------+-----------------------------------------
Comment (by robe):
Committed at
https://git.osgeo.org/gitea/sac/osgeo7/commit/1ae14bd40390441cf542ec53626bfe0945423847
--
Ticket URL: <https://trac.osgeo.org/osgeo/ticket/2499#comment:13>
OSGeo <https://osgeo.org/>
OSGeo committee and general foundation issue tracker.
#2499: svn http 502 bad gateway errors on MapGuide/FDO repos
---------------------------+-----------------------------------------
Reporter: jng | Owner: sac@…
Type: defect | Status: closed
Priority: normal | Milestone: Sysadmin Contract 2020-III
Component: Systems Admin | Resolution: fixed
Keywords: |
---------------------------+-----------------------------------------
Changes (by jng):
* status: new => closed
* resolution: => fixed
Comment:
I can confirm that SVN checkouts of MapGuide/FDO/CS-Map now run to
completion.
I still get sproadic failures when attempting to use `svn2git` on these
repos, but I can re-run `svn2git` again afterwards and it continues the
sync to completion.
I can also confirm that I no longer get 502 bad gateway errors when
viewing SVN log history through TortoiseSVN.
So for all intents and purposes, I think this issue is resolved. Thank you
very much for solving this problem.
--
Ticket URL: <https://trac.osgeo.org/osgeo/ticket/2499#comment:14>
OSGeo <https://osgeo.org/>
OSGeo committee and general foundation issue tracker.