[OSGeo] #3104: geoserver-users SF mailman to discourse?

#3104: geoserver-users SF mailman to discourse?
----------------------+---------------------------
Reporter: jive | Owner: sac-tickets@…
     Type: task | Status: new
Priority: normal | Milestone: Unplanned
Component: SysAdmin | Keywords:
----------------------+---------------------------
The GeoServer team has expressed interest in using discourse; initially
for the user-list content: GeoServer Mailing Lists

Is it possible to move the geoserver-users email list from source forge
mailman to discourse?

It looks like an export is available for admins:
SourceForge Support / Documentation / Mailing List Archives

But it says:

> mbox archives contain only obfuscated email addresses

I wonder if it will work?!
--
Ticket URL: <https://trac.osgeo.org/osgeo/ticket/3104&gt;
OSGeo <Gter - OSGeo;
OSGeo committee and general foundation issue tracker.

#3104: geoserver-users SF mailman to discourse?
----------------------+----------------------------------------
Reporter: jive | Owner: sac-tickets@…
     Type: task | Status: new
Priority: normal | Milestone: Sysadmin Contract 2024-II
Component: SysAdmin | Resolution:
Keywords: |
----------------------+----------------------------------------
Changes (by cvvergara):

* milestone: Unplanned => Sysadmin Contract 2024-II

Comment:

@jive

Please send me the mailbox, I will set it up in staging for you to
evaluate.
--
Ticket URL: <#3104 (geoserver-users SF mailman to discourse?) – OSGeo;
OSGeo <Gter - OSGeo;
OSGeo committee and general foundation issue tracker.

#3104: geoserver-users SF mailman to discourse?
----------------------+----------------------------------------
Reporter: jive | Owner: sac-tickets@…
     Type: task | Status: new
Priority: normal | Milestone: Sysadmin Contract 2024-II
Component: SysAdmin | Resolution:
Keywords: |
----------------------+----------------------------------------
Comment (by jive):

I was unable to download following this link:

* https://lists.sourceforge.net/mbox/geoserver-users

The error was:

> The mailing-list mbox was not found.

I will ask another geoserver admin to try, and then we may have to open a
source forge support ticket.
--
Ticket URL: <#3104 (geoserver-users SF mailman to discourse?) – OSGeo;
OSGeo <Gter - OSGeo;
OSGeo committee and general foundation issue tracker.

#3104: geoserver-users SF mailman to discourse?
----------------------+----------------------------------------
Reporter: jive | Owner: sac-tickets@…
     Type: task | Status: new
Priority: normal | Milestone: Sysadmin Contract 2024-II
Component: SysAdmin | Resolution:
Keywords: |
----------------------+----------------------------------------
Comment (by jive):

Andrea has been able to obtained the mbox download from SF so this
activity can now be worked. Please contact Andrea for a file (from a few
weeks ago) that can be used for testing.

We can try it out, and then communicate the transition to our community.
--
Ticket URL: <#3104 (geoserver-users SF mailman to discourse?) – OSGeo;
OSGeo <Gter - OSGeo;
OSGeo committee and general foundation issue tracker.

#3104: geoserver-users SF mailman to discourse?
----------------------+----------------------------------------
Reporter: jive | Owner: sac-tickets@…
     Type: task | Status: new
Priority: normal | Milestone: Sysadmin Contract 2024-II
Component: SysAdmin | Resolution:
Keywords: |
----------------------+----------------------------------------
Comment (by aaime):

Anyone with admin rights can now download the mbox from the link above,
fresh of the day, when the migration is ready to go.
--
Ticket URL: <#3104 (geoserver-users SF mailman to discourse?) – OSGeo;
OSGeo <Gter - OSGeo;
OSGeo committee and general foundation issue tracker.

#3104: geoserver-users SF mailman to discourse?
----------------------+----------------------------------------
Reporter: jive | Owner: sac-tickets@…
     Type: task | Status: new
Priority: normal | Milestone: Sysadmin Contract 2024-II
Component: SysAdmin | Resolution:
Keywords: |
----------------------+----------------------------------------
Comment (by cvvergara):

@aaime

I do not have admin rights there.

I need someone to send me the mbox via mail.
vicky at erosion.dev
--
Ticket URL: <#3104 (geoserver-users SF mailman to discourse?) – OSGeo;
OSGeo <Gter - OSGeo;
OSGeo committee and general foundation issue tracker.

#3104: geoserver-users SF mailman to discourse?
----------------------+----------------------------------------
Reporter: jive | Owner: sac-tickets@…
     Type: task | Status: new
Priority: normal | Milestone: Sysadmin Contract 2024-II
Component: SysAdmin | Resolution:
Keywords: |
----------------------+----------------------------------------
Comment (by cvvergara):

Mailbox received
--
Ticket URL: <#3104 (geoserver-users SF mailman to discourse?) – OSGeo;
OSGeo <Gter - OSGeo;
OSGeo committee and general foundation issue tracker.

#3104: geoserver-users SF mailman to discourse?
----------------------+----------------------------------------
Reporter: jive | Owner: sac-tickets@…
     Type: task | Status: new
Priority: normal | Milestone: Sysadmin Contract 2024-II
Component: SysAdmin | Resolution:
Keywords: |
----------------------+----------------------------------------
Comment (by cvvergara):

I tried the import, basically getting errors on all mails

Short version:
{{{
...
Failed to index message in /shared/import/data/geoserver-users/geoserver-
users.mbox at lines 33-1899
...
Failed to index message in /shared/import/data/geoserver-users/geoserver-
users.mbox at lines 1900-1953
...
Failed to index message in /shared/import/data/geoserver-users/geoserver-
users.mbox at lines 1954-2056
...
Failed to index message in /shared/import/data/geoserver-users/geoserver-
users.mbox at lines 2057-2118
...
Failed to index message in /shared/import/data/geoserver-users/geoserver-
users.mbox at lines 2119-2220
...
}}}

The error between each "Failed to index"
{{{
undefined method `sub' for :from:Symbol
["/var/www/discourse/script/import_scripts/mbox/support/indexer.rb:91:in
`block in index_emails'",
"/var/www/discourse/script/import_scripts/mbox/support/indexer.rb:158:in
`block (2 levels) in all_messages'", "/
var/www/discourse/script/import_scripts/mbox/support/indexer.rb:190:in
`block in each_mail'",
"/var/www/discourse/script/import_scripts/mbox/support/indexer.rb:210:in
`block in each_line'", "/var/www/discourse/s
cript/import_scripts/mbox/support/indexer.rb:210:in `each_line'",
"/var/www/discourse/script/import_scripts/mbox/support/indexer.rb:210:in
`each_line'", "/var/www/discourse/script/import_scripts/mbox/support/ind
exer.rb:187:in `each_mail'",
"/var/www/discourse/script/import_scripts/mbox/support/indexer.rb:150:in
`block in all_messages'",
"/var/www/discourse/script/import_scripts/mbox/support/indexer.rb:143:in
`foreach'"
, "/var/www/discourse/script/import_scripts/mbox/support/indexer.rb:143:in
`all_messages'",
"/var/www/discourse/script/import_scripts/mbox/support/indexer.rb:67:in
`index_emails'", "/var/www/discourse/script/imp
ort_scripts/mbox/support/indexer.rb:25:in `block in execute'",
"/var/www/discourse/script/import_scripts/mbox/support/indexer.rb:22:in
`each'",
"/var/www/discourse/script/import_scripts/mbox/support/indexer.rb:2
2:in `execute'",
"/var/www/discourse/script/import_scripts/mbox/importer.rb:43:in
`index_messages'",
"/var/www/discourse/script/import_scripts/mbox/importer.rb:27:in
`execute'", "/var/www/discourse/script/import
_scripts/base.rb:47:in `perform'", "script/import_scripts/mbox.rb:13:in
`<module:Mbox>'", "script/import_scripts/mbox.rb:11:in
`<module:ImportScripts>'", "script/import_scripts/mbox.rb:10:in `<main>'"]
}}}

Example problematic header:
{{{
From aurelio@...3... Mon Jan 19 10:58:27 2004
Message-ID: <001f01c3dec6$d49f0cb0$6801a8c0@...4...>
From: =?iso-8859-1?Q?Aur=E9lio_Calegari?= <aurelio@...3...>
To: <geoserver-users@lists.sourceforge.net>
Date: Mon, 19 Jan 2004 17:00:04 -0300
MIME-Version: 1.0
}}}

An example of a header on an osgeo mailinglist
{{{
From
sentto-3377054-1-991426944-postgis=postgis.refractions.net@returns.onelist.com
Fri Jun 1 13:22:16 2001
Return-Path:
<sentto-3377054-1-991426944-postgis=postgis.refractions.net@returns.onelist.com>
Delivered-To: postgis@offsite.refractions.net
Received: from fj.egroups.com (fj.egroups.com [64.211.240.231])
         by offsite.refractions.net (Postfix) with SMTP id 4C17A43490
         for <postgis@postgis.refractions.net>; Fri, 1 Jun 2001 13:22:15
-0700 (PDT)
X-eGroups-Return:
sentto-3377054-1-991426944-postgis=postgis.refractions.net@returns.onelist.com
}}}

Probably the `aurelio@...3...`
{{{
From aurelio@...3... Mon Jan 19 10:58:27 2004
}}}

Doing a `grep '^From ' geoserver-users.mbox`
{{{
From andrea.aime@...84... Fri Sep 27 19:40:34 2013
From mihail12@...84... Sun Sep 29 17:34:14 2013
From peter.vonminden@...2631... Mon Sep 30 02:45:30 2013
From andrea.aime@...84... Mon Sep 30 06:13:50 2013
From maurobartolomeoli@...84... Mon Sep 30 07:02:51 2013
From paulius@...4984... Mon Sep 30 07:36:57 2013
From alessio.fabiani@...84... Mon Sep 30 07:44:47 2013
From paulius@...4984... Mon Sep 30 08:01:16 2013
From andrea.aime@...84... Mon Sep 30 08:09:38 2013
}}}
--
Ticket URL: <#3104 (geoserver-users SF mailman to discourse?) – OSGeo;
OSGeo <Gter - OSGeo;
OSGeo committee and general foundation issue tracker.

#3104: geoserver-users SF mailman to discourse?
----------------------+----------------------------------------
Reporter: jive | Owner: sac-tickets@…
     Type: task | Status: new
Priority: normal | Milestone: Sysadmin Contract 2024-II
Component: SysAdmin | Resolution:
Keywords: |
----------------------+----------------------------------------
Comment (by peterafrigis):

Hi Vicky

I have used sed to make the anonymised email addresses @...\d+... into a
valid address, to see if this solves the import problem. Could you please
try again with the file link that I emailed you?

Peter
--
Ticket URL: <#3104 (geoserver-users SF mailman to discourse?) – OSGeo;
OSGeo <Gter - OSGeo;
OSGeo committee and general foundation issue tracker.

#3104: geoserver-users SF mailman to discourse?
----------------------+----------------------------------------
Reporter: jive | Owner: sac-tickets@…
     Type: task | Status: new
Priority: normal | Milestone: Sysadmin Contract 2024-II
Component: SysAdmin | Resolution:
Keywords: |
----------------------+----------------------------------------
Comment (by cvvergara):

The mailbox has been successfully created on staging for demonstration
purposes

geoserver-users - OSGeo Discourse

TODO to finish up the demo:
- create corresponding group
- add suitable description
- create the admins

Note: the demo does not include sending/receiving mails that feature is
blocked on staging.
--
Ticket URL: <#3104 (geoserver-users SF mailman to discourse?) – OSGeo;
OSGeo <Gter - OSGeo;
OSGeo committee and general foundation issue tracker.

#3104: geoserver-users SF mailman to discourse?
----------------------+----------------------------------------
Reporter: jive | Owner: sac-tickets@…
     Type: task | Status: new
Priority: normal | Milestone: Sysadmin Contract 2024-II
Component: SysAdmin | Resolution:
Keywords: |
----------------------+----------------------------------------
Comment (by jive):

In trying out staging I had some questions -
https://discourse.staging.osgeo.org/t/about-the-geoserver-users-
category/14078

I seem to be unable to edit the category description?
--
Ticket URL: <#3104 (geoserver-users SF mailman to discourse?) – OSGeo;
OSGeo <Gter - OSGeo;
OSGeo committee and general foundation issue tracker.

#3104: geoserver-users SF mailman to discourse?
----------------------+----------------------------------------
Reporter: jive | Owner: sac-tickets@…
     Type: task | Status: new
Priority: normal | Milestone: Sysadmin Contract 2024-II
Component: SysAdmin | Resolution:
Keywords: |
----------------------+----------------------------------------
Comment (by robe):

Replying to [comment:10 jive]:
> In trying out staging I had some questions -
https://discourse.staging.osgeo.org/t/about-the-geoserver-users-
category/14078
>
> I seem to be unable to edit the category description?
We were discussing that yesterday. It's a limitation of discourse that
only discourse admins can edit category information metadata, though group
admins can edit most metadata about a group.

Not sure if there is a workaround for that.
--
Ticket URL: <#3104 (geoserver-users SF mailman to discourse?) – OSGeo;
OSGeo <Gter - OSGeo;
OSGeo committee and general foundation issue tracker.

#3104: geoserver-users SF mailman to discourse?
----------------------+----------------------------------------
Reporter: jive | Owner: sac-tickets@…
     Type: task | Status: new
Priority: normal | Milestone: Sysadmin Contract 2024-II
Component: SysAdmin | Resolution:
Keywords: |
----------------------+----------------------------------------
Comment (by jive):

The limitation is fine, we should document that the group description
needs to be provided when requesting a new category.

Please use the text from the staging environment above (cut and paste for
the links to work).

{{{
Welcome to the discussion category for GeoServer user community.

This category is administered by the GeoServer PSC.

* As an Open Source Geospatial Foundation project we require all
participants follow our [code of
conduct](geoserver/CODE_OF_CONDUCT.md at main · geoserver/geoserver · GitHub).
* This is a public channel and not to be used for the discussion of
security vulnerability and exploits. See [security
policy](Security Overview · geoserver/geoserver · GitHub) for instructions
on responsible disclosure.
* Being an open community requires public communication, allowing
questions to be shared. Chances are someone else has the same question and
we all benefit from public discussion.
* The availability of a public forum means that sending email to an
individual developer is discouraged. If you are unable to communicate in
public [commercial support providers](Support - GeoServer) are
listed on our website.
* This forum is for use of the GeoServer software, development chatter is
[directed elsewhere](Communication - GeoServer Development).

To use this category as a mailing list first register and then use
geoserver-users@discourse.osgeo.org for communication.
}}}

The GeoServer [GSIP 225 · geoserver/geoserver Wiki · GitHub
proposal] is now approved.

How do we obtain and provide you with a new processed SF mailman dump?
--
Ticket URL: <#3104 (geoserver-users SF mailman to discourse?) – OSGeo;
OSGeo <Gter - OSGeo;
OSGeo committee and general foundation issue tracker.

#3104: geoserver-users SF mailman to discourse?
----------------------+----------------------------------------
Reporter: jive | Owner: sac-tickets@…
     Type: task | Status: new
Priority: normal | Milestone: Sysadmin Contract 2024-II
Component: SysAdmin | Resolution:
Keywords: |
----------------------+----------------------------------------
Comment (by robe):

@jive,

For what you have done above, I think that is generally done as a post on
the category and then pinned so it doesn't fall off the top. That is
something you can do yourself.

Kind of like what we have for pgrouting-dev category -
pgrouting-dev - OSGeo Discourse

The about post stays at the top.

The metadata I was talking about is like the logo and a very brief
description, kind of like you see on QGIS-fr

QGIS-fr-user - OSGeo Discourse

and that shows in the qgis category screen -
QGIS - OSGeo Discourse
--
Ticket URL: <#3104 (geoserver-users SF mailman to discourse?) – OSGeo;
OSGeo <Gter - OSGeo;
OSGeo committee and general foundation issue tracker.

#3104: geoserver-users SF mailman to discourse?
----------------------+----------------------------------------
Reporter: jive | Owner: sac-tickets@…
     Type: task | Status: new
Priority: normal | Milestone: Sysadmin Contract 2024-II
Component: SysAdmin | Resolution:
Keywords: |
----------------------+----------------------------------------
Comment (by jive):

Okay it is time to go, we have some instructions outlined, and once the
discourse channel is ready we will take screen shots, update our website
communication information, and share with our users.

Q: is `geoserver`/`geoserver-users` subcategory necessary to can the
structure be `geosever` / `users` ? and be clear enough?

Based on discussion I assume if the police changes to introduce a
"projects" or "osgeo projects" top-level category that things will shuffle
around a bit on us.
--
Ticket URL: <#3104 (geoserver-users SF mailman to discourse?) – OSGeo;
OSGeo <Gter - OSGeo;
OSGeo committee and general foundation issue tracker.

#3104: geoserver-users SF mailman to discourse?
----------------------+----------------------------------------
Reporter: jive | Owner: sac-tickets@…
     Type: task | Status: new
Priority: normal | Milestone: Sysadmin Contract 2024-II
Component: SysAdmin | Resolution:
Keywords: |
----------------------+----------------------------------------
Comment (by robe):

Replying to [comment:14 jive]:
> Okay it is time to go, we have some instructions outlined, and once the
discourse channel is ready we will take screen shots, update our website
communication information, and share with our users.
>
> * `geoserver`/`user`
>
> Notes:
>
> * You can let me know if `geosever` / `geoserver-users` is required for
disambiguation
>

I think you can do geoserver/users if you want.
I think the main impact I would be worried about is in the email subject
line.

Right now we have that set to: %{optional_cat}%{topic_title}

I had thought that optional_cat was just the category name, but now I
realize it includes the parent category
so maybe just having it called users is better cause then the subject line
would be shorter.

https://meta.discourse.org/t/superseded-customize-the-subject-format-for-
standard-emails/20801

> * I assume if the police changes to introduce a "projects" or "osgeo
projects" top-level category that things will shuffle around a bit on us.

I haven't looked at the email subject line of a category 3 levels deep -
we do have GFOSS.it - OSGeo Discourse which
is 3 levels deep. As stated this is something discourse discourages and I
had changed the config to allow it cause I figured we'd need it for what
you are describing. Again I'd be concerned of the subject line.

Speaking of which all the mirrors I set up for sac, I put SAC on them just
to replicate how the mailman mailing lists works, but probably a good test
to take that off. So I'll do that now and see how the subject lines look.
--
Ticket URL: <#3104 (geoserver-users SF mailman to discourse?) – OSGeo;
OSGeo <Gter - OSGeo;
OSGeo committee and general foundation issue tracker.

#3104: geoserver-users SF mailman to discourse?
----------------------+----------------------------------------
Reporter: jive | Owner: sac-tickets@…
     Type: task | Status: new
Priority: normal | Milestone: Sysadmin Contract 2024-II
Component: SysAdmin | Resolution:
Keywords: |
----------------------+----------------------------------------
Comment (by jive):

Thanks for experimenting with category names, advise what to do and I will
update the geoserver and geonetwork planning accordingly.

I made an export of the mbox yesterday ...resulting in a 1.3G text file.
What is the best way to provide that to you?
--
Ticket URL: <https://trac.osgeo.org/osgeo/ticket/3104#comment:16&gt;
OSGeo <Gter - OSGeo;
OSGeo committee and general foundation issue tracker.

#3104: geoserver-users SF mailman to discourse?
----------------------+----------------------------------------
Reporter: jive | Owner: sac-tickets@…
     Type: task | Status: new
Priority: normal | Milestone: Sysadmin Contract 2024-II
Component: SysAdmin | Resolution:
Keywords: |
----------------------+----------------------------------------
Comment (by robe):

Replying to [comment:16 jive]:
> Thanks for experimenting with category names, advise what to do and I
will update the geoserver and geonetwork planning accordingly.
>
> * option top-level minimal: `geoserver` / `user`
> * option top-level redundant: `geoserver` / `geoserver-user`
> * option nested minimal: `project` / `geoserver` / `user`
> * option nested redundant: `osgeo project` / `geoserver` / `geoserver-
user`
>

I'd start with the short version as you stated. I'm still feeling iffy
about burying projects under project as that affects more than geoserver
and QGIS main listings are planning to be coming on board in the next
month or 2.

I feel like for now we should go with:

* option top-level minimal: `geoserver` / `user`

and move one level down after further discussions with all project
stakeholders, which right now is just
QGIS, geoserver, and pgrouting

One thing I did notice as another consequence of going 3 levels, is the
description on the home page only shows for the uppest parent level even
if subparents have descriptions. I haven't investigated if there is a fix
for that.

Like if you look at SAC sub category, it has no description showing on
homepage though it does have a description.
So we might loose that for pgRouting and QGIS, their descriptions if they
are buried one level down in a project folder.

I also haven't investigated other ways for organizing the home page like
custom plugins etc. I think there might be many more we haven't
considered. Right now we are using the categories page like a lazy
homepage. It would be great if someone could take on that work to figure
out our options, cause I'd rather focus on system administration than UI
look and feel which I know I suck at.

> I made an export of the mbox yesterday ...resulting in a 1.3G text file.
What is the best way to provide that to you?
>
If it's the same format as you gave @cvvergara last time, it should work.
I think she was able to make sense of the part obfuscated email addresses.
I'll leave it up to her to respond.

> Notes:
> - I tend to prefer singular (`project`,`user`) rather than plural
(`projects`, `users`)
> - We may of already lost any hope of consistency, can categories be
renamed?

Yes as mentioned categories can be renamed easily cause internally
discourse just works on category id numbers.
In fact I think Vicky has to in order to load the mailing lists, put them
on the root and then rename them and move them down the category hierarchy
where they belong cause the mbox importer uses the mbox to arrive at the
name.

However if you start with one name and decide to change later, I think we
might need to put redirects in (which I recall there being a page for that
within discourse) so that if people
were navigating via the UI, they will be redirected to the new location.
--
Ticket URL: <#3104 (geoserver-users SF mailman to discourse?) – OSGeo;
OSGeo <Gter - OSGeo;
OSGeo committee and general foundation issue tracker.

#3104: geoserver-users SF mailman to discourse?
----------------------+----------------------------------------
Reporter: jive | Owner: sac-tickets@…
     Type: task | Status: new
Priority: normal | Milestone: Sysadmin Contract 2024-II
Component: SysAdmin | Resolution:
Keywords: |
----------------------+----------------------------------------
Comment (by cvvergara):

As I write I forwarded the mail that contains the way I got the
information for the test on staging.discourse

The load happens at the root, then the category needs to be moved to the
final destination.

Steps:
Create the geoserver category
   - logo (send me a link for the logo to use, I can use the logo on
staging)
   - description of the category
   - security: everyone can: see (no Reply no create)
Create geoserver-user category
   - Jody and Andrea as owners

The "user" category will be created on the root when importing
   - Once its created to avoid mishaps
     - security: `everyone` can: See (no Reply no create)
Move "user" under geoserver category
   - logo (send me a link for the logo to use if different from the above)
   - description of the category
   - security:
     - `everyone` can: See (no Reply no create) or can See, Reply (no
Create)
     - `geoserver-user` can See, Reply, Create

About the 3 levels, its new! I could not do it the last time I tried.
From other tickets, grouping categories like "Local Chapters",
"Initiatives", "GeoInquietos" have been created.

We are still the "few" in discourse.

So, lets focus on this ticket about GeoServer here, and open a new ticket
about reorganizing the categories. That one can take care of the
consequences of the changes (like redirection).

PS. Personally I like to see pgRouting on the root
--
Ticket URL: <#3104 (geoserver-users SF mailman to discourse?) – OSGeo;
OSGeo <Gter - OSGeo;
OSGeo committee and general foundation issue tracker.

#3104: geoserver-users SF mailman to discourse?
----------------------+----------------------------------------
Reporter: jive | Owner: sac-tickets@…
     Type: task | Status: new
Priority: normal | Milestone: Sysadmin Contract 2024-II
Component: SysAdmin | Resolution:
Keywords: |
----------------------+----------------------------------------
Comment (by jive):

branding is here: Home · geoserver/geoserver Wiki · GitHub
Branding
--
Ticket URL: <#3104 (geoserver-users SF mailman to discourse?) – OSGeo;
OSGeo <Gter - OSGeo;
OSGeo committee and general foundation issue tracker.