[Geoserver-devel] Big security patch before RC

Hi all

During the last weeks I worked on hardening the new security system and finished some days ago. The result is a big patch and Justin pointed out that we have to discuss the patch because of the the time frame. The mistake I did was not splitting the changes into smaller chunks.

What is in the patch:

  1. Finishing the role concept. Since it is possible to have more than one role service, role names must be unique across role services. This is extremely important concerning access control.
  2. Refactoring code marked as deprecated. This was quite a mechanical work.
  3. A lot of bug fixes concerning the admin gui. The core code itself is stable since some weeks.

The patch itself is here
https://github.com/mcrmcr/geoserver-1/commit/3672723e7a5656c0ea0d64cf6b78a34088c71831

At a first look, the patch appears big, but there are no new concepts, a better description of the role concept is here
http://jira.codehaus.org/browse/GEOS-5101

The question is how to continue, two facts I want to point out

  • We cannot make a 2.2.0 release without the changes. The system would not work correctly.
  • My next steps would be to review/complete the security documentation and during this work, make a next round hardening the code.

Opinions ?.

If there are questions, pleas ask, I will answer ASAP, I spent about one year of work in the new security architecture.

Thanks in advance
Christian

On Fri, Jun 15, 2012 at 1:05 PM, Christian Mueller <mcrmcr21@anonymised.com> wrote:

The question is how to continue, two facts I want to point out

- We cannot make a 2.2.0 release without the changes. The system would not
work correctly.
- My next steps would be to review/complete the security documentation and
during this work, make a next round hardening the code.

Opinions ?.

The fact that we still need a 4400+loc patch to fix the security
subsystem tells me
whatever we release next week cannot possibly be a release candidate, but at
best a beta3, especially since you say that a next round of hardening is in
the plans: nothing bad about it per se, but bad that it's needed since RC means
Release Candidate, means we believe we're done and ask the users to
check and eventually tell us otherwise.

Hopefully GSIP 77 will bring some sanity into all of this.

Btw, I have no time to review the patch, I can have a look during the weekend
but my familiarity with the new authentication system is not enough.
I'll trust Justin's judgement on it unless my quick review really
finds some red flag.

Cheers
Andrea

--
Ing. Andrea Aime
GeoSolutions S.A.S.
Tech lead

Via Poggio alle Viti 1187
55054 Massarosa (LU)
Italy

phone: +39 0584 962313
fax: +39 0584 962313
mob: +39 339 8844549

http://www.geo-solutions.it
http://geo-solutions.blogspot.com/
http://www.youtube.com/user/GeoSolutionsIT
http://www.linkedin.com/in/andreaaime
http://twitter.com/geowolf

Hi Andrea,

Yes, imho GSIP 77 will improve the situation. Big +1 for the proposal. The whole new security system is a monster change and I do not want to have a Geoserver release with an inconsistent security system. Let us wait for Justins opinion, I would vote for your proposal having a beta 3.

Christian

2012/6/15 Andrea Aime <andrea.aime@anonymised.com>

On Fri, Jun 15, 2012 at 1:05 PM, Christian Mueller <mcrmcr21@anonymised.com> wrote:

The question is how to continue, two facts I want to point out

  • We cannot make a 2.2.0 release without the changes. The system would not
    work correctly.
  • My next steps would be to review/complete the security documentation and
    during this work, make a next round hardening the code.

Opinions ?.

The fact that we still need a 4400+loc patch to fix the security
subsystem tells me
whatever we release next week cannot possibly be a release candidate, but at
best a beta3, especially since you say that a next round of hardening is in
the plans: nothing bad about it per se, but bad that it’s needed since RC means
Release Candidate, means we believe we’re done and ask the users to
check and eventually tell us otherwise.

Hopefully GSIP 77 will bring some sanity into all of this.

Btw, I have no time to review the patch, I can have a look during the weekend
but my familiarity with the new authentication system is not enough.
I’ll trust Justin’s judgement on it unless my quick review really
finds some red flag.

Cheers
Andrea


Ing. Andrea Aime
GeoSolutions S.A.S.
Tech lead

Via Poggio alle Viti 1187
55054 Massarosa (LU)
Italy

phone: +39 0584 962313
fax: +39 0584 962313
mob: +39 339 8844549

http://www.geo-solutions.it
http://geo-solutions.blogspot.com/
http://www.youtube.com/user/GeoSolutionsIT
http://www.linkedin.com/in/andreaaime
http://twitter.com/geowolf

While its tempting to say, yeah sure let’s get this fix in, my better judgement tells me that we should wait. Nothing is ever going to be perfect unfortunately and we could go on fixing issues forever if we really wanted to. I haven’t reviewed the patch in detail yet but not all the parts are trivial. Christian I sent you some questions i had. And whose to say that this fix won’t lead to another issue and delay the release. Things are “quiet” right now so I think we should move on it.

So, long story short, unless this fix is a blocker that severely prevents usage, or not doing it now will make it impossible to do cleanly in a later release I would say let’s stick with the plan of RC1.

There is also the folks at CSIRO to consider here. Ben went to his management and was able to get resources because we had a date we had agreed upon. Changing that date now is not really fair so imo there should be a very very good reason.

$0.02

On Fri, Jun 15, 2012 at 7:50 AM, Christian Mueller <mcrmcr21@anonymised.com> wrote:

Hi Andrea,

Yes, imho GSIP 77 will improve the situation. Big +1 for the proposal. The whole new security system is a monster change and I do not want to have a Geoserver release with an inconsistent security system. Let us wait for Justins opinion, I would vote for your proposal having a beta 3.

Christian

2012/6/15 Andrea Aime <andrea.aime@anonymised.com…>

On Fri, Jun 15, 2012 at 1:05 PM, Christian Mueller <mcrmcr21@anonymised.com> wrote:

The question is how to continue, two facts I want to point out

  • We cannot make a 2.2.0 release without the changes. The system would not
    work correctly.
  • My next steps would be to review/complete the security documentation and
    during this work, make a next round hardening the code.

Opinions ?.

The fact that we still need a 4400+loc patch to fix the security
subsystem tells me
whatever we release next week cannot possibly be a release candidate, but at
best a beta3, especially since you say that a next round of hardening is in
the plans: nothing bad about it per se, but bad that it’s needed since RC means
Release Candidate, means we believe we’re done and ask the users to
check and eventually tell us otherwise.

Hopefully GSIP 77 will bring some sanity into all of this.

Btw, I have no time to review the patch, I can have a look during the weekend
but my familiarity with the new authentication system is not enough.
I’ll trust Justin’s judgement on it unless my quick review really
finds some red flag.

Cheers
Andrea


Ing. Andrea Aime
GeoSolutions S.A.S.
Tech lead

Via Poggio alle Viti 1187
55054 Massarosa (LU)
Italy

phone: +39 0584 962313
fax: +39 0584 962313
mob: +39 339 8844549

http://www.geo-solutions.it
http://geo-solutions.blogspot.com/
http://www.youtube.com/user/GeoSolutionsIT
http://www.linkedin.com/in/andreaaime
http://twitter.com/geowolf


Live Security Virtual Conference
Exclusive live event will cover all the ways today’s security and
threat landscape has changed and how IT managers can respond. Discussions
will include endpoint security, mobile security and the latest in malware
threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/


Geoserver-devel mailing list
Geoserver-devel@anonymised.comsts.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geoserver-devel


Justin Deoliveira
OpenGeo - http://opengeo.org
Enterprise support for open source geospatial.

Is it feasible to reduce the size of the patch by including only the consistency fixes, and shipping a GeoServer 2.2 release that knowingly contains UI bugs and uses deprecated (but presumed working) code?


David Winslow
OpenGeo - http://opengeo.org/

On Fri, Jun 15, 2012 at 9:50 AM, Christian Mueller <mcrmcr21@anonymised.com> wrote:

Hi Andrea,

Yes, imho GSIP 77 will improve the situation. Big +1 for the proposal. The whole new security system is a monster change and I do not want to have a Geoserver release with an inconsistent security system. Let us wait for Justins opinion, I would vote for your proposal having a beta 3.

Christian

2012/6/15 Andrea Aime <andrea.aime@anonymised.com…>

On Fri, Jun 15, 2012 at 1:05 PM, Christian Mueller <mcrmcr21@anonymised.com> wrote:

The question is how to continue, two facts I want to point out

  • We cannot make a 2.2.0 release without the changes. The system would not
    work correctly.
  • My next steps would be to review/complete the security documentation and
    during this work, make a next round hardening the code.

Opinions ?.

The fact that we still need a 4400+loc patch to fix the security
subsystem tells me
whatever we release next week cannot possibly be a release candidate, but at
best a beta3, especially since you say that a next round of hardening is in
the plans: nothing bad about it per se, but bad that it’s needed since RC means
Release Candidate, means we believe we’re done and ask the users to
check and eventually tell us otherwise.

Hopefully GSIP 77 will bring some sanity into all of this.

Btw, I have no time to review the patch, I can have a look during the weekend
but my familiarity with the new authentication system is not enough.
I’ll trust Justin’s judgement on it unless my quick review really
finds some red flag.

Cheers
Andrea


Ing. Andrea Aime
GeoSolutions S.A.S.
Tech lead

Via Poggio alle Viti 1187
55054 Massarosa (LU)
Italy

phone: +39 0584 962313
fax: +39 0584 962313
mob: +39 339 8844549

http://www.geo-solutions.it
http://geo-solutions.blogspot.com/
http://www.youtube.com/user/GeoSolutionsIT
http://www.linkedin.com/in/andreaaime
http://twitter.com/geowolf


Live Security Virtual Conference
Exclusive live event will cover all the ways today’s security and
threat landscape has changed and how IT managers can respond. Discussions
will include endpoint security, mobile security and the latest in malware
threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/


Geoserver-devel mailing list
Geoserver-devel@anonymised.comsts.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geoserver-devel

Good point David. That is certainly one of the issues with the patch, in that it lumps a number of changes together. If it were broken up i would be fine with saying parts of it could go in and wait on only smaller parts of it.

On Fri, Jun 15, 2012 at 8:10 AM, David Winslow <dwinslow@anonymised.com> wrote:

Is it feasible to reduce the size of the patch by including only the consistency fixes, and shipping a GeoServer 2.2 release that knowingly contains UI bugs and uses deprecated (but presumed working) code?


David Winslow
OpenGeo - http://opengeo.org/

On Fri, Jun 15, 2012 at 9:50 AM, Christian Mueller <mcrmcr21@anonymised.com> wrote:

Hi Andrea,

Yes, imho GSIP 77 will improve the situation. Big +1 for the proposal. The whole new security system is a monster change and I do not want to have a Geoserver release with an inconsistent security system. Let us wait for Justins opinion, I would vote for your proposal having a beta 3.

Christian

2012/6/15 Andrea Aime <andrea.aime@anonymised.com>

On Fri, Jun 15, 2012 at 1:05 PM, Christian Mueller <mcrmcr21@anonymised.com> wrote:

The question is how to continue, two facts I want to point out

  • We cannot make a 2.2.0 release without the changes. The system would not
    work correctly.
  • My next steps would be to review/complete the security documentation and
    during this work, make a next round hardening the code.

Opinions ?.

The fact that we still need a 4400+loc patch to fix the security
subsystem tells me
whatever we release next week cannot possibly be a release candidate, but at
best a beta3, especially since you say that a next round of hardening is in
the plans: nothing bad about it per se, but bad that it’s needed since RC means
Release Candidate, means we believe we’re done and ask the users to
check and eventually tell us otherwise.

Hopefully GSIP 77 will bring some sanity into all of this.

Btw, I have no time to review the patch, I can have a look during the weekend
but my familiarity with the new authentication system is not enough.
I’ll trust Justin’s judgement on it unless my quick review really
finds some red flag.

Cheers
Andrea


Ing. Andrea Aime
GeoSolutions S.A.S.
Tech lead

Via Poggio alle Viti 1187
55054 Massarosa (LU)
Italy

phone: +39 0584 962313
fax: +39 0584 962313
mob: +39 339 8844549

http://www.geo-solutions.it
http://geo-solutions.blogspot.com/
http://www.youtube.com/user/GeoSolutionsIT
http://www.linkedin.com/in/andreaaime
http://twitter.com/geowolf


Live Security Virtual Conference
Exclusive live event will cover all the ways today’s security and
threat landscape has changed and how IT managers can respond. Discussions
will include endpoint security, mobile security and the latest in malware
threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/


Geoserver-devel mailing list
Geoserver-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geoserver-devel


Live Security Virtual Conference
Exclusive live event will cover all the ways today’s security and
threat landscape has changed and how IT managers can respond. Discussions
will include endpoint security, mobile security and the latest in malware
threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/


Geoserver-devel mailing list
Geoserver-devel@anonymised.comsts.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geoserver-devel


Justin Deoliveira
OpenGeo - http://opengeo.org
Enterprise support for open source geospatial.

Hi David, Justin, Andrea, …

sorry for all these inconveniences. During eliminating the deprecations I also fixed some bugs. The problem is that within the last month my work was always interrupted. (Finishing my Master Thesis, emergency situation at my customer). Splitting the patch in smaller chunks is possible, but this will require two days at a minimum.

At the end of the day, NOT applying the patch will cause more troubles. I carefully read the mailing lists to get feedback and I resolved all the problems reported. Again, the only mistake I did was to prepare one big patch. This will never happen.

Summary:
Applying the patch is a risk, of course, nobody is perfect. Not applying the patch is a disaster. I have seen users on the mailing list appreciating all these new features. The patch covers issues of IOErrors( broken jdbc connections) and more. I do not want to go into details here, but I have a programming experience of 28 years and I have a feeling what is better and what is not better. Not applying the patch is the worst case scenario and will cause many troubles in the future.

Again, sorry
Christian

2012/6/15 Justin Deoliveira <jdeolive@anonymised.com>

Good point David. That is certainly one of the issues with the patch, in that it lumps a number of changes together. If it were broken up i would be fine with saying parts of it could go in and wait on only smaller parts of it.

On Fri, Jun 15, 2012 at 8:10 AM, David Winslow <dwinslow@anonymised.com> wrote:

Is it feasible to reduce the size of the patch by including only the consistency fixes, and shipping a GeoServer 2.2 release that knowingly contains UI bugs and uses deprecated (but presumed working) code?


David Winslow
OpenGeo - http://opengeo.org/

On Fri, Jun 15, 2012 at 9:50 AM, Christian Mueller <mcrmcr21@anonymised.com> wrote:

Hi Andrea,

Yes, imho GSIP 77 will improve the situation. Big +1 for the proposal. The whole new security system is a monster change and I do not want to have a Geoserver release with an inconsistent security system. Let us wait for Justins opinion, I would vote for your proposal having a beta 3.

Christian

2012/6/15 Andrea Aime <andrea.aime@anonymised.com>

On Fri, Jun 15, 2012 at 1:05 PM, Christian Mueller <mcrmcr21@anonymised.com> wrote:

The question is how to continue, two facts I want to point out

  • We cannot make a 2.2.0 release without the changes. The system would not
    work correctly.
  • My next steps would be to review/complete the security documentation and
    during this work, make a next round hardening the code.

Opinions ?.

The fact that we still need a 4400+loc patch to fix the security
subsystem tells me
whatever we release next week cannot possibly be a release candidate, but at
best a beta3, especially since you say that a next round of hardening is in
the plans: nothing bad about it per se, but bad that it’s needed since RC means
Release Candidate, means we believe we’re done and ask the users to
check and eventually tell us otherwise.

Hopefully GSIP 77 will bring some sanity into all of this.

Btw, I have no time to review the patch, I can have a look during the weekend
but my familiarity with the new authentication system is not enough.
I’ll trust Justin’s judgement on it unless my quick review really
finds some red flag.

Cheers
Andrea


Ing. Andrea Aime
GeoSolutions S.A.S.
Tech lead

Via Poggio alle Viti 1187
55054 Massarosa (LU)
Italy

phone: +39 0584 962313
fax: +39 0584 962313
mob: +39 339 8844549

http://www.geo-solutions.it
http://geo-solutions.blogspot.com/
http://www.youtube.com/user/GeoSolutionsIT
http://www.linkedin.com/in/andreaaime
http://twitter.com/geowolf


Live Security Virtual Conference
Exclusive live event will cover all the ways today’s security and
threat landscape has changed and how IT managers can respond. Discussions
will include endpoint security, mobile security and the latest in malware
threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/


Geoserver-devel mailing list
Geoserver-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geoserver-devel


Live Security Virtual Conference
Exclusive live event will cover all the ways today’s security and
threat landscape has changed and how IT managers can respond. Discussions
will include endpoint security, mobile security and the latest in malware
threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/


Geoserver-devel mailing list
Geoserver-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geoserver-devel


Justin Deoliveira
OpenGeo - http://opengeo.org
Enterprise support for open source geospatial.

On Fri, Jun 15, 2012 at 9:37 AM, Christian Mueller <mcrmcr21@anonymised.com> wrote:

Hi David, Justin, Andrea, …

sorry for all these inconveniences. During eliminating the deprecations I also fixed some bugs. The problem is that within the last month my work was always interrupted. (Finishing my Master Thesis, emergency situation at my customer). Splitting the patch in smaller chunks is possible, but this will require two days at a minimum.

At the end of the day, NOT applying the patch will cause more troubles. I carefully read the mailing lists to get feedback and I resolved all the problems reported. Again, the only mistake I did was to prepare one big patch. This will never happen.

Summary:
Applying the patch is a risk, of course, nobody is perfect. Not applying the patch is a disaster. I have seen users on the mailing list appreciating all these new features. The patch covers issues of IOErrors( broken jdbc connections) and more. I do not want to go into details here, but I have a programming experience of 28 years and I have a feeling what is better and what is not better. Not applying the patch is the worst case scenario and will cause many troubles in the future.

Can you be more specific about what the disasters will be? Especially in light of the fact that we will be able to push out a release in approximately one-months time after 2.2.0 goes out the door?

To present an opposing view many folks I talk to say that 2.2.0 not being an official release yes is a disaster for them. Since we have waited so long to make it a stable release people are naturally using it because they need certain key features and bug fixes. But at the same time since its not stable they can’t expect the same level of support for it. There has to be a compromise somewhere. And given that i don’t hear too many people screaming at this point I think that compromise is now. For those who these issues really are a blocker for I don’t think its unreasonable to ask the to wait another 1-2 months for 2.2.1.

Again, sorry
Christian

2012/6/15 Justin Deoliveira <jdeolive@anonymised.com>

Good point David. That is certainly one of the issues with the patch, in that it lumps a number of changes together. If it were broken up i would be fine with saying parts of it could go in and wait on only smaller parts of it.

On Fri, Jun 15, 2012 at 8:10 AM, David Winslow <dwinslow@anonymised.com> wrote:

Is it feasible to reduce the size of the patch by including only the consistency fixes, and shipping a GeoServer 2.2 release that knowingly contains UI bugs and uses deprecated (but presumed working) code?


David Winslow
OpenGeo - http://opengeo.org/

On Fri, Jun 15, 2012 at 9:50 AM, Christian Mueller <mcrmcr21@anonymised.com> wrote:

Hi Andrea,

Yes, imho GSIP 77 will improve the situation. Big +1 for the proposal. The whole new security system is a monster change and I do not want to have a Geoserver release with an inconsistent security system. Let us wait for Justins opinion, I would vote for your proposal having a beta 3.

Christian

2012/6/15 Andrea Aime <andrea.aime@anonymised.com>

On Fri, Jun 15, 2012 at 1:05 PM, Christian Mueller <mcrmcr21@anonymised.com> wrote:

The question is how to continue, two facts I want to point out

  • We cannot make a 2.2.0 release without the changes. The system would not
    work correctly.
  • My next steps would be to review/complete the security documentation and
    during this work, make a next round hardening the code.

Opinions ?.

The fact that we still need a 4400+loc patch to fix the security
subsystem tells me
whatever we release next week cannot possibly be a release candidate, but at
best a beta3, especially since you say that a next round of hardening is in
the plans: nothing bad about it per se, but bad that it’s needed since RC means
Release Candidate, means we believe we’re done and ask the users to
check and eventually tell us otherwise.

Hopefully GSIP 77 will bring some sanity into all of this.

Btw, I have no time to review the patch, I can have a look during the weekend
but my familiarity with the new authentication system is not enough.
I’ll trust Justin’s judgement on it unless my quick review really
finds some red flag.

Cheers
Andrea


Ing. Andrea Aime
GeoSolutions S.A.S.
Tech lead

Via Poggio alle Viti 1187
55054 Massarosa (LU)
Italy

phone: +39 0584 962313
fax: +39 0584 962313
mob: +39 339 8844549

http://www.geo-solutions.it
http://geo-solutions.blogspot.com/
http://www.youtube.com/user/GeoSolutionsIT
http://www.linkedin.com/in/andreaaime
http://twitter.com/geowolf


Live Security Virtual Conference
Exclusive live event will cover all the ways today’s security and
threat landscape has changed and how IT managers can respond. Discussions
will include endpoint security, mobile security and the latest in malware
threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/


Geoserver-devel mailing list
Geoserver-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geoserver-devel


Live Security Virtual Conference
Exclusive live event will cover all the ways today’s security and
threat landscape has changed and how IT managers can respond. Discussions
will include endpoint security, mobile security and the latest in malware
threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/


Geoserver-devel mailing list
Geoserver-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geoserver-devel


Justin Deoliveira
OpenGeo - http://opengeo.org
Enterprise support for open source geospatial.


Justin Deoliveira
OpenGeo - http://opengeo.org
Enterprise support for open source geospatial.

Hi Justin

About the disaster. I do not want to talk about wrong error messages, this is a minor problem.

  1. Create a role service backed by Jdbc. Open the dialog for editing and press save without modifications. The password is lost. Reopen the configuration dialog and try to repair. No chance, you will see a stack trace.

  2. Try to work with user/group or role services causing an IOException. (You can simulate the situation by shutting down the database). The only thing you can see are stack traces

  3. Create an additional role service and user /group service. Add users/groups and roles. Now go the access control pages. (Layer,Services). If the role service is not the active one, you will not see your new roles.

There are a lot of such issues. IMHO, the patch is really a hardening of the system. Believe me, the last week, I really tried to set up a security infrastructure according to the mails of the user list. The most important thing is that we have our “root” user, otherwise I would have given up. The patch only tries to fix such kinds of issues and gives us the chance to improve the user feeling in the next release. And again, beyond finishing our agreed role concept, there is nothing new.

My primary target is to have a GeoServer version 2.2.0 with a consistent security concept and I am hoping for support of all developers.

Thanks in advance
Christan

2012/6/15 Justin Deoliveira <jdeolive@anonymised.com>

On Fri, Jun 15, 2012 at 9:37 AM, Christian Mueller <mcrmcr21@anonymised.com> wrote:

Hi David, Justin, Andrea, …

sorry for all these inconveniences. During eliminating the deprecations I also fixed some bugs. The problem is that within the last month my work was always interrupted. (Finishing my Master Thesis, emergency situation at my customer). Splitting the patch in smaller chunks is possible, but this will require two days at a minimum.

At the end of the day, NOT applying the patch will cause more troubles. I carefully read the mailing lists to get feedback and I resolved all the problems reported. Again, the only mistake I did was to prepare one big patch. This will never happen.

Summary:
Applying the patch is a risk, of course, nobody is perfect. Not applying the patch is a disaster. I have seen users on the mailing list appreciating all these new features. The patch covers issues of IOErrors( broken jdbc connections) and more. I do not want to go into details here, but I have a programming experience of 28 years and I have a feeling what is better and what is not better. Not applying the patch is the worst case scenario and will cause many troubles in the future.

Can you be more specific about what the disasters will be? Especially in light of the fact that we will be able to push out a release in approximately one-months time after 2.2.0 goes out the door?

To present an opposing view many folks I talk to say that 2.2.0 not being an official release yes is a disaster for them. Since we have waited so long to make it a stable release people are naturally using it because they need certain key features and bug fixes. But at the same time since its not stable they can’t expect the same level of support for it. There has to be a compromise somewhere. And given that i don’t hear too many people screaming at this point I think that compromise is now. For those who these issues really are a blocker for I don’t think its unreasonable to ask the to wait another 1-2 months for 2.2.1.

Again, sorry
Christian

2012/6/15 Justin Deoliveira <jdeolive@anonymised.com>

Good point David. That is certainly one of the issues with the patch, in that it lumps a number of changes together. If it were broken up i would be fine with saying parts of it could go in and wait on only smaller parts of it.

On Fri, Jun 15, 2012 at 8:10 AM, David Winslow <dwinslow@anonymised.com> wrote:

Is it feasible to reduce the size of the patch by including only the consistency fixes, and shipping a GeoServer 2.2 release that knowingly contains UI bugs and uses deprecated (but presumed working) code?


David Winslow
OpenGeo - http://opengeo.org/

On Fri, Jun 15, 2012 at 9:50 AM, Christian Mueller <mcrmcr21@anonymised.com> wrote:

Hi Andrea,

Yes, imho GSIP 77 will improve the situation. Big +1 for the proposal. The whole new security system is a monster change and I do not want to have a Geoserver release with an inconsistent security system. Let us wait for Justins opinion, I would vote for your proposal having a beta 3.

Christian

2012/6/15 Andrea Aime <andrea.aime@anonymised.com>

On Fri, Jun 15, 2012 at 1:05 PM, Christian Mueller <mcrmcr21@anonymised.com> wrote:

The question is how to continue, two facts I want to point out

  • We cannot make a 2.2.0 release without the changes. The system would not
    work correctly.
  • My next steps would be to review/complete the security documentation and
    during this work, make a next round hardening the code.

Opinions ?.

The fact that we still need a 4400+loc patch to fix the security
subsystem tells me
whatever we release next week cannot possibly be a release candidate, but at
best a beta3, especially since you say that a next round of hardening is in
the plans: nothing bad about it per se, but bad that it’s needed since RC means
Release Candidate, means we believe we’re done and ask the users to
check and eventually tell us otherwise.

Hopefully GSIP 77 will bring some sanity into all of this.

Btw, I have no time to review the patch, I can have a look during the weekend
but my familiarity with the new authentication system is not enough.
I’ll trust Justin’s judgement on it unless my quick review really
finds some red flag.

Cheers
Andrea


Ing. Andrea Aime
GeoSolutions S.A.S.
Tech lead

Via Poggio alle Viti 1187
55054 Massarosa (LU)
Italy

phone: +39 0584 962313
fax: +39 0584 962313
mob: +39 339 8844549

http://www.geo-solutions.it
http://geo-solutions.blogspot.com/
http://www.youtube.com/user/GeoSolutionsIT
http://www.linkedin.com/in/andreaaime
http://twitter.com/geowolf


Live Security Virtual Conference
Exclusive live event will cover all the ways today’s security and
threat landscape has changed and how IT managers can respond. Discussions
will include endpoint security, mobile security and the latest in malware
threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/


Geoserver-devel mailing list
Geoserver-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geoserver-devel


Live Security Virtual Conference
Exclusive live event will cover all the ways today’s security and
threat landscape has changed and how IT managers can respond. Discussions
will include endpoint security, mobile security and the latest in malware
threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/


Geoserver-devel mailing list
Geoserver-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geoserver-devel


Justin Deoliveira
OpenGeo - http://opengeo.org
Enterprise support for open source geospatial.


Justin Deoliveira
OpenGeo - http://opengeo.org
Enterprise support for open source geospatial.

Well the question is is this issue a blocker since really blockers are the only things that hold up releases. My gut tells me no since the jdbc backend is sort of outside of the core and its a user opt-in. Don’t get me wrong, a serious bug, but again is it a blocker. I think we could probably come up with a number of ways to muck up the configuration if we really tried, should they all hold up the release?

Anyways, this is something the PSC will have to vote on it seems. My position is that this should not hold up the release. However if you can break the patch up into the smaller chunks I will be happy to review them piecewise asap and am ok with the ones that we are really really sure are low risk.

-Justin

On Fri, Jun 15, 2012 at 11:02 AM, Christian Mueller <mcrmcr21@anonymised.com> wrote:

Hi Justin

About the disaster. I do not want to talk about wrong error messages, this is a minor problem.

  1. Create a role service backed by Jdbc. Open the dialog for editing and press save without modifications. The password is lost. Reopen the configuration dialog and try to repair. No chance, you will see a stack trace.

  2. Try to work with user/group or role services causing an IOException. (You can simulate the situation by shutting down the database). The only thing you can see are stack traces

  3. Create an additional role service and user /group service. Add users/groups and roles. Now go the access control pages. (Layer,Services). If the role service is not the active one, you will not see your new roles.

There are a lot of such issues. IMHO, the patch is really a hardening of the system. Believe me, the last week, I really tried to set up a security infrastructure according to the mails of the user list. The most important thing is that we have our “root” user, otherwise I would have given up. The patch only tries to fix such kinds of issues and gives us the chance to improve the user feeling in the next release. And again, beyond finishing our agreed role concept, there is nothing new.

My primary target is to have a GeoServer version 2.2.0 with a consistent security concept and I am hoping for support of all developers.

Thanks in advance
Christan

2012/6/15 Justin Deoliveira <jdeolive@anonymised.com>

On Fri, Jun 15, 2012 at 9:37 AM, Christian Mueller <mcrmcr21@anonymised.com> wrote:

Hi David, Justin, Andrea, …

sorry for all these inconveniences. During eliminating the deprecations I also fixed some bugs. The problem is that within the last month my work was always interrupted. (Finishing my Master Thesis, emergency situation at my customer). Splitting the patch in smaller chunks is possible, but this will require two days at a minimum.

At the end of the day, NOT applying the patch will cause more troubles. I carefully read the mailing lists to get feedback and I resolved all the problems reported. Again, the only mistake I did was to prepare one big patch. This will never happen.

Summary:
Applying the patch is a risk, of course, nobody is perfect. Not applying the patch is a disaster. I have seen users on the mailing list appreciating all these new features. The patch covers issues of IOErrors( broken jdbc connections) and more. I do not want to go into details here, but I have a programming experience of 28 years and I have a feeling what is better and what is not better. Not applying the patch is the worst case scenario and will cause many troubles in the future.

Can you be more specific about what the disasters will be? Especially in light of the fact that we will be able to push out a release in approximately one-months time after 2.2.0 goes out the door?

To present an opposing view many folks I talk to say that 2.2.0 not being an official release yes is a disaster for them. Since we have waited so long to make it a stable release people are naturally using it because they need certain key features and bug fixes. But at the same time since its not stable they can’t expect the same level of support for it. There has to be a compromise somewhere. And given that i don’t hear too many people screaming at this point I think that compromise is now. For those who these issues really are a blocker for I don’t think its unreasonable to ask the to wait another 1-2 months for 2.2.1.

Again, sorry
Christian

2012/6/15 Justin Deoliveira <jdeolive@anonymised.com>

Good point David. That is certainly one of the issues with the patch, in that it lumps a number of changes together. If it were broken up i would be fine with saying parts of it could go in and wait on only smaller parts of it.

On Fri, Jun 15, 2012 at 8:10 AM, David Winslow <dwinslow@anonymised.com> wrote:

Is it feasible to reduce the size of the patch by including only the consistency fixes, and shipping a GeoServer 2.2 release that knowingly contains UI bugs and uses deprecated (but presumed working) code?


David Winslow
OpenGeo - http://opengeo.org/

On Fri, Jun 15, 2012 at 9:50 AM, Christian Mueller <mcrmcr21@anonymised.com> wrote:

Hi Andrea,

Yes, imho GSIP 77 will improve the situation. Big +1 for the proposal. The whole new security system is a monster change and I do not want to have a Geoserver release with an inconsistent security system. Let us wait for Justins opinion, I would vote for your proposal having a beta 3.

Christian

2012/6/15 Andrea Aime <andrea.aime@anonymised.com>

On Fri, Jun 15, 2012 at 1:05 PM, Christian Mueller <mcrmcr21@anonymised.com> wrote:

The question is how to continue, two facts I want to point out

  • We cannot make a 2.2.0 release without the changes. The system would not
    work correctly.
  • My next steps would be to review/complete the security documentation and
    during this work, make a next round hardening the code.

Opinions ?.

The fact that we still need a 4400+loc patch to fix the security
subsystem tells me
whatever we release next week cannot possibly be a release candidate, but at
best a beta3, especially since you say that a next round of hardening is in
the plans: nothing bad about it per se, but bad that it’s needed since RC means
Release Candidate, means we believe we’re done and ask the users to
check and eventually tell us otherwise.

Hopefully GSIP 77 will bring some sanity into all of this.

Btw, I have no time to review the patch, I can have a look during the weekend
but my familiarity with the new authentication system is not enough.
I’ll trust Justin’s judgement on it unless my quick review really
finds some red flag.

Cheers
Andrea


Ing. Andrea Aime
GeoSolutions S.A.S.
Tech lead

Via Poggio alle Viti 1187
55054 Massarosa (LU)
Italy

phone: +39 0584 962313
fax: +39 0584 962313
mob: +39 339 8844549

http://www.geo-solutions.it
http://geo-solutions.blogspot.com/
http://www.youtube.com/user/GeoSolutionsIT
http://www.linkedin.com/in/andreaaime
http://twitter.com/geowolf


Live Security Virtual Conference
Exclusive live event will cover all the ways today’s security and
threat landscape has changed and how IT managers can respond. Discussions
will include endpoint security, mobile security and the latest in malware
threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/


Geoserver-devel mailing list
Geoserver-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geoserver-devel


Live Security Virtual Conference
Exclusive live event will cover all the ways today’s security and
threat landscape has changed and how IT managers can respond. Discussions
will include endpoint security, mobile security and the latest in malware
threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/


Geoserver-devel mailing list
Geoserver-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geoserver-devel


Justin Deoliveira
OpenGeo - http://opengeo.org
Enterprise support for open source geospatial.


Justin Deoliveira
OpenGeo - http://opengeo.org
Enterprise support for open source geospatial.


Justin Deoliveira
OpenGeo - http://opengeo.org
Enterprise support for open source geospatial.

On Fri, Jun 15, 2012 at 8:17 PM, Justin Deoliveira <jdeolive@anonymised.com> wrote:

Well the question is is this issue a blocker since really blockers are the
only things that hold up releases. My gut tells me no since the jdbc backend
is sort of outside of the core and its a user opt-in. Don't get me wrong, a
serious bug, but again is it a blocker. I think we could probably come up
with a number of ways to muck up the configuration if we really tried,
should they all hold up the release?

Anyways, this is something the PSC will have to vote on it seems. My
position is that this should not hold up the release. However if you can
break the patch up into the smaller chunks I will be happy to review them
piecewise asap and am ok with the ones that we are really really sure are
low risk.

If the issues are indeed mostly centralized in the sec-jdbc module I agree
we can just let it out of the release and bring it back for 2.1.1, or apply
the patch now but kick the module in extension/community land for the
time being.

If the pile of bugs is showing up with the out of the box
configuration of the security subsytem
that's another story, but maybe the set of fixes affecting core code is smaller?

Cheers
Andrea

--
Ing. Andrea Aime
GeoSolutions S.A.S.
Tech lead

Via Poggio alle Viti 1187
55054 Massarosa (LU)
Italy

phone: +39 0584 962313
fax: +39 0584 962313
mob: +39 339 8844549

http://www.geo-solutions.it
http://geo-solutions.blogspot.com/
http://www.youtube.com/user/GeoSolutionsIT
http://www.linkedin.com/in/andreaaime
http://twitter.com/geowolf

Hello, folks.

I apologize for intruding in something that is above my paygrade, but for whatever it’s worth I will drop my .2c as a user just in case:

I spent the last weak extending the JDBC service to have an alternative that uses our own databases structure and pass encoding hash algorithms (btw, thanks for the guidance, it works like a dream! :slight_smile: ) and the problems Christian described with the JDBC extension are, if not a complete deal breaker, frustrating to someone who is not aware of the inner workings of security subsystem. An average user would have a bad time if he screws up the configuration the first time.

Were not for the root password I would probably have flipped tables way more often than what I am known for and many times I had to manually edit the config.xml files because of some inconsistency or other odd things in the services I was creating.

Again: I dare not presume to pretend to have a say on anything, just sharing my experience. :slight_smile:

Thanks!

Rodrigo Del C. Andrade

Analista de Sistemas
SIC - SSE - Soluções Segurança Pública
DÍGITRO TECNOLOGIA
E-mail: rodrigo.andrade@anonymised.com
Fone: +55 48 3281-7000 Ramal: 7537
Fax: +55 48 3281-7299

Esta mensagem, incluindo seus anexos, é reservada somente à Dígitro e ao destinatário da mensagem. Caso você tenha recebido esta mensagem por engano, Who is John Galt?

On Sat, Jun 16, 2012 at 12:00 AM, Rodrigo Del C. Andrade
<rodrigo.andrade@anonymised.com> wrote:

Hello, folks.

I apologize for intruding in something that is above my paygrade, but for
whatever it's worth I will drop my .2c as a user just in case:

I spent the last weak extending the JDBC service to have an alternative that
uses our own databases structure and pass encoding hash algorithms (btw,
thanks for the guidance, it works like a dream! :slight_smile: ) and the problems
Christian described with the JDBC extension are, if not a complete deal
breaker, frustrating to someone who is not aware of the inner workings of
security subsystem. An average user would have a bad time if he screws up
the configuration the first time.

Were not for the root password I would probably have flipped tables way
more often than what I am known for and many times I had to manually edit
the config.xml files because of some inconsistency or other odd things in
the services I was creating.

Rodrigo, what we're discussing here is not the opportunity of committing the
patches at all, it's obvious that they must go in,
but the opportuntity commit them now instead of a 4-6 weeks time,
for a feature that's new and whose you're a early adopter of.

It is astonishing that someone cannot wait one month for a large amount
of changes to land where other contributors had to wait over four months
so far to get a release (half of the PSC was ready to release in
February, mind),
and had to pay consequences because of that.

Working in a community is also about balancing the needs of all the
contributors,
focusing on one particular need and forgetting all the other work that was done
is offensive for the peole that carried it on and makes it harder to justify
contributing as a company, or find reasons to spend a Sunday trying to
help out instead of having fun or relax.
This of course works both ways, I'm sure Christian is pretty frustrated and
wondering why he put all the effort and now he can't close it up for the 2.2.0
release, at the same time the people that wanted to release in February
have been frustrated for 4 months now and with the continous pushes
to delay the release to get in this or that it's really getting beyond
ridicolous.

Cheers
Andrea

--
Ing. Andrea Aime
GeoSolutions S.A.S.
Tech lead

Via Poggio alle Viti 1187
55054 Massarosa (LU)
Italy

phone: +39 0584 962313
fax: +39 0584 962313
mob: +39 339 8844549

http://www.geo-solutions.it
http://geo-solutions.blogspot.com/
http://www.youtube.com/user/GeoSolutionsIT
http://www.linkedin.com/in/andreaaime
http://twitter.com/geowolf

Hi all

I really want to find a solution for this unlucky situation. The core developers have the big picture about Geoserver and I do not want to cause further problems.

Waiting 4-6 weeks is no problem (I am 46 years old, small time frame for me). I have only one question.

The security code is brand new and introduced in the 2.2.x versions. Let us call it version A. The patch removes a lot of problems, let us call it version B.
I think it is important to say that the patch modifies only the brand new security code. The idea is to delay major changes like http://jira.codehaus.org/browse/GEOS-5155 and prefixed role names to a later release.

I cannot see a reason for using version A instead of version B. Again, modifications of B only touches security code, A is new and B is new too. Version A may cause a lot of problems for early adopters. I try to see it from the perspective of the core developers not involved in the development of the security module. (Justin is the exception). The risk is the same, why not make a 2.2.0 release with a lot of bug fixes ?

To close this issue, I need a decision of the PSC. I am not frustrated but I fear early adopters will be. On the other side, I do not have the big picture, please vote.

Again, I am sorry about the situation
Christian

2012/6/15 Andrea Aime <andrea.aime@anonymised.com>

On Sat, Jun 16, 2012 at 12:00 AM, Rodrigo Del C. Andrade
<rodrigo.andrade@anonymised.com.> wrote:

Hello, folks.

I apologize for intruding in something that is above my paygrade, but for
whatever it’s worth I will drop my .2c as a user just in case:

I spent the last weak extending the JDBC service to have an alternative that
uses our own databases structure and pass encoding hash algorithms (btw,
thanks for the guidance, it works like a dream! :slight_smile: ) and the problems
Christian described with the JDBC extension are, if not a complete deal
breaker, frustrating to someone who is not aware of the inner workings of
security subsystem. An average user would have a bad time if he screws up
the configuration the first time.

Were not for the root password I would probably have flipped tables way
more often than what I am known for and many times I had to manually edit
the config.xml files because of some inconsistency or other odd things in
the services I was creating.

Rodrigo, what we’re discussing here is not the opportunity of committing the
patches at all, it’s obvious that they must go in,
but the opportuntity commit them now instead of a 4-6 weeks time,
for a feature that’s new and whose you’re a early adopter of.

It is astonishing that someone cannot wait one month for a large amount
of changes to land where other contributors had to wait over four months
so far to get a release (half of the PSC was ready to release in
February, mind),
and had to pay consequences because of that.

Working in a community is also about balancing the needs of all the
contributors,
focusing on one particular need and forgetting all the other work that was done
is offensive for the peole that carried it on and makes it harder to justify
contributing as a company, or find reasons to spend a Sunday trying to
help out instead of having fun or relax.
This of course works both ways, I’m sure Christian is pretty frustrated and
wondering why he put all the effort and now he can’t close it up for the 2.2.0
release, at the same time the people that wanted to release in February
have been frustrated for 4 months now and with the continous pushes
to delay the release to get in this or that it’s really getting beyond
ridicolous.

Cheers
Andrea


Ing. Andrea Aime
GeoSolutions S.A.S.
Tech lead

Via Poggio alle Viti 1187
55054 Massarosa (LU)
Italy

phone: +39 0584 962313
fax: +39 0584 962313
mob: +39 339 8844549

http://www.geo-solutions.it
http://geo-solutions.blogspot.com/
http://www.youtube.com/user/GeoSolutionsIT
http://www.linkedin.com/in/andreaaime
http://twitter.com/geowolf


Live Security Virtual Conference
Exclusive live event will cover all the ways today’s security and
threat landscape has changed and how IT managers can respond. Discussions
will include endpoint security, mobile security and the latest in malware
threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/


Geoserver-devel mailing list
Geoserver-devel@anonymised.comsts.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geoserver-devel

On Sat, Jun 16, 2012 at 2:45 AM, Christian Mueller <mcrmcr21@anonymised.com> wrote:

Hi all

I really want to find a solution for this unlucky situation. The core developers have the big picture about Geoserver and I do not want to cause further problems.

Waiting 4-6 weeks is no problem (I am 46 years old, small time frame for me). I have only one question.

The security code is brand new and introduced in the 2.2.x versions. Let us call it version A. The patch removes a lot of problems, let us call it version B.
I think it is important to say that the patch modifies only the brand new security code. The idea is to delay major changes like http://jira.codehaus.org/browse/GEOS-5155 and prefixed role names to a later release.

Unfortunately the security module can’t really be viewed as a new part of code and hence lower risk to modify. It ties in in many places. Startup is one, a failure in security subsystem startup halts the starting of the server entirely. It also ties pretty deeply into our persistence layer / xstream with the encryption there, again making it a possible point of massive failure.

I cannot see a reason for using version A instead of version B. Again, modifications of B only touches security code, A is new and B is new too. Version A may cause a lot of problems for early adopters. I try to see it from the perspective of the core developers not involved in the development of the security module. (Justin is the exception). The risk is the same, why not make a 2.2.0 release with a lot of bug fixes ?

Its not about the changes in this case, we know they fix some existing bugs. That said I did have a question about the patch in one my emails to you asking for some clarification (the GeoServerSecurityManager part), i don’t think got answered. Regardless, its not about that. It is about drawing a line a some point and calling a release candidate. As Andrea noted that line has been pushed back for far too long to the point where it is starting to affect people who have built there core business around GeoServer. It is my opinion that also the majority of the community thinks it has been far too long as well.

There is also the issue of the patch lumping together many issues into one. If it were split up it would be easier to come to the compromise of picking some changes, the ones that are clearly lower risk, or are trivial changes. I take some responsibility for this as I have been encouraging you to work with git on a branch and to organize changes into cohesive commits.

To close this issue, I need a decision of the PSC. I am not frustrated but I fear early adopters will be. On the other side, I do not have the big picture, please vote.

My vote is to go ahead with the release and wait for the changes, for the above reasons. But also because i myself have fell to the temptation of trying to push in last minute changes in the past and have been burned for it.

Again, I am sorry about the situation
Christian

2012/6/15 Andrea Aime <andrea.aime@anonymised.com1268…>

On Sat, Jun 16, 2012 at 12:00 AM, Rodrigo Del C. Andrade
<rodrigo.andrade@anonymised.com> wrote:

Hello, folks.

I apologize for intruding in something that is above my paygrade, but for
whatever it’s worth I will drop my .2c as a user just in case:

I spent the last weak extending the JDBC service to have an alternative that
uses our own databases structure and pass encoding hash algorithms (btw,
thanks for the guidance, it works like a dream! :slight_smile: ) and the problems
Christian described with the JDBC extension are, if not a complete deal
breaker, frustrating to someone who is not aware of the inner workings of
security subsystem. An average user would have a bad time if he screws up
the configuration the first time.

Were not for the root password I would probably have flipped tables way
more often than what I am known for and many times I had to manually edit
the config.xml files because of some inconsistency or other odd things in
the services I was creating.

Rodrigo, what we’re discussing here is not the opportunity of committing the
patches at all, it’s obvious that they must go in,
but the opportuntity commit them now instead of a 4-6 weeks time,
for a feature that’s new and whose you’re a early adopter of.

It is astonishing that someone cannot wait one month for a large amount
of changes to land where other contributors had to wait over four months
so far to get a release (half of the PSC was ready to release in
February, mind),
and had to pay consequences because of that.

Working in a community is also about balancing the needs of all the
contributors,
focusing on one particular need and forgetting all the other work that was done
is offensive for the peole that carried it on and makes it harder to justify
contributing as a company, or find reasons to spend a Sunday trying to
help out instead of having fun or relax.
This of course works both ways, I’m sure Christian is pretty frustrated and
wondering why he put all the effort and now he can’t close it up for the 2.2.0
release, at the same time the people that wanted to release in February
have been frustrated for 4 months now and with the continous pushes
to delay the release to get in this or that it’s really getting beyond
ridicolous.

Cheers
Andrea


Ing. Andrea Aime
GeoSolutions S.A.S.
Tech lead

Via Poggio alle Viti 1187
55054 Massarosa (LU)
Italy

phone: +39 0584 962313
fax: +39 0584 962313
mob: +39 339 8844549

http://www.geo-solutions.it
http://geo-solutions.blogspot.com/
http://www.youtube.com/user/GeoSolutionsIT
http://www.linkedin.com/in/andreaaime
http://twitter.com/geowolf


Live Security Virtual Conference
Exclusive live event will cover all the ways today’s security and
threat landscape has changed and how IT managers can respond. Discussions
will include endpoint security, mobile security and the latest in malware
threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/


Geoserver-devel mailing list
Geoserver-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geoserver-devel


Live Security Virtual Conference
Exclusive live event will cover all the ways today’s security and
threat landscape has changed and how IT managers can respond. Discussions
will include endpoint security, mobile security and the latest in malware
threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/


Geoserver-devel mailing list
Geoserver-devel@anonymised.comsts.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geoserver-devel


Justin Deoliveira
OpenGeo - http://opengeo.org
Enterprise support for open source geospatial.

On Sun, Jun 17, 2012 at 10:31 PM, Justin Deoliveira <jdeolive@anonymised.com> wrote:

f the core developers not involved in the development of the security module. (Justin is the exception). The risk is the same, why not make a 2.2.0 release with a lot of bug fixes ?

There is also the issue of the patch lumping together many issues into one. If it were split up it would be easier to come to the compromise of picking some changes, the ones that are clearly lower risk, or are trivial changes. I take some responsibility for this as I have been encouraging you to work with git on a branch and to organize changes into cohesive commits.

I had a look at the patch today… my impression is that a large part of it is renaming constants and removing deprecations,
the constants bit is actually propagating itself quite a bit inflating the size of the patch significantly.

And then there are the actual changes, which might be, gut feeling (did not count) 30Kb maybe?
The thing is, I don’t understand squat about them, so I cannot judge how risky they are.

This is the way I see it. If we have serious bugs that can prevent users to use GeoServer in its default configuration
then we cannot release this as a RC, if they are affecting only the sec-jdbc module then I’d propose we go
on and kick that module out of the release for this release.

So please, can somebody point reports of issues with the default configuration, show how
they can affect users, what workarounds are there, if any?

Cheers
Andrea


Ing. Andrea Aime
GeoSolutions S.A.S.
Tech lead

Via Poggio alle Viti 1187
55054 Massarosa (LU)
Italy

phone: +39 0584 962313
fax: +39 0584 962313
mob: +39 339 8844549

http://www.geo-solutions.it
http://geo-solutions.blogspot.com/
http://www.youtube.com/user/GeoSolutionsIT
http://www.linkedin.com/in/andreaaime
http://twitter.com/geowolf

Short version: I vote to hold the security patch until after RC and then apply it only to trunk.

On 16/06/12 16:45, Christian Mueller wrote:

Waiting 4-6 weeks is no problem

It is worse than that. To clarify, when the RC is made, we will also be branching 2.2.x. Only essential bugfixes will made before 2.2.0 is released. New functionality will only be backported after testing on trunk and then only if low risk and with a PSC vote. Major refactoring is not allowed on stable.
http://geoserver.org/display/GEOS/GSIP+77+-+Time+boxed+release+model

I voted +1 for GSIP 77.

To close this issue, I need a decision of the PSC.

I vote to hold the security patch until after RC; this means that it will miss 2.2.x and only be applied to 2.3.x (trunk). Once it has been shown to work reliably on trunk, it might backported at some future date, but not if it requires major refactoring.

Yes, this means that a lot of the new security functionality will suck on 2.2.0 and later stable.

The purpose of the stable branch is to maintain stability, that it, to avoid introducing unpleasant surprises for users, such as new bugs, functional changes, or file format incompatibilities. It does not mean that stable is bug free, just that the behaviour of the software is known. Users should be able to upgrade a stable production system to a later patch version of the stable branch and have a reasonable expectation that nothing that was working will break. While security will be pretty hurty, that is just tough. It will be no worse than before your patch. New code equals new bugs. That is just the way it is. Stable is where we avoid adding bugs. While some users will feel pain, the herd will settle on one stable code base and learn how to cope.

Releasing more often is a good practice. Andrea has done a great job on GSIP 77 and it would be a pity for our good intentions to be derailed so soon. No dessert until we eat all our vegetables.

- Holding off releases has left stable so far behind app-schema on trunk that we have for over a year been warning users to never use 2.1.x for app-schema.

- I know at least two other developers who have work that cannot go in RC and must go on trunk. Delaying RC blocks their work.

Note that throughout, references to security mean security subsystem changes. Vulnerabilities to external attack (perhaps spelled "SECURITY"?) must always be patched on stable and trunk as soon as possible.

Kind regards,

--
Ben Caradoc-Davies <Ben.Caradoc-Davies@anonymised.com>
Software Engineer
CSIRO Earth Science and Resource Engineering
Australian Resources Research Centre

Hi all

@Justin, the modifications concerning GeoServerSecurityManager are described in http://jira.codehaus.org/browse/GEOS-5101. The patch finishes this concept we agreed on 17th of May. I added some additional description.

@Andrea, the default xml role service is also affected by GEOS-5101. As an example, a user can create two xml role services and add the identical role (Let us call it ROLE_XXX) to both services. Then the role in the first service is assigned to user1, the role in the second service is assigned to user2. Now open the access control page for data or services. You will see ROLE_XXX once. First confusion. You have to check what the actual role service is and perhaps, you have to modify this setting. The patch avoids such situations by checking uniqueness across all role services.

For access control, you need to see all the roles present in the system. At the moment, you only see the roles of the actual role service. Next confusion. Believe me, I tested with multiple role and user/group services and I was confused by myself very often.

As you pointed out, most of the patch is about replacing deprecations. During the deprecation work, I also found some bugs concerning wrong assignments of deprecated constants to not deprecated constants and some failures in the resource files, → wrong or missing error messages. The JDBC stuff also covers GEOS-5101 and bug fixes.

@Ben, the patch does not introduce new functionality. If finishes the role concept which is already in trunk partially , but it does not work correctly. The rest of the bug fixes is essential and I think, resolving deprecations is not a major refactoring.

But anyway, if the patch goes not it, it is ok. I want to point out some consequences.

  1. The security systems requires a migration. Without the patch, I have to split the migration into two steps, making things more complicated. The second step is migration code for the role service.

  2. Users using more than one role service may produce configurations making step 2 of the migration impossible. (Additionally to the problems described above).

  3. If it is not possible to bring the changes at least to 2.2.1 but only to 2.3.0, I have no idea how to continue. I even stopped working on the security documentation because I am not sure how to describe things related to roles.

Christian

2012/6/18 Ben Caradoc-Davies <Ben.Caradoc-Davies@anonymised.com>

Short version: I vote to hold the security patch until after RC and then apply it only to trunk.

On 16/06/12 16:45, Christian Mueller wrote:

Waiting 4-6 weeks is no problem

It is worse than that. To clarify, when the RC is made, we will also be branching 2.2.x. Only essential bugfixes will made before 2.2.0 is released. New functionality will only be backported after testing on trunk and then only if low risk and with a PSC vote. Major refactoring is not allowed on stable.
http://geoserver.org/display/GEOS/GSIP+77±+Time+boxed+release+model

I voted +1 for GSIP 77.

To close this issue, I need a decision of the PSC.

I vote to hold the security patch until after RC; this means that it will miss 2.2.x and only be applied to 2.3.x (trunk). Once it has been shown to work reliably on trunk, it might backported at some future date, but not if it requires major refactoring.

Yes, this means that a lot of the new security functionality will suck on 2.2.0 and later stable.

The purpose of the stable branch is to maintain stability, that it, to avoid introducing unpleasant surprises for users, such as new bugs, functional changes, or file format incompatibilities. It does not mean that stable is bug free, just that the behaviour of the software is known. Users should be able to upgrade a stable production system to a later patch version of the stable branch and have a reasonable expectation that nothing that was working will break. While security will be pretty hurty, that is just tough. It will be no worse than before your patch. New code equals new bugs. That is just the way it is. Stable is where we avoid adding bugs. While some users will feel pain, the herd will settle on one stable code base and learn how to cope.

Releasing more often is a good practice. Andrea has done a great job on GSIP 77 and it would be a pity for our good intentions to be derailed so soon. No dessert until we eat all our vegetables.

  • Holding off releases has left stable so far behind app-schema on trunk that we have for over a year been warning users to never use 2.1.x for app-schema.

  • I know at least two other developers who have work that cannot go in RC and must go on trunk. Delaying RC blocks their work.

Note that throughout, references to security mean security subsystem changes. Vulnerabilities to external attack (perhaps spelled “SECURITY”?) must always be patched on stable and trunk as soon as possible.

Kind regards,


Ben Caradoc-Davies Ben.Caradoc-Davies@anonymised.com
Software Engineer
CSIRO Earth Science and Resource Engineering
Australian Resources Research Centre

You certainly are persistent Christian. While it still goes against my better judgement I am inclined to let these changes in. The main reasons being:

  • our policy has always been fuzzy on this and GSIP 77 has not yet become official, so we have no hard and fast rule
  • Victor has stated the release is being pushed back a few days until Thursday

So if the other PSC members consent I would say go ahead for now.

But please note that this will really be the only exception to this rule. I believe Andrea is going to be updating the GSIP 77 proposal to be clearer about this. But after the proposal lands there will be no exceptions like this. If the developer doesn’t have things in order before hitting the stable deadline then its too bad. They must wait no matter how inconvenient it will be for them.

Also moving forward please ensure that you don’t lump large changesets into single commits in your git repo. A good rule of thumb is a single commit for a single JIRA issue. This would have made things a lot easier from the start.

-Justin

On Mon, Jun 18, 2012 at 8:03 AM, Christian Mueller <mcrmcr21@anonymised.com> wrote:

Hi all

@Justin, the modifications concerning GeoServerSecurityManager are described in http://jira.codehaus.org/browse/GEOS-5101. The patch finishes this concept we agreed on 17th of May. I added some additional description.

@Andrea, the default xml role service is also affected by GEOS-5101. As an example, a user can create two xml role services and add the identical role (Let us call it ROLE_XXX) to both services. Then the role in the first service is assigned to user1, the role in the second service is assigned to user2. Now open the access control page for data or services. You will see ROLE_XXX once. First confusion. You have to check what the actual role service is and perhaps, you have to modify this setting. The patch avoids such situations by checking uniqueness across all role services.

For access control, you need to see all the roles present in the system. At the moment, you only see the roles of the actual role service. Next confusion. Believe me, I tested with multiple role and user/group services and I was confused by myself very often.

As you pointed out, most of the patch is about replacing deprecations. During the deprecation work, I also found some bugs concerning wrong assignments of deprecated constants to not deprecated constants and some failures in the resource files, → wrong or missing error messages. The JDBC stuff also covers GEOS-5101 and bug fixes.

@Ben, the patch does not introduce new functionality. If finishes the role concept which is already in trunk partially , but it does not work correctly. The rest of the bug fixes is essential and I think, resolving deprecations is not a major refactoring.

But anyway, if the patch goes not it, it is ok. I want to point out some consequences.

  1. The security systems requires a migration. Without the patch, I have to split the migration into two steps, making things more complicated. The second step is migration code for the role service.

  2. Users using more than one role service may produce configurations making step 2 of the migration impossible. (Additionally to the problems described above).

  3. If it is not possible to bring the changes at least to 2.2.1 but only to 2.3.0, I have no idea how to continue. I even stopped working on the security documentation because I am not sure how to describe things related to roles.

Christian

2012/6/18 Ben Caradoc-Davies <Ben.Caradoc-Davies@anonymised.com>

Short version: I vote to hold the security patch until after RC and then apply it only to trunk.

On 16/06/12 16:45, Christian Mueller wrote:

Waiting 4-6 weeks is no problem

It is worse than that. To clarify, when the RC is made, we will also be branching 2.2.x. Only essential bugfixes will made before 2.2.0 is released. New functionality will only be backported after testing on trunk and then only if low risk and with a PSC vote. Major refactoring is not allowed on stable.
http://geoserver.org/display/GEOS/GSIP+77±+Time+boxed+release+model

I voted +1 for GSIP 77.

To close this issue, I need a decision of the PSC.

I vote to hold the security patch until after RC; this means that it will miss 2.2.x and only be applied to 2.3.x (trunk). Once it has been shown to work reliably on trunk, it might backported at some future date, but not if it requires major refactoring.

Yes, this means that a lot of the new security functionality will suck on 2.2.0 and later stable.

The purpose of the stable branch is to maintain stability, that it, to avoid introducing unpleasant surprises for users, such as new bugs, functional changes, or file format incompatibilities. It does not mean that stable is bug free, just that the behaviour of the software is known. Users should be able to upgrade a stable production system to a later patch version of the stable branch and have a reasonable expectation that nothing that was working will break. While security will be pretty hurty, that is just tough. It will be no worse than before your patch. New code equals new bugs. That is just the way it is. Stable is where we avoid adding bugs. While some users will feel pain, the herd will settle on one stable code base and learn how to cope.

Releasing more often is a good practice. Andrea has done a great job on GSIP 77 and it would be a pity for our good intentions to be derailed so soon. No dessert until we eat all our vegetables.

  • Holding off releases has left stable so far behind app-schema on trunk that we have for over a year been warning users to never use 2.1.x for app-schema.

  • I know at least two other developers who have work that cannot go in RC and must go on trunk. Delaying RC blocks their work.

Note that throughout, references to security mean security subsystem changes. Vulnerabilities to external attack (perhaps spelled “SECURITY”?) must always be patched on stable and trunk as soon as possible.

Kind regards,


Ben Caradoc-Davies Ben.Caradoc-Davies@anonymised.com
Software Engineer
CSIRO Earth Science and Resource Engineering
Australian Resources Research Centre


Live Security Virtual Conference
Exclusive live event will cover all the ways today’s security and
threat landscape has changed and how IT managers can respond. Discussions
will include endpoint security, mobile security and the latest in malware
threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/


Geoserver-devel mailing list
Geoserver-devel@anonymised.comsts.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geoserver-devel


Justin Deoliveira
OpenGeo - http://opengeo.org
Enterprise support for open source geospatial.

On Mon, Jun 18, 2012 at 5:10 PM, Justin Deoliveira <jdeolive@anonymised.com> wrote:

You certainly are persistent Christian. While it still goes against my better judgement I am inclined to let these changes in. The main reasons being:

  • our policy has always been fuzzy on this and GSIP 77 has not yet become official, so we have no hard and fast rule
  • Victor has stated the release is being pushed back a few days until Thursday

So if the other PSC members consent I would say go ahead for now.

Works for me

But please note that this will really be the only exception to this rule. I believe Andrea is going to be updating the GSIP 77 proposal to be clearer about this. But after the proposal lands there will be no exceptions like this. If the developer doesn’t have things in order before hitting the stable deadline then its too bad. They must wait no matter how inconvenient it will be for them.

Yep, either a work committed in the “green zone” is complete by the time we release a RC or we should roll it back somehow
(at least, for stuff that can be rolled back).
Will update the proposal also to make sure the PSC uses extra attention to resourcing and to the likelyness that a proposal
gets completed in time (one immediate consequence, unlikely to get large proposals in a week before the hardening starts)

Cheers
Andrea


Ing. Andrea Aime
GeoSolutions S.A.S.
Tech lead

Via Poggio alle Viti 1187
55054 Massarosa (LU)
Italy

phone: +39 0584 962313
fax: +39 0584 962313
mob: +39 339 8844549

http://www.geo-solutions.it
http://geo-solutions.blogspot.com/
http://www.youtube.com/user/GeoSolutionsIT
http://www.linkedin.com/in/andreaaime
http://twitter.com/geowolf