I really don't like the idea of forking, and really don't want to see a bunch of effort continue to be held up on the 1.3.x branch. At the very least I would like to see the branch currently labeled 1.4.x become trunk when GeoTools complex branch is merged in.
Perhaps a better solution is to have a trunk that does not contain all the risk of the complex branch, but does give a stable area for new developers to work, and more importantly to have their changes incorporated (which we've been incredibly bad about doing, since any nice new feature that _may_ introduce a bit of instability we've been holding off for the planned development trunk, and I fear we will really lose people). I propose that we have trunk with Justin's spring and build improvements, that functions against GeoTools 2.2.x, which is now stable. The complex stuff we turn in to the complex branch, and merge it in to trunk _after_ it reaches stability on the GeoTools side of the fence.
Additionally I propose a general policy of keeping the trunk of GeoServer aligned with the current stable branch of GeoTools. Currently this is 2.2.x. I hope that we can upgrade our development trunk whenever GeoTools merges in an RnD effort that is useful to us.
As for specific concerns that have arisen...
> What justin didnt mention is the fact that Geoserver has jumped from the
> geotools 2.1.x branch - this is the 'stable branch' which is *very
> stable*. Geoserver 1.3.x and udig 1.0.x are based on this. Its very
> good.
>
> 1.4 hasnt jumped just to 2.2 (which, as far as I know doesnt have a true
> stable branch) but straight to 2.3 (which doesnt really exist yet).
> 2.3 is, I believe, 2.2 with about six R&D efforts merging in. Its not
> stable - in any sense of the word - and its unlikely to be stable for
> "a while."
This definitely mis-characterizes the 2.3 branch. Yes, there are about six R+D efforts to be merged in, but they are not all going to come on to 2.3. GeoTools is moving towards a policy of having trunk be stable, and having all R+D efforts on a branch. This is an improvement over the past, when trunk was simply the experimental branch, and everyone's efforts would be dumped in. It was a hassle to time a 'stable' release. GeoTools has made good progress in changing, as 2.2 has just a couple R+D efforts, and 2.3 has the same plan. It will just have the complex feature model merged in, and then move towards stability. This will be the target of the GeoServer 1.4 release. Overall this change should not be too bad, and it's well funded from SCO, TOPP has committed resources to see it through, and many from GeoTools have looked over the proposed changes to make sure they are good. There is a suite of unit tests, with a higher level of quality than has been required in the past.
Part of the reason that GeoTools hasn't been as stable as it could be is because GeoServer has spent too long on the 2.1.x branch. Continuing to put slot our effort there is simply suicide in the long term. GeoTools success is due in no small part to GeoServer's leadership role. If we drop that, then GeoTools won't improve except where we put our effort, since no one else is going to be working with us on the 'extremely stable branch'. We need to invest the time to make GeoTools good and stable, or no one wins. A number of the RnD efforts are directly _for_ GeoServer. Complex Features and the Coverage stuff are both _major_ improvements for GeoServer, that we would be silly to not incorporate. They each contain features that no one else has. By making 1.3.x as trunk we actually _increase_ our risk over the medium term, since there will be far more to deal with if we do not keep the trunk incrementally upgrading. TOPP is now giving Justin more time to help improve GeoTools stability, but those efforts need to be supported by GeoServer's center of development gravity, or we are asking him to do super human work.
> Another big concern is that putting 1.3 on a branch just means we're
> going to marginalize and forget about it. I want to see a stable
> geoserver; a geoserver that people (including myself) can build stable
> applications on. I'm sure there's lots of people who want to see that
> too. Geoserver uptake is dependent on this.
I really don't see how putting 1.3 on a branch would marginalize it and have people forget about it. The standard practice among all open source projects is to keep development on trunk, and when you want to reach stability you branch. You generally do this when you start to go to RC's and feature freezes, so major development can continue. We've been bad about this, and several good code contributions have languished in JIRA awaiting this.
If one wants to develop stable applications on, you download the stable releases. We will continue to port back bug fixes to the 1.3.x branch until we reach at least 1.4-RC1, if not 1.4.0. Though I'd like RC's to start being stable, feature freezes, instead of the RC1-8 mess we got in last time. We should be releasing new stable releases every 3-4 months. We are on track to do this with 1.4.0 by the end of april, with the complex features.
>> My main concerns for developing GeoServer as a popular and usable tool
>> are:
>> - Each release has to be better than the last, overall
>> - Each release has to be more and more stable
>> - Get what is already there working
>>
>> If people can see progress every time they download a new version, I
>> think we are golden. If they download new versions that still contain
>> largely the same bugs as before but have a few new features, then I
>> think people will lose trust in the product.
I disagree. The new versions are marked as 'beta', or 'release candidate', and people downloading them know what they are getting in to. They are those who are willing to help us out in improving it. Each 'stable' release should be more stable than the last one, but each release can not be more stable than the last unless you are not introducing any new features. Before you do each stable release, you port all the bug fixes from the last stable branch over. If we are to focus all our energy on just stability, then GeoServer will not grow in terms of features that users really want. This is not a theoretical statement - both the complex features and the raster/coverage support are major advances in GeoServer that are waiting in the wings. If we don't support the developers who have worked on them then _users_ will not lost trust in our product, but developers will, as a base that they can expand and contribute to. To miss that is to miss the whole point of being an open source project.
There is obviously a tension between supporting users with stable versions and support developers with a base that they can get their changes incorporated to. I think the answer for us is to have Brent and Dave focus on the stable base, and Justin focus on the community contributions.
But it is vital that the trunk is a place where developers can write code against that will get included in the next release. If they just want to write an app on _top_ of GeoServer, then they should go with stable. But if they want to expand the capabilities, they should be able to get trunk.
>> Trunk vs. Branches:
>> I believe that developers should be able to check out the latest
>> version from trunk and have it stable and work right away. Trunk
>> should sort of be the live version of a very working product. And not
>> unstable due to R&D. Most of the development should be done on a
>> branch and then merged onto trunk to ensure that everything stays
>> stable (except for the several days of merging). This means that the
>> branch has to *really* work. I'm fine doing bug fixes on trunk and
>> merging the fixes onto branches to keep them up to date. I think that
>> has to be done or the branches will never come online. But keep
>> prototypes and R&D on branches until they are ready.
Agreed. But it's important here that we track the stable GeoTools. After RnD merges in GeoTools, we should have the GeoServer trunk upgrade to that release. If we have to spin a new stable branch of GeoServer, then that's what we should do. But our stable branches should be upgrading, instead of stuck on 1.3.x.
>> In summary:
>> I think we need to wait until 1.4 (complex features) is stable and
>> introduces almost no bugs (passing cite tests isn't enough, users need
>> to try it), then merge it into trunk. Rushing into it will just hurt
>> us. I really want to see the complex stuff in, but not at the cost of
>> losing users because it is too unstable or unfinished.
>>
>> For the spring framework, I am *really* looking forward to this.
>> Ideally it would be good to move to that fully, with a release or two
>> under the framework's belt, then merge in the complex feature model
>> and the new Geotools versions.
Let's do it then. Justin, can you take 1.4.x back to before you started merging Gabriel's changes, diff out any spring improvements you might have made, and call that trunk? Then we can punch out a 1.4-M0 from that, which is just as stable as 1.3.x, but has a new build system.
The current 1.4.x branch gets renamed 'complex'. We release off of that in a couple weeks, when Justing gets it working. Users can test it, we do bug fixes on the GeoTools branch. Then we can merge in the GeoTools branch, last half of march, and take that change to GeoServer trunk. This should be just about feature complete for 1.4.0, so we call it RC1, and spend the next release or two fixing bugs, release 1.4.0 near the end of April.
Chris
Justin Deoliveira wrote:
So it looks like there some opposition to making 1.4.x the new trunk, and rightfully so. As our project does not have a PMC (something me way wish to rectify) we cant really do a formal vote. So it looks like 1.4.x will remain on a branch. However with the amount of development that will be going on with it I think the term fork is more suitable.
-Justin
Brent Owens wrote:
My main concerns for developing GeoServer as a popular and usable tool are:
- Each release has to be better than the last, overall
- Each release has to be more and more stable
- Get what is already there working
If people can see progress every time they download a new version, I think we are golden. If they download new versions that still contain largely the same bugs as before but have a few new features, then I think people will lose trust in the product. GeoTools has been slipping with this for the last little while and it is starting to show. In my opinion, bug fixes show more progress than a new feature.
My worry with the complex branch and GeoServer is that we will have an unfinished product, due to deadline constraints, and it will result in a bad release or will force the next release to be held off for many months. Also, moving off of GeoTools 2.1.x means moving off a stable branch onto trunk that is having several branches merged into it at once. I think it might be too soon to jump onto GeoTools trunk. Another thing is how long-term the developers are committed to the addition. Since the complex feature model is at the core of everything, if the developers had to move out to work other development before the feature model is rock solid (read *and* write capabilities), I would definitely want it to stay on a branch as an optional plugin.
With respect to moving a branch onto trunk because someone has paid to have it done, I think that is a strong violation of what an open development project is. I think it should be left up to the community to decide what things will go into the project, especially if it impacts everything else. Not money. If it was just a bug fix or a little feature, than sure. But the complex feature model is high risk and has the potential to hose things over for a while until it has all been settled down and tested extensively. I want us to be careful.
Trunk vs. Branches:
I believe that developers should be able to check out the latest version from trunk and have it stable and work right away. Trunk should sort of be the live version of a very working product. And not unstable due to R&D. Most of the development should be done on a branch and then merged onto trunk to ensure that everything stays stable (except for the several days of merging). This means that the branch has to *really* work. I'm fine doing bug fixes on trunk and merging the fixes onto branches to keep them up to date. I think that has to be done or the branches will never come online. But keep prototypes and R&D on branches until they are ready.
In summary:
I think we need to wait until 1.4 (complex features) is stable and introduces almost no bugs (passing cite tests isn't enough, users need to try it), then merge it into trunk. Rushing into it will just hurt us. I really want to see the complex stuff in, but not at the cost of losing users because it is too unstable or unfinished.
For the spring framework, I am *really* looking forward to this. Ideally it would be good to move to that fully, with a release or two under the framework's belt, then merge in the complex feature model and the new Geotools versions.
Brent Owens
(The Open Planning Project)
Justin Deoliveira wrote:
Very valid concerns. Additonal comments inline.
I am quite concerned with going to the 1.4 branch on trunk. Mostly for
what justin *didnt* mention than what he did.
The source code re-organization, maven build, and adding spring are all
great - they are not major (in the risky sense of the word) changes.
NOTE: I asked justin about the spring changes, and it seems to be there
"for future work" and is basically unused at the moment. So, this
shouldnt be a concern for anyone.
What justin didnt mention is the fact that Geoserver has jumped from the
geotools 2.1.x branch - this is the 'stable branch' which is *very
stable*. Geoserver 1.3.x and udig 1.0.x are based on this. Its very
good
I apologize if I didnt' make it clear that 1.4.x now runs of geotools trunk (well not quite, version 2.2.x to be more accurate), but getting onto trunk will only require a few changes.
I have mentioned this a couple of times in email but should have mentioned it here as well.
1.4 hasnt jumped just to 2.2 (which, as far as I know doesnt have a true
stable branch) but straight to 2.3 (which doesnt really exist yet).
2.3 is, I believe, 2.2 with about six R&D efforts merging in. Its not
stable - in any sense of the word - and its unlikely to be stable for
"a while."
In fact, as far as I can tell, over the last 60 days, geotools trunk has
only actual been able to *build* for about 23 of those days.
And, if you try to build geoserver 1.4 right now, you'll find that it
doesnt actually compile because the geotools API has changed (again)
and is incompatible with whats currently in 1.4.x.
This doesnt bode well for me as I want a stable, useable Geoserver.
Geoserver only uses the very high level geotools api (up until very
recently geoserver was compatible with both 2.1.x and 2.2.x).
Not sure if I understand your statement correctly but Geoserver 1.3.x was definitley not compatable with geotools 2.2.x, it hasn't been for months.
I asked
justin if the 1.4.x branch is compatible with 2.1.x. He said there
were "a lot" of modifications to geoserver to get it working with 2.2.
There are "just a few" modification necessary to get it working with
2.3.
Most of the changes are simple yes.
Perhaps justin could give us a summary of what changes were necessary?
Geotools is supposed to be backwards compatible by one version so,
theoretically, there shouldnt be any changes necessary (and up until
recently none were required).
Pretty much all the changes are the FactoryFinder system for style and filter. A simple matter of replaceing StyleFactory.createStyleFactory() to StyleFactoryFinder.createStyleFactory().
In the ideal world, I'd like to see geoserver compatible with both 2.1.x
geotools and 2.2 geotools. This way people who are doing Geoserver R&D
experimental work can just set the "use geotools 2.2 or 2.3" switch and
the people who want a stable geoserver can set the "use geotools 2.1"
switch. That way we can all work together and everybody are "shinny
happy people".
Pretty sure this is immpossible. The geotools feature model is pretty screwed up right now and *needs* to be fixed. Which will means api has to break.
You can probably convince me to move to geotools 2.2 (assuming there is
an actual stable branch), but I think its suicide to go right on
geotools trunk. Especially considering the number of changes that are
going to be happening Real Soon Now.
Since geoserver is driving the changes in the feature model, 1.4.x *has* to be on geotools trunk for the new complex stuff to go. However the complex stuff being in its state of flux will be on a branch anyways. But remember that people have paid to have it on trunk.
Another big concern is that putting 1.3 on a branch just means we're
going to marginalize and forget about it. I want to see a stable
geoserver; a geoserver that people (including myself) can build stable
applications on. I'm sure there's lots of people who want to see that
too. Geoserver uptake is dependent on this.
I undersand the concerns of being on trunk and wanting a stable branch. And I think its a great idea, however i dont think its a good idea to do for 6 months like we did last time. I think the actual problem is that we need to be more involved in driving geotools development. As one of the two major clients of the geotools library, Geoserver has a *major* say about what kind of changes get made and what dont. I think that is a method of ensuring stability that is getting overlooked.
Another way of maintaining stability for applications that use geoserver is to actually create api in geoserver. The lines between what is api and implementation in geoserver are pretty blurred. So its not surprising that changes from geotools trickel all the way up to the application tier. Says something about having an architecture.
I dont see isolating us on a branch as acceptible. Its essentially a well maintained fork of geotools. I mean if all we want is a few people to maintain geotools 2.1.x and do geoserver bug fixes why are we bothering with the overhead of an open development process. It would make more sense to close development and just make geoserver freely available.
I'd like to keep 1.3 on trunk until at least the time that there's a
stable (and good) geotools 2.2 branch and 1.4.x is working well with
it. That means one thats as good as the current 1.3. There's no point
in going *backwards*.
I understand the need to maintain stability when building applications. However I see uDig do this very well with frequent shifts from trunk to stable branches. Perhaps we needs to take a few notes from their development process. However I am pretty sure that it boils down to Jody being so involved in geootols development, and being the person primary driving api changes which work for udig. I would like to see us be able to do that with GeoServer.
People who require geotools 2.3 or geotools trunk should be on a
1.4-like branch. We can merge in when geotools has a good, stable 2.3
branch. Justin says there's only a few lines in 3 classes that are
different for geoserver to go from 2.2 to 2.3, so maintaining sync
between the two branches should be very little work. In fact, it could
be possible to make geoserver compatible with both 2.2 and 2.3 and then
we're really cooking with gas!
Sorry there is some misunderstanding here, the 3 classes are for the complex work, not the shift from 2.1.x to 2.2.x.
dave
-------------------------------------------------------
This SF.net email is sponsored by: Splunk Inc. Do you grep through log files
for problems? Stop! Download the new AJAX search engine that makes
searching your log files as easy as surfing the web. DOWNLOAD SPLUNK!
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=103432&bid=230486&dat=121642
_______________________________________________
Geoserver-devel mailing list
Geoserver-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geoserver-devel
-------------------------------------------------------
This SF.Net email is sponsored by xPML, a groundbreaking scripting language
that extends applications into web and mobile media. Attend the live webcast
and join the prime developer group breaking into this new coding territory!
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=110944&bid=241720&dat=121642
_______________________________________________
Geoserver-devel mailing list
Geoserver-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geoserver-devel
--
Chris Holmes
The Open Planning Project
thoughts at: http://cholmes.wordpress.com