[Geoserver-devel] QGIS caching WFS responses without caching headers, and making sure a client is not allowed to do that

GeoServer GeoServer Developer
,
1

Hi,
I am looking at some editing issue with QGIS and found out that the HTTP client
in QGIS seems to be caching all GetCapabilities and DescribeFeature type responses
because GeoServer does not add any caching header.

So for example, if a layer is added or its structure is changed, the edits will fail, or adding
the layer becomes impossible (first the layer is listed in the “add layer” dialog, then
qgis claims the document is not in the capabilities document… that’s a bit hilarious).

Anyone minds if we add by default “Cache-Control: no-cache” to the capabilites and
describe feature type responses?
Clients would be supposed to use the sequence number checks, but apparently qgis
does not, and indeed he caching behavior of a http client is not clearly defined
when there are no caching headers in the responses.

I believe most desktop clients and browsers play it safe and request again, but it does
not hurt to be explicit, no?
For Capabilities one could even add a “Cache-Control: revalidate” along with an eTag
that happens to be the same as the update sequence number, to allow for conditional
requests (If-None-Match) support.

Cheers
Andrea

···

==
GeoServer Professional Services from the experts! Visit
http://goo.gl/it488V for more information.

Ing. Andrea Aime

@geowolf
Technical Lead

GeoSolutions S.A.S.
Via di Montramito 3/A
55054 Massarosa (LU)
phone: +39 0584 962313

fax: +39 0584 1660272
mob: +39 339 8844549

http://www.geo-solutions.it
http://twitter.com/geosolutions_it

AVVERTENZE AI SENSI DEL D.Lgs. 196/2003

Le informazioni contenute in questo messaggio di posta elettronica e/o nel/i file/s allegato/i sono da considerarsi strettamente riservate. Il loro utilizzo è consentito esclusivamente al destinatario del messaggio, per le finalità indicate nel messaggio stesso. Qualora riceviate questo messaggio senza esserne il destinatario, Vi preghiamo cortesemente di darcene notizia via e-mail e di procedere alla distruzione del messaggio stesso, cancellandolo dal Vostro sistema. Conservare il messaggio stesso, divulgarlo anche in parte, distribuirlo ad altri soggetti, copiarlo, od utilizzarlo per finalità diverse, costituisce comportamento contrario ai principi dettati dal D.Lgs. 196/2003.

The information in this message and/or attachments, is intended solely for the attention and use of the named addressee(s) and may be confidential or proprietary in nature or covered by the provisions of privacy act (Legislative Decree June, 30 2003, no.196 - Italy’s New Data Protection Code).Any use not in accord with its purpose, any disclosure, reproduction, copying, distribution, or either dissemination, either whole or partial, is strictly forbidden except previous formal approval of the named addressee(s). If you are not the intended recipient, please contact immediately the sender by telephone, fax or e-mail and delete the information in this message that has been received in error. The sender does not give any warranty or accept liability as the content, accuracy or completeness of sent messages and accepts no responsibility for changes made after they were sent or for other risks which arise as a result of e-mail transmission, viruses, etc.

2

sounds like a good plan to me,

Ian

···

On 24 November 2016 at 11:10, Andrea Aime <andrea.aime@anonymised.com> wrote:

Hi,
I am looking at some editing issue with QGIS and found out that the HTTP client
in QGIS seems to be caching all GetCapabilities and DescribeFeature type responses
because GeoServer does not add any caching header.

So for example, if a layer is added or its structure is changed, the edits will fail, or adding
the layer becomes impossible (first the layer is listed in the “add layer” dialog, then
qgis claims the document is not in the capabilities document… that’s a bit hilarious).

Anyone minds if we add by default “Cache-Control: no-cache” to the capabilites and
describe feature type responses?
Clients would be supposed to use the sequence number checks, but apparently qgis
does not, and indeed he caching behavior of a http client is not clearly defined
when there are no caching headers in the responses.

I believe most desktop clients and browsers play it safe and request again, but it does
not hurt to be explicit, no?
For Capabilities one could even add a “Cache-Control: revalidate” along with an eTag
that happens to be the same as the update sequence number, to allow for conditional
requests (If-None-Match) support.

Cheers
Andrea

–

==
GeoServer Professional Services from the experts! Visit
http://goo.gl/it488V for more information.

Ing. Andrea Aime

@geowolf
Technical Lead

GeoSolutions S.A.S.
Via di Montramito 3/A
55054 Massarosa (LU)
phone: +39 0584 962313

fax: +39 0584 1660272
mob: +39 339 8844549

http://www.geo-solutions.it
http://twitter.com/geosolutions_it

AVVERTENZE AI SENSI DEL D.Lgs. 196/2003

Le informazioni contenute in questo messaggio di posta elettronica e/o nel/i file/s allegato/i sono da considerarsi strettamente riservate. Il loro utilizzo è consentito esclusivamente al destinatario del messaggio, per le finalità indicate nel messaggio stesso. Qualora riceviate questo messaggio senza esserne il destinatario, Vi preghiamo cortesemente di darcene notizia via e-mail e di procedere alla distruzione del messaggio stesso, cancellandolo dal Vostro sistema. Conservare il messaggio stesso, divulgarlo anche in parte, distribuirlo ad altri soggetti, copiarlo, od utilizzarlo per finalità diverse, costituisce comportamento contrario ai principi dettati dal D.Lgs. 196/2003.

The information in this message and/or attachments, is intended solely for the attention and use of the named addressee(s) and may be confidential or proprietary in nature or covered by the provisions of privacy act (Legislative Decree June, 30 2003, no.196 - Italy’s New Data Protection Code).Any use not in accord with its purpose, any disclosure, reproduction, copying, distribution, or either dissemination, either whole or partial, is strictly forbidden except previous formal approval of the named addressee(s). If you are not the intended recipient, please contact immediately the sender by telephone, fax or e-mail and delete the information in this message that has been received in error. The sender does not give any warranty or accept liability as the content, accuracy or completeness of sent messages and accepts no responsibility for changes made after they were sent or for other risks which arise as a result of e-mail transmission, viruses, etc.

Geoserver-devel mailing list
Geoserver-devel@anonymised.com.366…sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geoserver-devel

–

Ian Turton

3

Ah, for those that might ship QGIS and wonder what’s going on, this change in behavior happened around 2.16.x, 2.14.x works fine :-p

Cheers
Andrea

···

On Thu, Nov 24, 2016 at 12:10 PM, Andrea Aime <andrea.aime@anonymised.com> wrote:

Hi,
I am looking at some editing issue with QGIS and found out that the HTTP client
in QGIS seems to be caching all GetCapabilities and DescribeFeature type responses
because GeoServer does not add any caching header.

So for example, if a layer is added or its structure is changed, the edits will fail, or adding
the layer becomes impossible (first the layer is listed in the “add layer” dialog, then
qgis claims the document is not in the capabilities document… that’s a bit hilarious).

Anyone minds if we add by default “Cache-Control: no-cache” to the capabilites and
describe feature type responses?
Clients would be supposed to use the sequence number checks, but apparently qgis
does not, and indeed he caching behavior of a http client is not clearly defined
when there are no caching headers in the responses.

I believe most desktop clients and browsers play it safe and request again, but it does
not hurt to be explicit, no?
For Capabilities one could even add a “Cache-Control: revalidate” along with an eTag
that happens to be the same as the update sequence number, to allow for conditional
requests (If-None-Match) support.

Cheers
Andrea

–

==
GeoServer Professional Services from the experts! Visit
http://goo.gl/it488V for more information.

Ing. Andrea Aime

@geowolf
Technical Lead

GeoSolutions S.A.S.
Via di Montramito 3/A
55054 Massarosa (LU)
phone: +39 0584 962313

fax: +39 0584 1660272
mob: +39 339 8844549

http://www.geo-solutions.it
http://twitter.com/geosolutions_it

AVVERTENZE AI SENSI DEL D.Lgs. 196/2003

Le informazioni contenute in questo messaggio di posta elettronica e/o nel/i file/s allegato/i sono da considerarsi strettamente riservate. Il loro utilizzo è consentito esclusivamente al destinatario del messaggio, per le finalità indicate nel messaggio stesso. Qualora riceviate questo messaggio senza esserne il destinatario, Vi preghiamo cortesemente di darcene notizia via e-mail e di procedere alla distruzione del messaggio stesso, cancellandolo dal Vostro sistema. Conservare il messaggio stesso, divulgarlo anche in parte, distribuirlo ad altri soggetti, copiarlo, od utilizzarlo per finalità diverse, costituisce comportamento contrario ai principi dettati dal D.Lgs. 196/2003.

The information in this message and/or attachments, is intended solely for the attention and use of the named addressee(s) and may be confidential or proprietary in nature or covered by the provisions of privacy act (Legislative Decree June, 30 2003, no.196 - Italy’s New Data Protection Code).Any use not in accord with its purpose, any disclosure, reproduction, copying, distribution, or either dissemination, either whole or partial, is strictly forbidden except previous formal approval of the named addressee(s). If you are not the intended recipient, please contact immediately the sender by telephone, fax or e-mail and delete the information in this message that has been received in error. The sender does not give any warranty or accept liability as the content, accuracy or completeness of sent messages and accepts no responsibility for changes made after they were sent or for other risks which arise as a result of e-mail transmission, viruses, etc.

–

==
GeoServer Professional Services from the experts! Visit
http://goo.gl/it488V for more information.

Ing. Andrea Aime

@geowolf
Technical Lead

GeoSolutions S.A.S.
Via di Montramito 3/A
55054 Massarosa (LU)
phone: +39 0584 962313

fax: +39 0584 1660272
mob: +39 339 8844549

http://www.geo-solutions.it
http://twitter.com/geosolutions_it

AVVERTENZE AI SENSI DEL D.Lgs. 196/2003

Le informazioni contenute in questo messaggio di posta elettronica e/o nel/i file/s allegato/i sono da considerarsi strettamente riservate. Il loro utilizzo è consentito esclusivamente al destinatario del messaggio, per le finalità indicate nel messaggio stesso. Qualora riceviate questo messaggio senza esserne il destinatario, Vi preghiamo cortesemente di darcene notizia via e-mail e di procedere alla distruzione del messaggio stesso, cancellandolo dal Vostro sistema. Conservare il messaggio stesso, divulgarlo anche in parte, distribuirlo ad altri soggetti, copiarlo, od utilizzarlo per finalità diverse, costituisce comportamento contrario ai principi dettati dal D.Lgs. 196/2003.

The information in this message and/or attachments, is intended solely for the attention and use of the named addressee(s) and may be confidential or proprietary in nature or covered by the provisions of privacy act (Legislative Decree June, 30 2003, no.196 - Italy’s New Data Protection Code).Any use not in accord with its purpose, any disclosure, reproduction, copying, distribution, or either dissemination, either whole or partial, is strictly forbidden except previous formal approval of the named addressee(s). If you are not the intended recipient, please contact immediately the sender by telephone, fax or e-mail and delete the information in this message that has been received in error. The sender does not give any warranty or accept liability as the content, accuracy or completeness of sent messages and accepts no responsibility for changes made after they were sent or for other risks which arise as a result of e-mail transmission, viruses, etc.

4

Andrea,

I agree with the need for "Cache-Control:" in this use-case, but will this change increase the load on a GeoServer serving mostly static content to many capabilities-driven clients? Does your revalidate / eTag idea address this?

Kind regards,
Ben.

On 25/11/16 00:10, Andrea Aime wrote:

Hi,
I am looking at some editing issue with QGIS and found out that the HTTP
client
in QGIS seems to be caching all GetCapabilities and DescribeFeature type
responses
because GeoServer does not add any caching header.

So for example, if a layer is added or its structure is changed, the edits
will fail, or adding
the layer becomes impossible (first the layer is listed in the "add layer"
dialog, then
qgis claims the document is not in the capabilities document... that's a
bit hilarious).

Anyone minds if we add by default "Cache-Control: no-cache" to the
capabilites and
describe feature type responses?
Clients would be supposed to use the sequence number checks, but apparently
qgis
does not, and indeed he caching behavior of a http client is not clearly
defined
when there are no caching headers in the responses.

I believe most desktop clients and browsers play it safe and request again,
but it does
not hurt to be explicit, no?
For Capabilities one could even add a "Cache-Control: revalidate" along
with an eTag
that happens to be the same as the update sequence number, to allow for
conditional
requests (If-None-Match) support.

Cheers
Andrea

------------------------------------------------------------------------------

_______________________________________________
Geoserver-devel mailing list
Geoserver-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geoserver-devel

--
Ben Caradoc-Davies <ben@anonymised.com>
Director
Transient Software Limited <http://transient.nz/&gt;
New Zealand

5

On Thu, Nov 24, 2016 at 5:49 PM, Ben Caradoc-Davies <ben@anonymised.com>
wrote:

Andrea,

I agree with the need for "Cache-Control:" in this use-case, but will this
change increase the load on a GeoServer serving mostly static content to
many capabilities-driven clients? Does your revalidate / eTag idea address
this?

Hi Ben,
sorry for the late reply, got distracted.

Yes, the revalidate/eTag idea addresses this and, if they clients properly
implement HTTP cache handling (like browsers do) it will
actually reduce the load on such GeoServers.

Browsers behavior in case of lack of headers changes, but they mostly tend
to ask again in case the headers are not found.
Once the headers are setup with a revalidate + eTag, if the browser hits a
GeoServer with static contents, providing as they should
the "If-None-Match" header, they will get back a empty HTTP 304 with no
capabilites being computed.

So in the end, it will actually save the server some capabilities
generation effort.

eTag wise, I would take a combination of the current update sequence number
plus the last startup/reload timestamp,
to care for cases in which someone changed the configuration by direct
filesystem manipulation.

Cheers
Andrea

--

GeoServer Professional Services from the experts! Visit
http://goo.gl/it488V for more information.

Ing. Andrea Aime
@geowolf
Technical Lead

GeoSolutions S.A.S.
Via di Montramito 3/A
55054 Massarosa (LU)
phone: +39 0584 962313
fax: +39 0584 1660272
mob: +39 339 8844549

http://www.geo-solutions.it
http://twitter.com/geosolutions_it

*AVVERTENZE AI SENSI DEL D.Lgs. 196/2003*

Le informazioni contenute in questo messaggio di posta elettronica e/o
nel/i file/s allegato/i sono da considerarsi strettamente riservate. Il
loro utilizzo è consentito esclusivamente al destinatario del messaggio,
per le finalità indicate nel messaggio stesso. Qualora riceviate questo
messaggio senza esserne il destinatario, Vi preghiamo cortesemente di
darcene notizia via e-mail e di procedere alla distruzione del messaggio
stesso, cancellandolo dal Vostro sistema. Conservare il messaggio stesso,
divulgarlo anche in parte, distribuirlo ad altri soggetti, copiarlo, od
utilizzarlo per finalità diverse, costituisce comportamento contrario ai
principi dettati dal D.Lgs. 196/2003.

The information in this message and/or attachments, is intended solely for
the attention and use of the named addressee(s) and may be confidential or
proprietary in nature or covered by the provisions of privacy act
(Legislative Decree June, 30 2003, no.196 - Italy's New Data Protection
Code).Any use not in accord with its purpose, any disclosure, reproduction,
copying, distribution, or either dissemination, either whole or partial, is
strictly forbidden except previous formal approval of the named
addressee(s). If you are not the intended recipient, please contact
immediately the sender by telephone, fax or e-mail and delete the
information in this message that has been received in error. The sender
does not give any warranty or accept liability as the content, accuracy or
completeness of sent messages and accepts no responsibility for changes
made after they were sent or for other risks which arise as a result of
e-mail transmission, viruses, etc.

-------------------------------------------------------

6

Pull request here: https://github.com/geoserver/geoserver/pull/2010

Would be nice if we could get these fixes in 2.10.1

Cheers
Andrea

···

On Fri, Dec 2, 2016 at 4:10 PM, Andrea Aime <andrea.aime@anonymised.com> wrote:

–

On Thu, Nov 24, 2016 at 5:49 PM, Ben Caradoc-Davies <ben@anonymised.com> wrote:

Andrea,

I agree with the need for “Cache-Control:” in this use-case, but will this change increase the load on a GeoServer serving mostly static content to many capabilities-driven clients? Does your revalidate / eTag idea address this?

Hi Ben,
sorry for the late reply, got distracted.

Yes, the revalidate/eTag idea addresses this and, if they clients properly implement HTTP cache handling (like browsers do) it will
actually reduce the load on such GeoServers.

Browsers behavior in case of lack of headers changes, but they mostly tend to ask again in case the headers are not found.
Once the headers are setup with a revalidate + eTag, if the browser hits a GeoServer with static contents, providing as they should
the “If-None-Match” header, they will get back a empty HTTP 304 with no capabilites being computed.

So in the end, it will actually save the server some capabilities generation effort.

eTag wise, I would take a combination of the current update sequence number plus the last startup/reload timestamp,
to care for cases in which someone changed the configuration by direct filesystem manipulation.

Cheers

Andrea

–

==
GeoServer Professional Services from the experts! Visit
http://goo.gl/it488V for more information.

Ing. Andrea Aime

@geowolf
Technical Lead

GeoSolutions S.A.S.
Via di Montramito 3/A
55054 Massarosa (LU)
phone: +39 0584 962313

fax: +39 0584 1660272
mob: +39 339 8844549

http://www.geo-solutions.it
http://twitter.com/geosolutions_it

AVVERTENZE AI SENSI DEL D.Lgs. 196/2003

Le informazioni contenute in questo messaggio di posta elettronica e/o nel/i file/s allegato/i sono da considerarsi strettamente riservate. Il loro utilizzo è consentito esclusivamente al destinatario del messaggio, per le finalità indicate nel messaggio stesso. Qualora riceviate questo messaggio senza esserne il destinatario, Vi preghiamo cortesemente di darcene notizia via e-mail e di procedere alla distruzione del messaggio stesso, cancellandolo dal Vostro sistema. Conservare il messaggio stesso, divulgarlo anche in parte, distribuirlo ad altri soggetti, copiarlo, od utilizzarlo per finalità diverse, costituisce comportamento contrario ai principi dettati dal D.Lgs. 196/2003.

The information in this message and/or attachments, is intended solely for the attention and use of the named addressee(s) and may be confidential or proprietary in nature or covered by the provisions of privacy act (Legislative Decree June, 30 2003, no.196 - Italy’s New Data Protection Code).Any use not in accord with its purpose, any disclosure, reproduction, copying, distribution, or either dissemination, either whole or partial, is strictly forbidden except previous formal approval of the named addressee(s). If you are not the intended recipient, please contact immediately the sender by telephone, fax or e-mail and delete the information in this message that has been received in error. The sender does not give any warranty or accept liability as the content, accuracy or completeness of sent messages and accepts no responsibility for changes made after they were sent or for other risks which arise as a result of e-mail transmission, viruses, etc.

==
GeoServer Professional Services from the experts! Visit
http://goo.gl/it488V for more information.

Ing. Andrea Aime

@geowolf
Technical Lead

GeoSolutions S.A.S.
Via di Montramito 3/A
55054 Massarosa (LU)
phone: +39 0584 962313

fax: +39 0584 1660272
mob: +39 339 8844549

http://www.geo-solutions.it
http://twitter.com/geosolutions_it

AVVERTENZE AI SENSI DEL D.Lgs. 196/2003

Le informazioni contenute in questo messaggio di posta elettronica e/o nel/i file/s allegato/i sono da considerarsi strettamente riservate. Il loro utilizzo è consentito esclusivamente al destinatario del messaggio, per le finalità indicate nel messaggio stesso. Qualora riceviate questo messaggio senza esserne il destinatario, Vi preghiamo cortesemente di darcene notizia via e-mail e di procedere alla distruzione del messaggio stesso, cancellandolo dal Vostro sistema. Conservare il messaggio stesso, divulgarlo anche in parte, distribuirlo ad altri soggetti, copiarlo, od utilizzarlo per finalità diverse, costituisce comportamento contrario ai principi dettati dal D.Lgs. 196/2003.

The information in this message and/or attachments, is intended solely for the attention and use of the named addressee(s) and may be confidential or proprietary in nature or covered by the provisions of privacy act (Legislative Decree June, 30 2003, no.196 - Italy’s New Data Protection Code).Any use not in accord with its purpose, any disclosure, reproduction, copying, distribution, or either dissemination, either whole or partial, is strictly forbidden except previous formal approval of the named addressee(s). If you are not the intended recipient, please contact immediately the sender by telephone, fax or e-mail and delete the information in this message that has been received in error. The sender does not give any warranty or accept liability as the content, accuracy or completeness of sent messages and accepts no responsibility for changes made after they were sent or for other risks which arise as a result of e-mail transmission, viruses, etc.