[Geoserver-devel] windows build server failures on master

From Simone’s email to getools-devel:

a while ago we agree on having a windows build server for geotools
which is reachable here:
http://winbuild.geo-solutions.it/jenkins
credentials are the same for the linux build server.

I was able to restore the build for geotools-devel, but it looks like we have some failures for the geoserver master build target.

Failed tests: testSEStyleWithRelativePathProtocol(org.geoserver.catalog.ResourcePoolTest): expected:<C:.jenkins\jobs\GeoServer-Master\workspace\src\main\target\default4702095627624841408data\styles\images\rockFillSymbol.png> but was:<\images\rockFillSymbol.png>

A general call out on this one, I would like to ensure we build cleanly on windows (and take advantage of this machine that has been provided).

···


Jody Garnett

On Sun, Jan 18, 2015 at 8:06 PM, Jody Garnett <jody.garnett@anonymised.com>
wrote:

From Simone's email to getools-devel:

a while ago we agree on having a windows build server for geotools

which is reachable here:
http://winbuild.geo-solutions.it/jenkins
credentials are the same for the linux build server.

I was able to restore the build for geotools-devel, but it looks like we
have some failures for the geoserver master build target.

Failed tests:

testSEStyleWithRelativePathProtocol(org.geoserver.catalog.ResourcePoolTest):
expected:<C:\.jenkins\jobs\GeoServer-Master\workspace\src\main\target\default4702095627624841408data\styles\images\rockFillSymbol.png>
but was:<\\images\rockFillSymbol.png>

A general call out on this one, I would like to ensure we build cleanly on
windows (and take advantage of this machine that has been provided).

Hi Jody,
the GeoServer windows build server has never been made "official" because
I did not manage to get the build working on Windows in a stable way
(and got no help doing so either).

I have a windows laptop now that I can use to do some bits on Windows, but
honestly I despise the platform, so it's more "call of duty" than something
I want
to do in my spare time, it would need some champion that really
pushes for it.

Last time I checked there were some random failures in the WCS modules,
failure to delete some files in the temp data dirs, I guess we are still
keeping some reader/file handle open, but could not locate where.
I guess it got worse from there.

Cheers
Andrea

--

GeoServer Professional Services from the experts! Visit
http://goo.gl/NWWaa2 for more information.

Ing. Andrea Aime
@geowolf
Technical Lead

GeoSolutions S.A.S.
Via Poggio alle Viti 1187
55054 Massarosa (LU)
Italy
phone: +39 0584 962313
fax: +39 0584 1660272
mob: +39 339 8844549

http://www.geo-solutions.it
http://twitter.com/geosolutions_it

*AVVERTENZE AI SENSI DEL D.Lgs. 196/2003*

Le informazioni contenute in questo messaggio di posta elettronica e/o
nel/i file/s allegato/i sono da considerarsi strettamente riservate. Il
loro utilizzo è consentito esclusivamente al destinatario del messaggio,
per le finalità indicate nel messaggio stesso. Qualora riceviate questo
messaggio senza esserne il destinatario, Vi preghiamo cortesemente di
darcene notizia via e-mail e di procedere alla distruzione del messaggio
stesso, cancellandolo dal Vostro sistema. Conservare il messaggio stesso,
divulgarlo anche in parte, distribuirlo ad altri soggetti, copiarlo, od
utilizzarlo per finalità diverse, costituisce comportamento contrario ai
principi dettati dal D.Lgs. 196/2003.

The information in this message and/or attachments, is intended solely for
the attention and use of the named addressee(s) and may be confidential or
proprietary in nature or covered by the provisions of privacy act
(Legislative Decree June, 30 2003, no.196 - Italy's New Data Protection
Code).Any use not in accord with its purpose, any disclosure, reproduction,
copying, distribution, or either dissemination, either whole or partial, is
strictly forbidden except previous formal approval of the named
addressee(s). If you are not the intended recipient, please contact
immediately the sender by telephone, fax or e-mail and delete the
information in this message that has been received in error. The sender
does not give any warranty or accept liability as the content, accuracy or
completeness of sent messages and accepts no responsibility for changes
made after they were sent or for other risks which arise as a result of
e-mail transmission, viruses, etc.

-------------------------------------------------------

Okay we can consider it a goal for the year ( or a sprint activity if we get a Windows volunteer ).

···

On Sun, Jan 18, 2015 at 8:06 PM, Jody Garnett <jody.garnett@anonymised.com> wrote:

From Simone’s email to getools-devel:

a while ago we agree on having a windows build server for geotools
which is reachable here:
http://winbuild.geo-solutions.it/jenkins
credentials are the same for the linux build server.

I was able to restore the build for geotools-devel, but it looks like we have some failures for the geoserver master build target.

Failed tests: testSEStyleWithRelativePathProtocol(org.geoserver.catalog.ResourcePoolTest): expected:<C:.jenkins\jobs\GeoServer-Master\workspace\src\main\target\default4702095627624841408data\styles\images\rockFillSymbol.png> but was:<\images\rockFillSymbol.png>

A general call out on this one, I would like to ensure we build cleanly on windows (and take advantage of this machine that has been provided).

Hi Jody,
the GeoServer windows build server has never been made “official” because
I did not manage to get the build working on Windows in a stable way
(and got no help doing so either).

I have a windows laptop now that I can use to do some bits on Windows, but
honestly I despise the platform, so it’s more “call of duty” than something I want
to do in my spare time, it would need some champion that really
pushes for it.

Last time I checked there were some random failures in the WCS modules,
failure to delete some files in the temp data dirs, I guess we are still
keeping some reader/file handle open, but could not locate where.
I guess it got worse from there.

Cheers
Andrea

==

GeoServer Professional Services from the experts! Visit
http://goo.gl/NWWaa2 for more information.

==

Ing. Andrea Aime

@geowolf
Technical Lead

GeoSolutions S.A.S.
Via Poggio alle Viti 1187
55054 Massarosa (LU)
Italy
phone: +39 0584 962313
fax: +39 0584 1660272
mob: +39 339 8844549

http://www.geo-solutions.it
http://twitter.com/geosolutions_it

AVVERTENZE AI SENSI DEL D.Lgs. 196/2003

Le informazioni contenute in questo messaggio di posta elettronica e/o nel/i file/s allegato/i sono da considerarsi strettamente riservate. Il loro utilizzo è consentito esclusivamente al destinatario del messaggio, per le finalità indicate nel messaggio stesso. Qualora riceviate questo messaggio senza esserne il destinatario, Vi preghiamo cortesemente di darcene notizia via e-mail e di procedere alla distruzione del messaggio stesso, cancellandolo dal Vostro sistema. Conservare il messaggio stesso, divulgarlo anche in parte, distribuirlo ad altri soggetti, copiarlo, od utilizzarlo per finalità diverse, costituisce comportamento contrario ai principi dettati dal D.Lgs. 196/2003.

The information in this message and/or attachments, is intended solely for the attention and use of the named addressee(s) and may be confidential or proprietary in nature or covered by the provisions of privacy act (Legislative Decree June, 30 2003, no.196 - Italy’s New Data Protection Code).Any use not in accord with its purpose, any disclosure, reproduction, copying, distribution, or either dissemination, either whole or partial, is strictly forbidden except previous formal approval of the named addressee(s). If you are not the intended recipient, please contact immediately the sender by telephone, fax or e-mail and delete the information in this message that has been received in error. The sender does not give any warranty or accept liability as the content, accuracy or completeness of sent messages and accepts no responsibility for changes made after they were sent or for other risks which arise as a result of e-mail transmission, viruses, etc.


Hi,
I can try to help. We use Windows and GeoServer at work.
But perhaps I need some guidance…
This is what the tests gives me. The failing one using file:// and the successful one using ./

SLD attribute
“file://images/rockFillSymbol.png”

Linkage
file://images/rockFillSymbol.png

Converted to
\images\rockFillSymbol.png

SLD attribute
“./images/rockFillSymbol.png”

Linkage
file:/C:/GitHub/geoserver/src/main/./target/default3970162196491181932data/styles/images/rockFillSymbol.png

Converted to
C:\GitHub\geoserver\src\main\target\default896598906399223139data\styles\images\rockFillSymbol.png

Should “file://images/rockFillSymbol.png” be interpreted as equal to “images/rockFillSymbol.png” and “./images/rockFillSymbol.png”?

According to http://stackoverflow.com/questions/7857416/file-uri-scheme-and-relative-files you cannot use the file protocol with relative paths.
So, perhaps this case shouldn’t be supported and the test should fail.

/Olle

···

On Sun, Jan 18, 2015 at 9:05 PM, Jody Garnett <jody.garnett@anonymised.com> wrote:

Okay we can consider it a goal for the year ( or a sprint activity if we get a Windows volunteer ).

On Sun, Jan 18, 2015 at 12:01 PM Andrea Aime <andrea.aime@anonymised.com.> wrote:


New Year. New Location. New Benefits. New Data Center in Ashburn, VA.
GigeNET is offering a free month of service with a new server in Ashburn.
Choose from 2 high performing configs, both with 100TB of bandwidth.
Higher redundancy.Lower latency.Increased capacity.Completely compliant.
http://p.sf.net/sfu/gigenet


Geoserver-devel mailing list
Geoserver-devel@anonymised.comsts.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geoserver-devel

On Sun, Jan 18, 2015 at 8:06 PM, Jody Garnett <jody.garnett@anonymised.com> wrote:

From Simone’s email to getools-devel:

a while ago we agree on having a windows build server for geotools
which is reachable here:
http://winbuild.geo-solutions.it/jenkins
credentials are the same for the linux build server.

I was able to restore the build for geotools-devel, but it looks like we have some failures for the geoserver master build target.

Failed tests: testSEStyleWithRelativePathProtocol(org.geoserver.catalog.ResourcePoolTest): expected:<C:.jenkins\jobs\GeoServer-Master\workspace\src\main\target\default4702095627624841408data\styles\images\rockFillSymbol.png> but was:<\images\rockFillSymbol.png>

A general call out on this one, I would like to ensure we build cleanly on windows (and take advantage of this machine that has been provided).

Hi Jody,
the GeoServer windows build server has never been made “official” because
I did not manage to get the build working on Windows in a stable way
(and got no help doing so either).

I have a windows laptop now that I can use to do some bits on Windows, but
honestly I despise the platform, so it’s more “call of duty” than something I want
to do in my spare time, it would need some champion that really
pushes for it.

Last time I checked there were some random failures in the WCS modules,
failure to delete some files in the temp data dirs, I guess we are still
keeping some reader/file handle open, but could not locate where.
I guess it got worse from there.

Cheers
Andrea

==

GeoServer Professional Services from the experts! Visit
http://goo.gl/NWWaa2 for more information.

==

Ing. Andrea Aime

@geowolf
Technical Lead

GeoSolutions S.A.S.
Via Poggio alle Viti 1187
55054 Massarosa (LU)
Italy
phone: +39 0584 962313
fax: +39 0584 1660272
mob: +39 339 8844549

http://www.geo-solutions.it
http://twitter.com/geosolutions_it

AVVERTENZE AI SENSI DEL D.Lgs. 196/2003

Le informazioni contenute in questo messaggio di posta elettronica e/o nel/i file/s allegato/i sono da considerarsi strettamente riservate. Il loro utilizzo è consentito esclusivamente al destinatario del messaggio, per le finalità indicate nel messaggio stesso. Qualora riceviate questo messaggio senza esserne il destinatario, Vi preghiamo cortesemente di darcene notizia via e-mail e di procedere alla distruzione del messaggio stesso, cancellandolo dal Vostro sistema. Conservare il messaggio stesso, divulgarlo anche in parte, distribuirlo ad altri soggetti, copiarlo, od utilizzarlo per finalità diverse, costituisce comportamento contrario ai principi dettati dal D.Lgs. 196/2003.

The information in this message and/or attachments, is intended solely for the attention and use of the named addressee(s) and may be confidential or proprietary in nature or covered by the provisions of privacy act (Legislative Decree June, 30 2003, no.196 - Italy’s New Data Protection Code).Any use not in accord with its purpose, any disclosure, reproduction, copying, distribution, or either dissemination, either whole or partial, is strictly forbidden except previous formal approval of the named addressee(s). If you are not the intended recipient, please contact immediately the sender by telephone, fax or e-mail and delete the information in this message that has been received in error. The sender does not give any warranty or accept liability as the content, accuracy or completeness of sent messages and accepts no responsibility for changes made after they were sent or for other risks which arise as a result of e-mail transmission, viruses, etc.


In this case we started with relative paths (and the base data directory) and turned the results into absolute paths as part of the test to make sure we are pointing at the right file.

The GeoTools method DataUtiltieis.urlToFile is what is being tested here, and it is letting us down!

public static File urlToFile(URL url) {
if (!“file”.equals(url.getProtocol())) {
return null; // not a File URL
}
String string = url.toExternalForm();
if (string.contains(“+”)) {
// this represents an invalid URL created using either
// file.toURL(); or
// file.toURI().toURL() on a specific version of Java 5 on Mac
string = string.replace(“+”, “%2B”);
}
try {
string = URLDecoder.decode(string, “UTF-8”);
} catch (UnsupportedEncodingException e) {
throw new RuntimeException(“Could not decode the URL to UTF-8 format”, e);
}
String path3;
String simplePrefix = “file:/”;
String standardPrefix = “file://”;
if (IS_WINDOWS_OS && string.startsWith(standardPrefix)) {
// win32: host/share reference
path3 = string.substring(standardPrefix.length() - 2);
} else if (string.startsWith(standardPrefix)) {
path3 = string.substring(standardPrefix.length());
} else if (string.startsWith(simplePrefix)) {
path3 = string.substring(simplePrefix.length() - 1);
} else {
String auth = url.getAuthority();
String path2 = url.getPath().replace(“%20”, " “);
if (auth != null && !auth.equals(”")) {
path3 = “//” + auth + path2;
} else {
path3 = path2;
}
}
return new File(path3);
}

As you can see we have one windows specific control path which we are not testing anywhere else! I think I may refactor this code to just be string based and take that windows flag as a parameter so we can test this on all platforms …

Note these methods were introduced when Java was doing a terrible job of File to/from URL conversion. They have correcting things a bit with file.toURI().toURL() …

If you could grab the geotools codebase and set up a test case that passes in the same strings that are failing in GeoServer it would be a great help…

···

On 18 January 2015 at 15:58, Olle Markljung <markljung@anonymised.com> wrote:

Hi,
I can try to help. We use Windows and GeoServer at work.
But perhaps I need some guidance…
This is what the tests gives me. The failing one using file:// and the successful one using ./

SLD attribute
“file://images/rockFillSymbol.png”

Linkage
file://images/rockFillSymbol.png

Converted to
\images\rockFillSymbol.png

SLD attribute
“./images/rockFillSymbol.png”

Linkage
file:/C:/GitHub/geoserver/src/main/./target/default3970162196491181932data/styles/images/rockFillSymbol.png

Converted to
C:\GitHub\geoserver\src\main\target\default896598906399223139data\styles\images\rockFillSymbol.png

Should “file://images/rockFillSymbol.png” be interpreted as equal to “images/rockFillSymbol.png” and “./images/rockFillSymbol.png”?

According to http://stackoverflow.com/questions/7857416/file-uri-scheme-and-relative-files you cannot use the file protocol with relative paths.
So, perhaps this case shouldn’t be supported and the test should fail.

/Olle


Jody Garnett

On Sun, Jan 18, 2015 at 9:05 PM, Jody Garnett <jody.garnett@anonymised.com> wrote:

Okay we can consider it a goal for the year ( or a sprint activity if we get a Windows volunteer ).

On Sun, Jan 18, 2015 at 12:01 PM Andrea Aime <andrea.aime@anonymised.com> wrote:


New Year. New Location. New Benefits. New Data Center in Ashburn, VA.
GigeNET is offering a free month of service with a new server in Ashburn.
Choose from 2 high performing configs, both with 100TB of bandwidth.
Higher redundancy.Lower latency.Increased capacity.Completely compliant.
http://p.sf.net/sfu/gigenet


Geoserver-devel mailing list
Geoserver-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geoserver-devel

On Sun, Jan 18, 2015 at 8:06 PM, Jody Garnett <jody.garnett@anonymised.com> wrote:

From Simone’s email to getools-devel:

a while ago we agree on having a windows build server for geotools
which is reachable here:
http://winbuild.geo-solutions.it/jenkins
credentials are the same for the linux build server.

I was able to restore the build for geotools-devel, but it looks like we have some failures for the geoserver master build target.

Failed tests: testSEStyleWithRelativePathProtocol(org.geoserver.catalog.ResourcePoolTest): expected:<C:.jenkins\jobs\GeoServer-Master\workspace\src\main\target\default4702095627624841408data\styles\images\rockFillSymbol.png> but was:<\images\rockFillSymbol.png>

A general call out on this one, I would like to ensure we build cleanly on windows (and take advantage of this machine that has been provided).

Hi Jody,
the GeoServer windows build server has never been made “official” because
I did not manage to get the build working on Windows in a stable way
(and got no help doing so either).

I have a windows laptop now that I can use to do some bits on Windows, but
honestly I despise the platform, so it’s more “call of duty” than something I want
to do in my spare time, it would need some champion that really
pushes for it.

Last time I checked there were some random failures in the WCS modules,
failure to delete some files in the temp data dirs, I guess we are still
keeping some reader/file handle open, but could not locate where.
I guess it got worse from there.

Cheers
Andrea

==

GeoServer Professional Services from the experts! Visit
http://goo.gl/NWWaa2 for more information.

==

Ing. Andrea Aime

@geowolf
Technical Lead

GeoSolutions S.A.S.
Via Poggio alle Viti 1187
55054 Massarosa (LU)
Italy
phone: +39 0584 962313
fax: +39 0584 1660272
mob: +39 339 8844549

http://www.geo-solutions.it
http://twitter.com/geosolutions_it

AVVERTENZE AI SENSI DEL D.Lgs. 196/2003

Le informazioni contenute in questo messaggio di posta elettronica e/o nel/i file/s allegato/i sono da considerarsi strettamente riservate. Il loro utilizzo è consentito esclusivamente al destinatario del messaggio, per le finalità indicate nel messaggio stesso. Qualora riceviate questo messaggio senza esserne il destinatario, Vi preghiamo cortesemente di darcene notizia via e-mail e di procedere alla distruzione del messaggio stesso, cancellandolo dal Vostro sistema. Conservare il messaggio stesso, divulgarlo anche in parte, distribuirlo ad altri soggetti, copiarlo, od utilizzarlo per finalità diverse, costituisce comportamento contrario ai principi dettati dal D.Lgs. 196/2003.

The information in this message and/or attachments, is intended solely for the attention and use of the named addressee(s) and may be confidential or proprietary in nature or covered by the provisions of privacy act (Legislative Decree June, 30 2003, no.196 - Italy’s New Data Protection Code).Any use not in accord with its purpose, any disclosure, reproduction, copying, distribution, or either dissemination, either whole or partial, is strictly forbidden except previous formal approval of the named addressee(s). If you are not the intended recipient, please contact immediately the sender by telephone, fax or e-mail and delete the information in this message that has been received in error. The sender does not give any warranty or accept liability as the content, accuracy or completeness of sent messages and accepts no responsibility for changes made after they were sent or for other risks which arise as a result of e-mail transmission, viruses, etc.


Ok!

I’ll look into it.

/Olle

måndag 19 januari 2015 skrev Jody Garnett <jody.garnett@anonymised.com>:

···


Jody Garnett

On 18 January 2015 at 15:58, Olle Markljung markljung@anonymised.com wrote:

Hi,
I can try to help. We use Windows and GeoServer at work.
But perhaps I need some guidance…
This is what the tests gives me. The failing one using file:// and the successful one using ./

SLD attribute
“file://images/rockFillSymbol.png”

Linkage
file://images/rockFillSymbol.png

Converted to
\images\rockFillSymbol.png

SLD attribute
“./images/rockFillSymbol.png”

Linkage
file:/C:/GitHub/geoserver/src/main/./target/default3970162196491181932data/styles/images/rockFillSymbol.png

Converted to
C:\GitHub\geoserver\src\main\target\default896598906399223139data\styles\images\rockFillSymbol.png

Should “file://images/rockFillSymbol.png” be interpreted as equal to “images/rockFillSymbol.png” and “./images/rockFillSymbol.png”?

According to http://stackoverflow.com/questions/7857416/file-uri-scheme-and-relative-files you cannot use the file protocol with relative paths.
So, perhaps this case shouldn’t be supported and the test should fail.

/Olle

On Sun, Jan 18, 2015 at 9:05 PM, Jody Garnett jody.garnett@anonymised.com wrote:

Okay we can consider it a goal for the year ( or a sprint activity if we get a Windows volunteer ).

On Sun, Jan 18, 2015 at 12:01 PM Andrea Aime andrea.aime@anonymised.com wrote:


New Year. New Location. New Benefits. New Data Center in Ashburn, VA.
GigeNET is offering a free month of service with a new server in Ashburn.
Choose from 2 high performing configs, both with 100TB of bandwidth.
Higher redundancy.Lower latency.Increased capacity.Completely compliant.
http://p.sf.net/sfu/gigenet


Geoserver-devel mailing list
Geoserver-devel@anonymised.comnet
https://lists.sourceforge.net/lists/listinfo/geoserver-devel

On Sun, Jan 18, 2015 at 8:06 PM, Jody Garnett jody.garnett@anonymised.com wrote:

From Simone’s email to getools-devel:

a while ago we agree on having a windows build server for geotools
which is reachable here:
http://winbuild.geo-solutions.it/jenkins
credentials are the same for the linux build server.

I was able to restore the build for geotools-devel, but it looks like we have some failures for the geoserver master build target.

Failed tests: testSEStyleWithRelativePathProtocol(org.geoserver.catalog.ResourcePoolTest): expected:<C:.jenkins\jobs\GeoServer-Master\workspace\src\main\target\default4702095627624841408data\styles\images\rockFillSymbol.png> but was:<\images\rockFillSymbol.png>

A general call out on this one, I would like to ensure we build cleanly on windows (and take advantage of this machine that has been provided).

Hi Jody,
the GeoServer windows build server has never been made “official” because
I did not manage to get the build working on Windows in a stable way
(and got no help doing so either).

I have a windows laptop now that I can use to do some bits on Windows, but
honestly I despise the platform, so it’s more “call of duty” than something I want
to do in my spare time, it would need some champion that really
pushes for it.

Last time I checked there were some random failures in the WCS modules,
failure to delete some files in the temp data dirs, I guess we are still
keeping some reader/file handle open, but could not locate where.
I guess it got worse from there.

Cheers
Andrea

==

GeoServer Professional Services from the experts! Visit
http://goo.gl/NWWaa2 for more information.

==

Ing. Andrea Aime

@geowolf
Technical Lead

GeoSolutions S.A.S.
Via Poggio alle Viti 1187
55054 Massarosa (LU)
Italy
phone: +39 0584 962313
fax: +39 0584 1660272
mob: +39 339 8844549

http://www.geo-solutions.it
http://twitter.com/geosolutions_it

AVVERTENZE AI SENSI DEL D.Lgs. 196/2003

Le informazioni contenute in questo messaggio di posta elettronica e/o nel/i file/s allegato/i sono da considerarsi strettamente riservate. Il loro utilizzo è consentito esclusivamente al destinatario del messaggio, per le finalità indicate nel messaggio stesso. Qualora riceviate questo messaggio senza esserne il destinatario, Vi preghiamo cortesemente di darcene notizia via e-mail e di procedere alla distruzione del messaggio stesso, cancellandolo dal Vostro sistema. Conservare il messaggio stesso, divulgarlo anche in parte, distribuirlo ad altri soggetti, copiarlo, od utilizzarlo per finalità diverse, costituisce comportamento contrario ai principi dettati dal D.Lgs. 196/2003.

The information in this message and/or attachments, is intended solely for the attention and use of the named addressee(s) and may be confidential or proprietary in nature or covered by the provisions of privacy act (Legislative Decree June, 30 2003, no.196 - Italy’s New Data Protection Code).Any use not in accord with its purpose, any disclosure, reproduction, copying, distribution, or either dissemination, either whole or partial, is strictly forbidden except previous formal approval of the named addressee(s). If you are not the intended recipient, please contact immediately the sender by telephone, fax or e-mail and delete the information in this message that has been received in error. The sender does not give any warranty or accept liability as the content, accuracy or completeness of sent messages and accepts no responsibility for changes made after they were sent or for other risks which arise as a result of e-mail transmission, viruses, etc.


On Mon, Jan 19, 2015 at 12:58 AM, Olle Markljung <markljung@anonymised.com>
wrote:

Should "file://images/rockFillSymbol.png" be interpreted as equal to
"images/rockFillSymbol.png" and "./images/rockFillSymbol.png"?
According to
http://stackoverflow.com/questions/7857416/file-uri-scheme-and-relative-files
you cannot use the file protocol with relative paths.
So, perhaps this case shouldn't be supported and the test should fail.

This approach has been working for years, people depend on it, so we are
definitely not going
to stop supporting it.
You'll find that any suggestion intentionally breaking backwards
compatibility will get a solid
and immediate -1 around here :wink:

Cheers
Andrea

--

GeoServer Professional Services from the experts! Visit
http://goo.gl/NWWaa2 for more information.

Ing. Andrea Aime
@geowolf
Technical Lead

GeoSolutions S.A.S.
Via Poggio alle Viti 1187
55054 Massarosa (LU)
Italy
phone: +39 0584 962313
fax: +39 0584 1660272
mob: +39 339 8844549

http://www.geo-solutions.it
http://twitter.com/geosolutions_it

*AVVERTENZE AI SENSI DEL D.Lgs. 196/2003*

Le informazioni contenute in questo messaggio di posta elettronica e/o
nel/i file/s allegato/i sono da considerarsi strettamente riservate. Il
loro utilizzo è consentito esclusivamente al destinatario del messaggio,
per le finalità indicate nel messaggio stesso. Qualora riceviate questo
messaggio senza esserne il destinatario, Vi preghiamo cortesemente di
darcene notizia via e-mail e di procedere alla distruzione del messaggio
stesso, cancellandolo dal Vostro sistema. Conservare il messaggio stesso,
divulgarlo anche in parte, distribuirlo ad altri soggetti, copiarlo, od
utilizzarlo per finalità diverse, costituisce comportamento contrario ai
principi dettati dal D.Lgs. 196/2003.

The information in this message and/or attachments, is intended solely for
the attention and use of the named addressee(s) and may be confidential or
proprietary in nature or covered by the provisions of privacy act
(Legislative Decree June, 30 2003, no.196 - Italy's New Data Protection
Code).Any use not in accord with its purpose, any disclosure, reproduction,
copying, distribution, or either dissemination, either whole or partial, is
strictly forbidden except previous formal approval of the named
addressee(s). If you are not the intended recipient, please contact
immediately the sender by telephone, fax or e-mail and delete the
information in this message that has been received in error. The sender
does not give any warranty or accept liability as the content, accuracy or
completeness of sent messages and accepts no responsibility for changes
made after they were sent or for other risks which arise as a result of
e-mail transmission, viruses, etc.

-------------------------------------------------------

Yes, I see now that I was a bit unclear with my intentions of the last paragraph.

However, I believe that the usage of the file protocol makes it unclear as when the file path will be interpreted as relative over absolute.

I can get back to you with some exemples and maybe we can document the requirements and expected behavior.

/Olle

måndag 19 januari 2015 skrev Andrea Aime <andrea.aime@anonymised.com>:

···

On Mon, Jan 19, 2015 at 12:58 AM, Olle Markljung markljung@anonymised.com wrote:

Should “file://images/rockFillSymbol.png” be interpreted as equal to “images/rockFillSymbol.png” and “./images/rockFillSymbol.png”?

According to http://stackoverflow.com/questions/7857416/file-uri-scheme-and-relative-files you cannot use the file protocol with relative paths.
So, perhaps this case shouldn’t be supported and the test should fail.

This approach has been working for years, people depend on it, so we are definitely not going
to stop supporting it.
You’ll find that any suggestion intentionally breaking backwards compatibility will get a solid
and immediate -1 around here :wink:

Cheers
Andrea

==

GeoServer Professional Services from the experts! Visit
http://goo.gl/NWWaa2 for more information.

==

Ing. Andrea Aime

@geowolf
Technical Lead

GeoSolutions S.A.S.
Via Poggio alle Viti 1187
55054 Massarosa (LU)
Italy
phone: +39 0584 962313
fax: +39 0584 1660272
mob: +39 339 8844549

http://www.geo-solutions.it
http://twitter.com/geosolutions_it

AVVERTENZE AI SENSI DEL D.Lgs. 196/2003

Le informazioni contenute in questo messaggio di posta elettronica e/o nel/i file/s allegato/i sono da considerarsi strettamente riservate. Il loro utilizzo è consentito esclusivamente al destinatario del messaggio, per le finalità indicate nel messaggio stesso. Qualora riceviate questo messaggio senza esserne il destinatario, Vi preghiamo cortesemente di darcene notizia via e-mail e di procedere alla distruzione del messaggio stesso, cancellandolo dal Vostro sistema. Conservare il messaggio stesso, divulgarlo anche in parte, distribuirlo ad altri soggetti, copiarlo, od utilizzarlo per finalità diverse, costituisce comportamento contrario ai principi dettati dal D.Lgs. 196/2003.

The information in this message and/or attachments, is intended solely for the attention and use of the named addressee(s) and may be confidential or proprietary in nature or covered by the provisions of privacy act (Legislative Decree June, 30 2003, no.196 - Italy’s New Data Protection Code).Any use not in accord with its purpose, any disclosure, reproduction, copying, distribution, or either dissemination, either whole or partial, is strictly forbidden except previous formal approval of the named addressee(s). If you are not the intended recipient, please contact immediately the sender by telephone, fax or e-mail and delete the information in this message that has been received in error. The sender does not give any warranty or accept liability as the content, accuracy or completeness of sent messages and accepts no responsibility for changes made after they were sent or for other risks which arise as a result of e-mail transmission, viruses, etc.


On Mon, Jan 19, 2015 at 9:29 AM, Olle Markljung <markljung@anonymised.com> wrote:

Yes, I see now that I was a bit unclear with my intentions of the last
paragraph.

However, I believe that the usage of the file protocol makes it unclear as
when the file path will be interpreted as relative over absolute.

I can get back to you with some exemples and maybe we can document the
requirements and expected behavior.

The test is checking for behavior that was working up to 2.5.x. Both worked:
file://dest.png
file.//./dest.png

Cheers
Andrea

--

GeoServer Professional Services from the experts! Visit
http://goo.gl/NWWaa2 for more information.

Ing. Andrea Aime
@geowolf
Technical Lead

GeoSolutions S.A.S.
Via Poggio alle Viti 1187
55054 Massarosa (LU)
Italy
phone: +39 0584 962313
fax: +39 0584 1660272
mob: +39 339 8844549

http://www.geo-solutions.it
http://twitter.com/geosolutions_it

*AVVERTENZE AI SENSI DEL D.Lgs. 196/2003*

Le informazioni contenute in questo messaggio di posta elettronica e/o
nel/i file/s allegato/i sono da considerarsi strettamente riservate. Il
loro utilizzo è consentito esclusivamente al destinatario del messaggio,
per le finalità indicate nel messaggio stesso. Qualora riceviate questo
messaggio senza esserne il destinatario, Vi preghiamo cortesemente di
darcene notizia via e-mail e di procedere alla distruzione del messaggio
stesso, cancellandolo dal Vostro sistema. Conservare il messaggio stesso,
divulgarlo anche in parte, distribuirlo ad altri soggetti, copiarlo, od
utilizzarlo per finalità diverse, costituisce comportamento contrario ai
principi dettati dal D.Lgs. 196/2003.

The information in this message and/or attachments, is intended solely for
the attention and use of the named addressee(s) and may be confidential or
proprietary in nature or covered by the provisions of privacy act
(Legislative Decree June, 30 2003, no.196 - Italy's New Data Protection
Code).Any use not in accord with its purpose, any disclosure, reproduction,
copying, distribution, or either dissemination, either whole or partial, is
strictly forbidden except previous formal approval of the named
addressee(s). If you are not the intended recipient, please contact
immediately the sender by telephone, fax or e-mail and delete the
information in this message that has been received in error. The sender
does not give any warranty or accept liability as the content, accuracy or
completeness of sent messages and accepts no responsibility for changes
made after they were sent or for other risks which arise as a result of
e-mail transmission, viruses, etc.

-------------------------------------------------------

Ok.
Looking at the history I can’t find anything that changed. Version of Java did change to 7.

These urls also fail in GeoTools.
It’s not the usage of urlToFile in the test that’s the problem but the usage of urlToFile in the DefaultResourceLocator.locateResource.
The file:// is converted to // by urlToFile and this means it’s not relative. You’ll get the same behavior if you use file:/ instead.
Therefore locateResource will leave the URI as is and not make it relative to the styles folder.

So, on Windows the urls will be interpreted as absolute but on other platforms it will be relative. Atleast the file:// case.
Should file://host/share/dest.png be supported on Windows but not elsewhere?
Would \host\share\dest.png work an be a acceptable replacement?
Is it intentional that file:/ will be an absolute path and file:// not?

Something that does work is removing the forward slashes. So, file:dest.png will give you the correct behavior. As https://jira.codehaus.org/browse/GEOT-4311 says.
I’m merely trying to understand the requirements so go easy on me :slight_smile:

/Olle

···

On Mon, Jan 19, 2015 at 9:40 AM, Andrea Aime <andrea.aime@anonymised.com> wrote:

On Mon, Jan 19, 2015 at 9:29 AM, Olle Markljung <markljung@anonymised.com> wrote:

Yes, I see now that I was a bit unclear with my intentions of the last paragraph.

However, I believe that the usage of the file protocol makes it unclear as when the file path will be interpreted as relative over absolute.

I can get back to you with some exemples and maybe we can document the requirements and expected behavior.

The test is checking for behavior that was working up to 2.5.x. Both worked:
file://dest.png
file.//./dest.png

Cheers

Andrea

==

GeoServer Professional Services from the experts! Visit
http://goo.gl/NWWaa2 for more information.

==

Ing. Andrea Aime

@geowolf
Technical Lead

GeoSolutions S.A.S.
Via Poggio alle Viti 1187
55054 Massarosa (LU)
Italy
phone: +39 0584 962313
fax: +39 0584 1660272
mob: +39 339 8844549

http://www.geo-solutions.it
http://twitter.com/geosolutions_it

AVVERTENZE AI SENSI DEL D.Lgs. 196/2003

Le informazioni contenute in questo messaggio di posta elettronica e/o nel/i file/s allegato/i sono da considerarsi strettamente riservate. Il loro utilizzo è consentito esclusivamente al destinatario del messaggio, per le finalità indicate nel messaggio stesso. Qualora riceviate questo messaggio senza esserne il destinatario, Vi preghiamo cortesemente di darcene notizia via e-mail e di procedere alla distruzione del messaggio stesso, cancellandolo dal Vostro sistema. Conservare il messaggio stesso, divulgarlo anche in parte, distribuirlo ad altri soggetti, copiarlo, od utilizzarlo per finalità diverse, costituisce comportamento contrario ai principi dettati dal D.Lgs. 196/2003.

The information in this message and/or attachments, is intended solely for the attention and use of the named addressee(s) and may be confidential or proprietary in nature or covered by the provisions of privacy act (Legislative Decree June, 30 2003, no.196 - Italy’s New Data Protection Code).Any use not in accord with its purpose, any disclosure, reproduction, copying, distribution, or either dissemination, either whole or partial, is strictly forbidden except previous formal approval of the named addressee(s). If you are not the intended recipient, please contact immediately the sender by telephone, fax or e-mail and delete the information in this message that has been received in error. The sender does not give any warranty or accept liability as the content, accuracy or completeness of sent messages and accepts no responsibility for changes made after they were sent or for other risks which arise as a result of e-mail transmission, viruses, etc.


Ok,
Perhaps it is not easy to say in what ways it all should work. Someone ought to be depending on the code doing this specific thing on Windows since the code exists.
So, I got a proposition.
What if we in DataUtilities.urlToFile do the same as DefaultResourceLocator.locateResource already does.
That is to check if the file exists. If it doesn’t we remove the extra slashes and makes the URI relative.

I extended the tests and added such code to the Windows specific case and it works as expected.
Should I create a ticket and send a PR with these changes for your review?

/Olle

···

On Mon, Jan 19, 2015 at 11:42 PM, Olle Markljung <markljung@anonymised.com> wrote:

Ok.
Looking at the history I can’t find anything that changed. Version of Java did change to 7.

These urls also fail in GeoTools.
It’s not the usage of urlToFile in the test that’s the problem but the usage of urlToFile in the DefaultResourceLocator.locateResource.
The file:// is converted to // by urlToFile and this means it’s not relative. You’ll get the same behavior if you use file:/ instead.
Therefore locateResource will leave the URI as is and not make it relative to the styles folder.

So, on Windows the urls will be interpreted as absolute but on other platforms it will be relative. Atleast the file:// case.
Should file://host/share/dest.png be supported on Windows but not elsewhere?
Would \host\share\dest.png work an be a acceptable replacement?
Is it intentional that file:/ will be an absolute path and file:// not?

Something that does work is removing the forward slashes. So, file:dest.png will give you the correct behavior. As https://jira.codehaus.org/browse/GEOT-4311 says.
I’m merely trying to understand the requirements so go easy on me :slight_smile:

/Olle

On Mon, Jan 19, 2015 at 9:40 AM, Andrea Aime <andrea.aime@anonymised.com> wrote:

On Mon, Jan 19, 2015 at 9:29 AM, Olle Markljung <markljung@anonymised.com> wrote:

Yes, I see now that I was a bit unclear with my intentions of the last paragraph.

However, I believe that the usage of the file protocol makes it unclear as when the file path will be interpreted as relative over absolute.

I can get back to you with some exemples and maybe we can document the requirements and expected behavior.

The test is checking for behavior that was working up to 2.5.x. Both worked:
file://dest.png
file.//./dest.png

Cheers

Andrea

==

GeoServer Professional Services from the experts! Visit
http://goo.gl/NWWaa2 for more information.

==

Ing. Andrea Aime

@geowolf
Technical Lead

GeoSolutions S.A.S.
Via Poggio alle Viti 1187
55054 Massarosa (LU)
Italy
phone: +39 0584 962313
fax: +39 0584 1660272
mob: +39 339 8844549

http://www.geo-solutions.it
http://twitter.com/geosolutions_it

AVVERTENZE AI SENSI DEL D.Lgs. 196/2003

Le informazioni contenute in questo messaggio di posta elettronica e/o nel/i file/s allegato/i sono da considerarsi strettamente riservate. Il loro utilizzo è consentito esclusivamente al destinatario del messaggio, per le finalità indicate nel messaggio stesso. Qualora riceviate questo messaggio senza esserne il destinatario, Vi preghiamo cortesemente di darcene notizia via e-mail e di procedere alla distruzione del messaggio stesso, cancellandolo dal Vostro sistema. Conservare il messaggio stesso, divulgarlo anche in parte, distribuirlo ad altri soggetti, copiarlo, od utilizzarlo per finalità diverse, costituisce comportamento contrario ai principi dettati dal D.Lgs. 196/2003.

The information in this message and/or attachments, is intended solely for the attention and use of the named addressee(s) and may be confidential or proprietary in nature or covered by the provisions of privacy act (Legislative Decree June, 30 2003, no.196 - Italy’s New Data Protection Code).Any use not in accord with its purpose, any disclosure, reproduction, copying, distribution, or either dissemination, either whole or partial, is strictly forbidden except previous formal approval of the named addressee(s). If you are not the intended recipient, please contact immediately the sender by telephone, fax or e-mail and delete the information in this message that has been received in error. The sender does not give any warranty or accept liability as the content, accuracy or completeness of sent messages and accepts no responsibility for changes made after they were sent or for other risks which arise as a result of e-mail transmission, viruses, etc.


Sounds like a good approach - we may also be able to use Java 7 Path API.

I should also point out that we may be using this to figure out a the location of a new file (like one that does not exist yet). Perhaps the Java 7 path api can help.

···

On 20 January 2015 at 15:28, Olle Markljung <markljung@anonymised.com> wrote:

Ok,
Perhaps it is not easy to say in what ways it all should work. Someone ought to be depending on the code doing this specific thing on Windows since the code exists.
So, I got a proposition.
What if we in DataUtilities.urlToFile do the same as DefaultResourceLocator.locateResource already does.
That is to check if the file exists. If it doesn’t we remove the extra slashes and makes the URI relative.

I extended the tests and added such code to the Windows specific case and it works as expected.
Should I create a ticket and send a PR with these changes for your review?

/Olle


Jody Garnett

On Mon, Jan 19, 2015 at 11:42 PM, Olle Markljung <markljung@anonymised.com> wrote:

Ok.
Looking at the history I can’t find anything that changed. Version of Java did change to 7.

These urls also fail in GeoTools.
It’s not the usage of urlToFile in the test that’s the problem but the usage of urlToFile in the DefaultResourceLocator.locateResource.
The file:// is converted to // by urlToFile and this means it’s not relative. You’ll get the same behavior if you use file:/ instead.
Therefore locateResource will leave the URI as is and not make it relative to the styles folder.

So, on Windows the urls will be interpreted as absolute but on other platforms it will be relative. Atleast the file:// case.
Should file://host/share/dest.png be supported on Windows but not elsewhere?
Would \host\share\dest.png work an be a acceptable replacement?
Is it intentional that file:/ will be an absolute path and file:// not?

Something that does work is removing the forward slashes. So, file:dest.png will give you the correct behavior. As https://jira.codehaus.org/browse/GEOT-4311 says.
I’m merely trying to understand the requirements so go easy on me :slight_smile:

/Olle

On Mon, Jan 19, 2015 at 9:40 AM, Andrea Aime <andrea.aime@anonymised.com> wrote:

On Mon, Jan 19, 2015 at 9:29 AM, Olle Markljung <markljung@anonymised.com…> wrote:

Yes, I see now that I was a bit unclear with my intentions of the last paragraph.

However, I believe that the usage of the file protocol makes it unclear as when the file path will be interpreted as relative over absolute.

I can get back to you with some exemples and maybe we can document the requirements and expected behavior.

The test is checking for behavior that was working up to 2.5.x. Both worked:
file://dest.png
file.//./dest.png

Cheers

Andrea

==

GeoServer Professional Services from the experts! Visit
http://goo.gl/NWWaa2 for more information.

==

Ing. Andrea Aime

@geowolf
Technical Lead

GeoSolutions S.A.S.
Via Poggio alle Viti 1187
55054 Massarosa (LU)
Italy
phone: +39 0584 962313
fax: +39 0584 1660272
mob: +39 339 8844549

http://www.geo-solutions.it
http://twitter.com/geosolutions_it

AVVERTENZE AI SENSI DEL D.Lgs. 196/2003

Le informazioni contenute in questo messaggio di posta elettronica e/o nel/i file/s allegato/i sono da considerarsi strettamente riservate. Il loro utilizzo è consentito esclusivamente al destinatario del messaggio, per le finalità indicate nel messaggio stesso. Qualora riceviate questo messaggio senza esserne il destinatario, Vi preghiamo cortesemente di darcene notizia via e-mail e di procedere alla distruzione del messaggio stesso, cancellandolo dal Vostro sistema. Conservare il messaggio stesso, divulgarlo anche in parte, distribuirlo ad altri soggetti, copiarlo, od utilizzarlo per finalità diverse, costituisce comportamento contrario ai principi dettati dal D.Lgs. 196/2003.

The information in this message and/or attachments, is intended solely for the attention and use of the named addressee(s) and may be confidential or proprietary in nature or covered by the provisions of privacy act (Legislative Decree June, 30 2003, no.196 - Italy’s New Data Protection Code).Any use not in accord with its purpose, any disclosure, reproduction, copying, distribution, or either dissemination, either whole or partial, is strictly forbidden except previous formal approval of the named addressee(s). If you are not the intended recipient, please contact immediately the sender by telephone, fax or e-mail and delete the information in this message that has been received in error. The sender does not give any warranty or accept liability as the content, accuracy or completeness of sent messages and accepts no responsibility for changes made after they were sent or for other risks which arise as a result of e-mail transmission, viruses, etc.


Sorry for the delay.

Ticket: http://jira.codehaus.org/browse/GEOT-4990
PR: https://github.com/geotools/geotools/pull/717

This builds clean using “mvn clean install” on my machine (building geotools).
Should I communicate this on the geotools list as well?

Not sure that I understand what you mean Jody.
If I have gotten this right the problem is to know if the user provide an absolute or relative path.
How would the path API help in knowing the intentions of the user?

/Olle

···

On Wed, Jan 21, 2015 at 12:33 AM, Jody Garnett <jody.garnett@anonymised.com> wrote:

Sounds like a good approach - we may also be able to use Java 7 Path API.

I should also point out that we may be using this to figure out a the location of a new file (like one that does not exist yet). Perhaps the Java 7 path api can help.

Jody


Jody Garnett

On 20 January 2015 at 15:28, Olle Markljung <markljung@anonymised.com> wrote:

Ok,
Perhaps it is not easy to say in what ways it all should work. Someone ought to be depending on the code doing this specific thing on Windows since the code exists.
So, I got a proposition.
What if we in DataUtilities.urlToFile do the same as DefaultResourceLocator.locateResource already does.
That is to check if the file exists. If it doesn’t we remove the extra slashes and makes the URI relative.

I extended the tests and added such code to the Windows specific case and it works as expected.
Should I create a ticket and send a PR with these changes for your review?

/Olle

On Mon, Jan 19, 2015 at 11:42 PM, Olle Markljung <markljung@anonymised.com> wrote:

Ok.
Looking at the history I can’t find anything that changed. Version of Java did change to 7.

These urls also fail in GeoTools.
It’s not the usage of urlToFile in the test that’s the problem but the usage of urlToFile in the DefaultResourceLocator.locateResource.
The file:// is converted to // by urlToFile and this means it’s not relative. You’ll get the same behavior if you use file:/ instead.
Therefore locateResource will leave the URI as is and not make it relative to the styles folder.

So, on Windows the urls will be interpreted as absolute but on other platforms it will be relative. Atleast the file:// case.
Should file://host/share/dest.png be supported on Windows but not elsewhere?
Would \host\share\dest.png work an be a acceptable replacement?
Is it intentional that file:/ will be an absolute path and file:// not?

Something that does work is removing the forward slashes. So, file:dest.png will give you the correct behavior. As https://jira.codehaus.org/browse/GEOT-4311 says.
I’m merely trying to understand the requirements so go easy on me :slight_smile:

/Olle

On Mon, Jan 19, 2015 at 9:40 AM, Andrea Aime <andrea.aime@anonymised.com> wrote:

On Mon, Jan 19, 2015 at 9:29 AM, Olle Markljung <markljung@anonymised.com> wrote:

Yes, I see now that I was a bit unclear with my intentions of the last paragraph.

However, I believe that the usage of the file protocol makes it unclear as when the file path will be interpreted as relative over absolute.

I can get back to you with some exemples and maybe we can document the requirements and expected behavior.

The test is checking for behavior that was working up to 2.5.x. Both worked:
file://dest.png
file.//./dest.png

Cheers

Andrea

==

GeoServer Professional Services from the experts! Visit
http://goo.gl/NWWaa2 for more information.

==

Ing. Andrea Aime

@geowolf
Technical Lead

GeoSolutions S.A.S.
Via Poggio alle Viti 1187
55054 Massarosa (LU)
Italy
phone: +39 0584 962313
fax: +39 0584 1660272
mob: +39 339 8844549

http://www.geo-solutions.it
http://twitter.com/geosolutions_it

AVVERTENZE AI SENSI DEL D.Lgs. 196/2003

Le informazioni contenute in questo messaggio di posta elettronica e/o nel/i file/s allegato/i sono da considerarsi strettamente riservate. Il loro utilizzo è consentito esclusivamente al destinatario del messaggio, per le finalità indicate nel messaggio stesso. Qualora riceviate questo messaggio senza esserne il destinatario, Vi preghiamo cortesemente di darcene notizia via e-mail e di procedere alla distruzione del messaggio stesso, cancellandolo dal Vostro sistema. Conservare il messaggio stesso, divulgarlo anche in parte, distribuirlo ad altri soggetti, copiarlo, od utilizzarlo per finalità diverse, costituisce comportamento contrario ai principi dettati dal D.Lgs. 196/2003.

The information in this message and/or attachments, is intended solely for the attention and use of the named addressee(s) and may be confidential or proprietary in nature or covered by the provisions of privacy act (Legislative Decree June, 30 2003, no.196 - Italy’s New Data Protection Code).Any use not in accord with its purpose, any disclosure, reproduction, copying, distribution, or either dissemination, either whole or partial, is strictly forbidden except previous formal approval of the named addressee(s). If you are not the intended recipient, please contact immediately the sender by telephone, fax or e-mail and delete the information in this message that has been received in error. The sender does not give any warranty or accept liability as the content, accuracy or completeness of sent messages and accepts no responsibility for changes made after they were sent or for other risks which arise as a result of e-mail transmission, viruses, etc.


One of the lines in your pull request uses the test if (!f.exists())

This only works if the DataUtilities.urlToFile method is referring to a file that has been created yet. If we try the same logic on a file that does not exist yet it will fail…

···

On 1 February 2015 at 06:36, Olle Markljung <markljung@anonymised.com> wrote:

Sorry for the delay.

Ticket: http://jira.codehaus.org/browse/GEOT-4990
PR: https://github.com/geotools/geotools/pull/717

This builds clean using “mvn clean install” on my machine (building geotools).
Should I communicate this on the geotools list as well?

Not sure that I understand what you mean Jody.
If I have gotten this right the problem is to know if the user provide an absolute or relative path.
How would the path API help in knowing the intentions of the user?

/Olle


Jody Garnett

On Wed, Jan 21, 2015 at 12:33 AM, Jody Garnett <jody.garnett@anonymised.com> wrote:

Sounds like a good approach - we may also be able to use Java 7 Path API.

I should also point out that we may be using this to figure out a the location of a new file (like one that does not exist yet). Perhaps the Java 7 path api can help.

Jody


Jody Garnett

On 20 January 2015 at 15:28, Olle Markljung <markljung@anonymised.com> wrote:

Ok,
Perhaps it is not easy to say in what ways it all should work. Someone ought to be depending on the code doing this specific thing on Windows since the code exists.
So, I got a proposition.
What if we in DataUtilities.urlToFile do the same as DefaultResourceLocator.locateResource already does.
That is to check if the file exists. If it doesn’t we remove the extra slashes and makes the URI relative.

I extended the tests and added such code to the Windows specific case and it works as expected.
Should I create a ticket and send a PR with these changes for your review?

/Olle

On Mon, Jan 19, 2015 at 11:42 PM, Olle Markljung <markljung@anonymised.com> wrote:

Ok.
Looking at the history I can’t find anything that changed. Version of Java did change to 7.

These urls also fail in GeoTools.
It’s not the usage of urlToFile in the test that’s the problem but the usage of urlToFile in the DefaultResourceLocator.locateResource.
The file:// is converted to // by urlToFile and this means it’s not relative. You’ll get the same behavior if you use file:/ instead.
Therefore locateResource will leave the URI as is and not make it relative to the styles folder.

So, on Windows the urls will be interpreted as absolute but on other platforms it will be relative. Atleast the file:// case.
Should file://host/share/dest.png be supported on Windows but not elsewhere?
Would \host\share\dest.png work an be a acceptable replacement?
Is it intentional that file:/ will be an absolute path and file:// not?

Something that does work is removing the forward slashes. So, file:dest.png will give you the correct behavior. As https://jira.codehaus.org/browse/GEOT-4311 says.
I’m merely trying to understand the requirements so go easy on me :slight_smile:

/Olle

On Mon, Jan 19, 2015 at 9:40 AM, Andrea Aime <andrea.aime@anonymised.com> wrote:

On Mon, Jan 19, 2015 at 9:29 AM, Olle Markljung <markljung@anonymised.com> wrote:

Yes, I see now that I was a bit unclear with my intentions of the last paragraph.

However, I believe that the usage of the file protocol makes it unclear as when the file path will be interpreted as relative over absolute.

I can get back to you with some exemples and maybe we can document the requirements and expected behavior.

The test is checking for behavior that was working up to 2.5.x. Both worked:
file://dest.png
file.//./dest.png

Cheers

Andrea

==

GeoServer Professional Services from the experts! Visit
http://goo.gl/NWWaa2 for more information.

==

Ing. Andrea Aime

@geowolf
Technical Lead

GeoSolutions S.A.S.
Via Poggio alle Viti 1187
55054 Massarosa (LU)
Italy
phone: +39 0584 962313
fax: +39 0584 1660272
mob: +39 339 8844549

http://www.geo-solutions.it
http://twitter.com/geosolutions_it

AVVERTENZE AI SENSI DEL D.Lgs. 196/2003

Le informazioni contenute in questo messaggio di posta elettronica e/o nel/i file/s allegato/i sono da considerarsi strettamente riservate. Il loro utilizzo è consentito esclusivamente al destinatario del messaggio, per le finalità indicate nel messaggio stesso. Qualora riceviate questo messaggio senza esserne il destinatario, Vi preghiamo cortesemente di darcene notizia via e-mail e di procedere alla distruzione del messaggio stesso, cancellandolo dal Vostro sistema. Conservare il messaggio stesso, divulgarlo anche in parte, distribuirlo ad altri soggetti, copiarlo, od utilizzarlo per finalità diverse, costituisce comportamento contrario ai principi dettati dal D.Lgs. 196/2003.

The information in this message and/or attachments, is intended solely for the attention and use of the named addressee(s) and may be confidential or proprietary in nature or covered by the provisions of privacy act (Legislative Decree June, 30 2003, no.196 - Italy’s New Data Protection Code).Any use not in accord with its purpose, any disclosure, reproduction, copying, distribution, or either dissemination, either whole or partial, is strictly forbidden except previous formal approval of the named addressee(s). If you are not the intended recipient, please contact immediately the sender by telephone, fax or e-mail and delete the information in this message that has been received in error. The sender does not give any warranty or accept liability as the content, accuracy or completeness of sent messages and accepts no responsibility for changes made after they were sent or for other risks which arise as a result of e-mail transmission, viruses, etc.


Yes, it checks if the file at the absolute path exists and if not assumes the path is relative.
You could swap the order making the relative path default.

Fail? The tests use paths to files that does not exist.
The result will be that the path to the file will be relative. And that’s the same result that you get on other platforms.

But yes, if your intention is to use an absolute path to something that does not exist yet it will fail.
So, by swapping the order you would get the absoulte file path as the default on windows as before for files that do not exist yet.
But if there is a file there for the relative case that will be used instead. However that seems a bit unlikely.
Should I update the PR to do that?

Still not sure how the Java Path API would help here.

/Olle

···

On Sun, Feb 1, 2015 at 11:26 PM, Jody Garnett <jody.garnett@anonymised.com> wrote:

One of the lines in your pull request uses the test if (!f.exists())

This only works if the DataUtilities.urlToFile method is referring to a file that has been created yet. If we try the same logic on a file that does not exist yet it will fail…


Jody Garnett

On 1 February 2015 at 06:36, Olle Markljung <markljung@anonymised.com> wrote:

Sorry for the delay.

Ticket: http://jira.codehaus.org/browse/GEOT-4990
PR: https://github.com/geotools/geotools/pull/717

This builds clean using “mvn clean install” on my machine (building geotools).
Should I communicate this on the geotools list as well?

Not sure that I understand what you mean Jody.
If I have gotten this right the problem is to know if the user provide an absolute or relative path.
How would the path API help in knowing the intentions of the user?

/Olle

On Wed, Jan 21, 2015 at 12:33 AM, Jody Garnett <jody.garnett@anonymised.com> wrote:

Sounds like a good approach - we may also be able to use Java 7 Path API.

I should also point out that we may be using this to figure out a the location of a new file (like one that does not exist yet). Perhaps the Java 7 path api can help.

Jody


Jody Garnett

On 20 January 2015 at 15:28, Olle Markljung <markljung@anonymised.com> wrote:

Ok,
Perhaps it is not easy to say in what ways it all should work. Someone ought to be depending on the code doing this specific thing on Windows since the code exists.
So, I got a proposition.
What if we in DataUtilities.urlToFile do the same as DefaultResourceLocator.locateResource already does.
That is to check if the file exists. If it doesn’t we remove the extra slashes and makes the URI relative.

I extended the tests and added such code to the Windows specific case and it works as expected.
Should I create a ticket and send a PR with these changes for your review?

/Olle

On Mon, Jan 19, 2015 at 11:42 PM, Olle Markljung <markljung@anonymised.com> wrote:

Ok.
Looking at the history I can’t find anything that changed. Version of Java did change to 7.

These urls also fail in GeoTools.
It’s not the usage of urlToFile in the test that’s the problem but the usage of urlToFile in the DefaultResourceLocator.locateResource.
The file:// is converted to // by urlToFile and this means it’s not relative. You’ll get the same behavior if you use file:/ instead.
Therefore locateResource will leave the URI as is and not make it relative to the styles folder.

So, on Windows the urls will be interpreted as absolute but on other platforms it will be relative. Atleast the file:// case.
Should file://host/share/dest.png be supported on Windows but not elsewhere?
Would \host\share\dest.png work an be a acceptable replacement?
Is it intentional that file:/ will be an absolute path and file:// not?

Something that does work is removing the forward slashes. So, file:dest.png will give you the correct behavior. As https://jira.codehaus.org/browse/GEOT-4311 says.
I’m merely trying to understand the requirements so go easy on me :slight_smile:

/Olle

On Mon, Jan 19, 2015 at 9:40 AM, Andrea Aime <andrea.aime@anonymised.com> wrote:

On Mon, Jan 19, 2015 at 9:29 AM, Olle Markljung <markljung@anonymised.com> wrote:

Yes, I see now that I was a bit unclear with my intentions of the last paragraph.

However, I believe that the usage of the file protocol makes it unclear as when the file path will be interpreted as relative over absolute.

I can get back to you with some exemples and maybe we can document the requirements and expected behavior.

The test is checking for behavior that was working up to 2.5.x. Both worked:
file://dest.png
file.//./dest.png

Cheers

Andrea

==

GeoServer Professional Services from the experts! Visit
http://goo.gl/NWWaa2 for more information.

==

Ing. Andrea Aime

@geowolf
Technical Lead

GeoSolutions S.A.S.
Via Poggio alle Viti 1187
55054 Massarosa (LU)
Italy
phone: +39 0584 962313
fax: +39 0584 1660272
mob: +39 339 8844549

http://www.geo-solutions.it
http://twitter.com/geosolutions_it

AVVERTENZE AI SENSI DEL D.Lgs. 196/2003

Le informazioni contenute in questo messaggio di posta elettronica e/o nel/i file/s allegato/i sono da considerarsi strettamente riservate. Il loro utilizzo è consentito esclusivamente al destinatario del messaggio, per le finalità indicate nel messaggio stesso. Qualora riceviate questo messaggio senza esserne il destinatario, Vi preghiamo cortesemente di darcene notizia via e-mail e di procedere alla distruzione del messaggio stesso, cancellandolo dal Vostro sistema. Conservare il messaggio stesso, divulgarlo anche in parte, distribuirlo ad altri soggetti, copiarlo, od utilizzarlo per finalità diverse, costituisce comportamento contrario ai principi dettati dal D.Lgs. 196/2003.

The information in this message and/or attachments, is intended solely for the attention and use of the named addressee(s) and may be confidential or proprietary in nature or covered by the provisions of privacy act (Legislative Decree June, 30 2003, no.196 - Italy’s New Data Protection Code).Any use not in accord with its purpose, any disclosure, reproduction, copying, distribution, or either dissemination, either whole or partial, is strictly forbidden except previous formal approval of the named addressee(s). If you are not the intended recipient, please contact immediately the sender by telephone, fax or e-mail and delete the information in this message that has been received in error. The sender does not give any warranty or accept liability as the content, accuracy or completeness of sent messages and accepts no responsibility for changes made after they were sent or for other risks which arise as a result of e-mail transmission, viruses, etc.


Since (in GeoServer) we try to discourage the use of absolute paths I guess I am fine with the way things stand.

If we were just focused on manipulating the file path in a safe manner the Java Paths API would help (since we actually want to check a file on disk I expect the Files API is fine).

Correct me if I am wrong, and absolute path on windows will end up using the C:\ notation or the \host\share\ notation - in both cases these are pretty specific and not confusable as a relative path?

···

On 2 February 2015 at 14:52, Olle Markljung <markljung@anonymised.com> wrote:

Yes, it checks if the file at the absolute path exists and if not assumes the path is relative.
You could swap the order making the relative path default.

Fail? The tests use paths to files that does not exist.
The result will be that the path to the file will be relative. And that’s the same result that you get on other platforms.

But yes, if your intention is to use an absolute path to something that does not exist yet it will fail.
So, by swapping the order you would get the absoulte file path as the default on windows as before for files that do not exist yet.
But if there is a file there for the relative case that will be used instead. However that seems a bit unlikely.
Should I update the PR to do that?

Still not sure how the Java Path API would help here.

/Olle


Jody Garnett

On Sun, Feb 1, 2015 at 11:26 PM, Jody Garnett <jody.garnett@…403…> wrote:

One of the lines in your pull request uses the test if (!f.exists())

This only works if the DataUtilities.urlToFile method is referring to a file that has been created yet. If we try the same logic on a file that does not exist yet it will fail…


Jody Garnett

On 1 February 2015 at 06:36, Olle Markljung <markljung@anonymised.com> wrote:

Sorry for the delay.

Ticket: http://jira.codehaus.org/browse/GEOT-4990
PR: https://github.com/geotools/geotools/pull/717

This builds clean using “mvn clean install” on my machine (building geotools).
Should I communicate this on the geotools list as well?

Not sure that I understand what you mean Jody.
If I have gotten this right the problem is to know if the user provide an absolute or relative path.
How would the path API help in knowing the intentions of the user?

/Olle

On Wed, Jan 21, 2015 at 12:33 AM, Jody Garnett <jody.garnett@anonymised.com> wrote:

Sounds like a good approach - we may also be able to use Java 7 Path API.

I should also point out that we may be using this to figure out a the location of a new file (like one that does not exist yet). Perhaps the Java 7 path api can help.

Jody


Jody Garnett

On 20 January 2015 at 15:28, Olle Markljung <markljung@anonymised.com> wrote:

Ok,
Perhaps it is not easy to say in what ways it all should work. Someone ought to be depending on the code doing this specific thing on Windows since the code exists.
So, I got a proposition.
What if we in DataUtilities.urlToFile do the same as DefaultResourceLocator.locateResource already does.
That is to check if the file exists. If it doesn’t we remove the extra slashes and makes the URI relative.

I extended the tests and added such code to the Windows specific case and it works as expected.
Should I create a ticket and send a PR with these changes for your review?

/Olle

On Mon, Jan 19, 2015 at 11:42 PM, Olle Markljung <markljung@anonymised.com> wrote:

Ok.
Looking at the history I can’t find anything that changed. Version of Java did change to 7.

These urls also fail in GeoTools.
It’s not the usage of urlToFile in the test that’s the problem but the usage of urlToFile in the DefaultResourceLocator.locateResource.
The file:// is converted to // by urlToFile and this means it’s not relative. You’ll get the same behavior if you use file:/ instead.
Therefore locateResource will leave the URI as is and not make it relative to the styles folder.

So, on Windows the urls will be interpreted as absolute but on other platforms it will be relative. Atleast the file:// case.
Should file://host/share/dest.png be supported on Windows but not elsewhere?
Would \host\share\dest.png work an be a acceptable replacement?
Is it intentional that file:/ will be an absolute path and file:// not?

Something that does work is removing the forward slashes. So, file:dest.png will give you the correct behavior. As https://jira.codehaus.org/browse/GEOT-4311 says.
I’m merely trying to understand the requirements so go easy on me :slight_smile:

/Olle

On Mon, Jan 19, 2015 at 9:40 AM, Andrea Aime <andrea.aime@anonymised.com> wrote:

On Mon, Jan 19, 2015 at 9:29 AM, Olle Markljung <markljung@anonymised.com> wrote:

Yes, I see now that I was a bit unclear with my intentions of the last paragraph.

However, I believe that the usage of the file protocol makes it unclear as when the file path will be interpreted as relative over absolute.

I can get back to you with some exemples and maybe we can document the requirements and expected behavior.

The test is checking for behavior that was working up to 2.5.x. Both worked:
file://dest.png
file.//./dest.png

Cheers

Andrea

==

GeoServer Professional Services from the experts! Visit
http://goo.gl/NWWaa2 for more information.

==

Ing. Andrea Aime

@geowolf
Technical Lead

GeoSolutions S.A.S.
Via Poggio alle Viti 1187
55054 Massarosa (LU)
Italy
phone: +39 0584 962313
fax: +39 0584 1660272
mob: +39 339 8844549

http://www.geo-solutions.it
http://twitter.com/geosolutions_it

AVVERTENZE AI SENSI DEL D.Lgs. 196/2003

Le informazioni contenute in questo messaggio di posta elettronica e/o nel/i file/s allegato/i sono da considerarsi strettamente riservate. Il loro utilizzo è consentito esclusivamente al destinatario del messaggio, per le finalità indicate nel messaggio stesso. Qualora riceviate questo messaggio senza esserne il destinatario, Vi preghiamo cortesemente di darcene notizia via e-mail e di procedere alla distruzione del messaggio stesso, cancellandolo dal Vostro sistema. Conservare il messaggio stesso, divulgarlo anche in parte, distribuirlo ad altri soggetti, copiarlo, od utilizzarlo per finalità diverse, costituisce comportamento contrario ai principi dettati dal D.Lgs. 196/2003.

The information in this message and/or attachments, is intended solely for the attention and use of the named addressee(s) and may be confidential or proprietary in nature or covered by the provisions of privacy act (Legislative Decree June, 30 2003, no.196 - Italy’s New Data Protection Code).Any use not in accord with its purpose, any disclosure, reproduction, copying, distribution, or either dissemination, either whole or partial, is strictly forbidden except previous formal approval of the named addressee(s). If you are not the intended recipient, please contact immediately the sender by telephone, fax or e-mail and delete the information in this message that has been received in error. The sender does not give any warranty or accept liability as the content, accuracy or completeness of sent messages and accepts no responsibility for changes made after they were sent or for other risks which arise as a result of e-mail transmission, viruses, etc.


Yes it will.
But the problem here is that we do not know the intention of the user.
Existing code has a different handling of file:// urls on windows than on other platforms.
On Windows file://images/poi.png will be converted into \images\poi.png. An absolute path to the host images.
On other platforms the same url will be converted into images/poi.png. A relative path.

So, by running existing code and se if we find a file at that location the backwards compatibility will not be broken.

And by assuming relative path if no file exist you get the same result as for other platforms.

Since the Paths.VALID does not allow for backslashes “backslash (sorry windows users we are following linux conventions here)” conversion code is needed when converting URLs to resource in GeoServerDataDirectory.java.
If I add that the build is successful with mvn clean install.
I’ll try to formulate a suiting JIRA and send a PR for the change.

/Olle

···

On Tue, Feb 3, 2015 at 2:26 AM, Jody Garnett <jody.garnett@anonymised.com> wrote:

Since (in GeoServer) we try to discourage the use of absolute paths I guess I am fine with the way things stand.

If we were just focused on manipulating the file path in a safe manner the Java Paths API would help (since we actually want to check a file on disk I expect the Files API is fine).

Correct me if I am wrong, and absolute path on windows will end up using the C:\ notation or the \host\share\ notation - in both cases these are pretty specific and not confusable as a relative path?


Jody Garnett

On 2 February 2015 at 14:52, Olle Markljung <markljung@anonymised.com> wrote:

Yes, it checks if the file at the absolute path exists and if not assumes the path is relative.
You could swap the order making the relative path default.

Fail? The tests use paths to files that does not exist.
The result will be that the path to the file will be relative. And that’s the same result that you get on other platforms.

But yes, if your intention is to use an absolute path to something that does not exist yet it will fail.
So, by swapping the order you would get the absoulte file path as the default on windows as before for files that do not exist yet.
But if there is a file there for the relative case that will be used instead. However that seems a bit unlikely.
Should I update the PR to do that?

Still not sure how the Java Path API would help here.

/Olle

On Sun, Feb 1, 2015 at 11:26 PM, Jody Garnett <jody.garnett@anonymised.com> wrote:

One of the lines in your pull request uses the test if (!f.exists())

This only works if the DataUtilities.urlToFile method is referring to a file that has been created yet. If we try the same logic on a file that does not exist yet it will fail…


Jody Garnett

On 1 February 2015 at 06:36, Olle Markljung <markljung@anonymised.com> wrote:

Sorry for the delay.

Ticket: http://jira.codehaus.org/browse/GEOT-4990
PR: https://github.com/geotools/geotools/pull/717

This builds clean using “mvn clean install” on my machine (building geotools).
Should I communicate this on the geotools list as well?

Not sure that I understand what you mean Jody.
If I have gotten this right the problem is to know if the user provide an absolute or relative path.
How would the path API help in knowing the intentions of the user?

/Olle

On Wed, Jan 21, 2015 at 12:33 AM, Jody Garnett <jody.garnett@anonymised.com> wrote:

Sounds like a good approach - we may also be able to use Java 7 Path API.

I should also point out that we may be using this to figure out a the location of a new file (like one that does not exist yet). Perhaps the Java 7 path api can help.

Jody


Jody Garnett

On 20 January 2015 at 15:28, Olle Markljung <markljung@anonymised.com> wrote:

Ok,
Perhaps it is not easy to say in what ways it all should work. Someone ought to be depending on the code doing this specific thing on Windows since the code exists.
So, I got a proposition.
What if we in DataUtilities.urlToFile do the same as DefaultResourceLocator.locateResource already does.
That is to check if the file exists. If it doesn’t we remove the extra slashes and makes the URI relative.

I extended the tests and added such code to the Windows specific case and it works as expected.
Should I create a ticket and send a PR with these changes for your review?

/Olle

On Mon, Jan 19, 2015 at 11:42 PM, Olle Markljung <markljung@anonymised.com> wrote:

Ok.
Looking at the history I can’t find anything that changed. Version of Java did change to 7.

These urls also fail in GeoTools.
It’s not the usage of urlToFile in the test that’s the problem but the usage of urlToFile in the DefaultResourceLocator.locateResource.
The file:// is converted to // by urlToFile and this means it’s not relative. You’ll get the same behavior if you use file:/ instead.
Therefore locateResource will leave the URI as is and not make it relative to the styles folder.

So, on Windows the urls will be interpreted as absolute but on other platforms it will be relative. Atleast the file:// case.
Should file://host/share/dest.png be supported on Windows but not elsewhere?
Would \host\share\dest.png work an be a acceptable replacement?
Is it intentional that file:/ will be an absolute path and file:// not?

Something that does work is removing the forward slashes. So, file:dest.png will give you the correct behavior. As https://jira.codehaus.org/browse/GEOT-4311 says.
I’m merely trying to understand the requirements so go easy on me :slight_smile:

/Olle

On Mon, Jan 19, 2015 at 9:40 AM, Andrea Aime <andrea.aime@anonymised.com> wrote:

On Mon, Jan 19, 2015 at 9:29 AM, Olle Markljung <markljung@anonymised.com> wrote:

Yes, I see now that I was a bit unclear with my intentions of the last paragraph.

However, I believe that the usage of the file protocol makes it unclear as when the file path will be interpreted as relative over absolute.

I can get back to you with some exemples and maybe we can document the requirements and expected behavior.

The test is checking for behavior that was working up to 2.5.x. Both worked:
file://dest.png
file.//./dest.png

Cheers

Andrea

==

GeoServer Professional Services from the experts! Visit
http://goo.gl/NWWaa2 for more information.

==

Ing. Andrea Aime

@geowolf
Technical Lead

GeoSolutions S.A.S.
Via Poggio alle Viti 1187
55054 Massarosa (LU)
Italy
phone: +39 0584 962313
fax: +39 0584 1660272
mob: +39 339 8844549

http://www.geo-solutions.it
http://twitter.com/geosolutions_it

AVVERTENZE AI SENSI DEL D.Lgs. 196/2003

Le informazioni contenute in questo messaggio di posta elettronica e/o nel/i file/s allegato/i sono da considerarsi strettamente riservate. Il loro utilizzo è consentito esclusivamente al destinatario del messaggio, per le finalità indicate nel messaggio stesso. Qualora riceviate questo messaggio senza esserne il destinatario, Vi preghiamo cortesemente di darcene notizia via e-mail e di procedere alla distruzione del messaggio stesso, cancellandolo dal Vostro sistema. Conservare il messaggio stesso, divulgarlo anche in parte, distribuirlo ad altri soggetti, copiarlo, od utilizzarlo per finalità diverse, costituisce comportamento contrario ai principi dettati dal D.Lgs. 196/2003.

The information in this message and/or attachments, is intended solely for the attention and use of the named addressee(s) and may be confidential or proprietary in nature or covered by the provisions of privacy act (Legislative Decree June, 30 2003, no.196 - Italy’s New Data Protection Code).Any use not in accord with its purpose, any disclosure, reproduction, copying, distribution, or either dissemination, either whole or partial, is strictly forbidden except previous formal approval of the named addressee(s). If you are not the intended recipient, please contact immediately the sender by telephone, fax or e-mail and delete the information in this message that has been received in error. The sender does not give any warranty or accept liability as the content, accuracy or completeness of sent messages and accepts no responsibility for changes made after they were sent or for other risks which arise as a result of e-mail transmission, viruses, etc.


Thanks for the discussion Olle, the fix has been merged.

···

On 3 February 2015 at 14:45, Olle Markljung <markljung@anonymised.com> wrote:

Yes it will.
But the problem here is that we do not know the intention of the user.
Existing code has a different handling of file:// urls on windows than on other platforms.
On Windows file://images/poi.png will be converted into \images\poi.png. An absolute path to the host images.
On other platforms the same url will be converted into images/poi.png. A relative path.

So, by running existing code and se if we find a file at that location the backwards compatibility will not be broken.

And by assuming relative path if no file exist you get the same result as for other platforms.

Since the Paths.VALID does not allow for backslashes “backslash (sorry windows users we are following linux conventions here)” conversion code is needed when converting URLs to resource in GeoServerDataDirectory.java.
If I add that the build is successful with mvn clean install.
I’ll try to formulate a suiting JIRA and send a PR for the change.

/Olle


Jody Garnett

On Tue, Feb 3, 2015 at 2:26 AM, Jody Garnett <jody.garnett@anonymised.com> wrote:

Since (in GeoServer) we try to discourage the use of absolute paths I guess I am fine with the way things stand.

If we were just focused on manipulating the file path in a safe manner the Java Paths API would help (since we actually want to check a file on disk I expect the Files API is fine).

Correct me if I am wrong, and absolute path on windows will end up using the C:\ notation or the \host\share\ notation - in both cases these are pretty specific and not confusable as a relative path?


Jody Garnett

On 2 February 2015 at 14:52, Olle Markljung <markljung@anonymised.com> wrote:

Yes, it checks if the file at the absolute path exists and if not assumes the path is relative.
You could swap the order making the relative path default.

Fail? The tests use paths to files that does not exist.
The result will be that the path to the file will be relative. And that’s the same result that you get on other platforms.

But yes, if your intention is to use an absolute path to something that does not exist yet it will fail.
So, by swapping the order you would get the absoulte file path as the default on windows as before for files that do not exist yet.
But if there is a file there for the relative case that will be used instead. However that seems a bit unlikely.
Should I update the PR to do that?

Still not sure how the Java Path API would help here.

/Olle

On Sun, Feb 1, 2015 at 11:26 PM, Jody Garnett <jody.garnett@anonymised.com> wrote:

One of the lines in your pull request uses the test if (!f.exists())

This only works if the DataUtilities.urlToFile method is referring to a file that has been created yet. If we try the same logic on a file that does not exist yet it will fail…


Jody Garnett

On 1 February 2015 at 06:36, Olle Markljung <markljung@anonymised.com> wrote:

Sorry for the delay.

Ticket: http://jira.codehaus.org/browse/GEOT-4990
PR: https://github.com/geotools/geotools/pull/717

This builds clean using “mvn clean install” on my machine (building geotools).
Should I communicate this on the geotools list as well?

Not sure that I understand what you mean Jody.
If I have gotten this right the problem is to know if the user provide an absolute or relative path.
How would the path API help in knowing the intentions of the user?

/Olle

On Wed, Jan 21, 2015 at 12:33 AM, Jody Garnett <jody.garnett@anonymised.com> wrote:

Sounds like a good approach - we may also be able to use Java 7 Path API.

I should also point out that we may be using this to figure out a the location of a new file (like one that does not exist yet). Perhaps the Java 7 path api can help.

Jody


Jody Garnett

On 20 January 2015 at 15:28, Olle Markljung <markljung@anonymised.com> wrote:

Ok,
Perhaps it is not easy to say in what ways it all should work. Someone ought to be depending on the code doing this specific thing on Windows since the code exists.
So, I got a proposition.
What if we in DataUtilities.urlToFile do the same as DefaultResourceLocator.locateResource already does.
That is to check if the file exists. If it doesn’t we remove the extra slashes and makes the URI relative.

I extended the tests and added such code to the Windows specific case and it works as expected.
Should I create a ticket and send a PR with these changes for your review?

/Olle

On Mon, Jan 19, 2015 at 11:42 PM, Olle Markljung <markljung@anonymised.com> wrote:

Ok.
Looking at the history I can’t find anything that changed. Version of Java did change to 7.

These urls also fail in GeoTools.
It’s not the usage of urlToFile in the test that’s the problem but the usage of urlToFile in the DefaultResourceLocator.locateResource.
The file:// is converted to // by urlToFile and this means it’s not relative. You’ll get the same behavior if you use file:/ instead.
Therefore locateResource will leave the URI as is and not make it relative to the styles folder.

So, on Windows the urls will be interpreted as absolute but on other platforms it will be relative. Atleast the file:// case.
Should file://host/share/dest.png be supported on Windows but not elsewhere?
Would \host\share\dest.png work an be a acceptable replacement?
Is it intentional that file:/ will be an absolute path and file:// not?

Something that does work is removing the forward slashes. So, file:dest.png will give you the correct behavior. As https://jira.codehaus.org/browse/GEOT-4311 says.
I’m merely trying to understand the requirements so go easy on me :slight_smile:

/Olle

On Mon, Jan 19, 2015 at 9:40 AM, Andrea Aime <andrea.aime@anonymised.com> wrote:

On Mon, Jan 19, 2015 at 9:29 AM, Olle Markljung <markljung@anonymised.com> wrote:

Yes, I see now that I was a bit unclear with my intentions of the last paragraph.

However, I believe that the usage of the file protocol makes it unclear as when the file path will be interpreted as relative over absolute.

I can get back to you with some exemples and maybe we can document the requirements and expected behavior.

The test is checking for behavior that was working up to 2.5.x. Both worked:
file://dest.png
file.//./dest.png

Cheers

Andrea

==

GeoServer Professional Services from the experts! Visit
http://goo.gl/NWWaa2 for more information.

==

Ing. Andrea Aime

@geowolf
Technical Lead

GeoSolutions S.A.S.
Via Poggio alle Viti 1187
55054 Massarosa (LU)
Italy
phone: +39 0584 962313
fax: +39 0584 1660272
mob: +39 339 8844549

http://www.geo-solutions.it
http://twitter.com/geosolutions_it

AVVERTENZE AI SENSI DEL D.Lgs. 196/2003

Le informazioni contenute in questo messaggio di posta elettronica e/o nel/i file/s allegato/i sono da considerarsi strettamente riservate. Il loro utilizzo è consentito esclusivamente al destinatario del messaggio, per le finalità indicate nel messaggio stesso. Qualora riceviate questo messaggio senza esserne il destinatario, Vi preghiamo cortesemente di darcene notizia via e-mail e di procedere alla distruzione del messaggio stesso, cancellandolo dal Vostro sistema. Conservare il messaggio stesso, divulgarlo anche in parte, distribuirlo ad altri soggetti, copiarlo, od utilizzarlo per finalità diverse, costituisce comportamento contrario ai principi dettati dal D.Lgs. 196/2003.

The information in this message and/or attachments, is intended solely for the attention and use of the named addressee(s) and may be confidential or proprietary in nature or covered by the provisions of privacy act (Legislative Decree June, 30 2003, no.196 - Italy’s New Data Protection Code).Any use not in accord with its purpose, any disclosure, reproduction, copying, distribution, or either dissemination, either whole or partial, is strictly forbidden except previous formal approval of the named addressee(s). If you are not the intended recipient, please contact immediately the sender by telephone, fax or e-mail and delete the information in this message that has been received in error. The sender does not give any warranty or accept liability as the content, accuracy or completeness of sent messages and accepts no responsibility for changes made after they were sent or for other risks which arise as a result of e-mail transmission, viruses, etc.