[GRASS-dev] Regarding testing

GRASS GRASS developer
1

On Fri, May 4, 2018 at 10:23 PM, Anna Petrášová <kratochanna@gmail.com> wrote:

On Fri, May 4, 2018 at 1:19 PM, Sanjeet <edu.sanjeet@gmail.com> wrote:

Also, I was wondering if there is any priority for working with testsuites?

Look at the tests in Python scripting library
(lib/python/script/testsuite). Test it under Python 3 to see how they
work, specifically test_utils.py will definitely fail for Python 3,
there will be changes needed in utils.py and its test to make it work
(we can discuss those). I am not sure if the other tests work with
Python 3, so let us know. You should try to add some where you feel
there is not enough testing coverage. Please keep asking if
something's unclear.

Hi Anna,

I made some minor changes in 'utils.py' in lib/python/script and
'test_utils.py' in its testsuite.
https://github.com/sanjeetbhatti/FullSupportPython3/blob/master/patch.diff

The Python 3's bytes cannot contain ASCII literal characters,
therefore I can't seem to test it as such, I encoded it first before
checking for an assertion, which I am not sure is a correct way to do
so as we are ensuring that we do not pass any bytes with ASCII literal
characters.
Same thing is in test_start_command_functions.py.
Moreover, I generalized the encode and decode functions by using
string conversion for int and float values.

Please let me know if this is the correct way to go about it.

Thanks
--
Sanjeet Bhatti

2

On 8 May 2018 at 07:39, Sanjeet <edu.sanjeet@gmail.com> wrote:

Hi Anna,

Hi,

I made some minor changes in 'utils.py' in lib/python/script and
'test_utils.py' in its testsuite.
https://github.com/sanjeetbhatti/FullSupportPython3/blob/master/patch.diff

The Python 3's bytes cannot contain ASCII literal characters,
therefore I can't seem to test it as such, I encoded it first before
checking for an assertion, which I am not sure is a correct way to do
so as we are ensuring that we do not pass any bytes with ASCII literal
characters.

could be this a problem with languages using strange accent mark?
We had and have problem with some languages in translations

Same thing is in test_start_command_functions.py.
Moreover, I generalized the encode and decode functions by using
string conversion for int and float values.

Please let me know if this is the correct way to go about it.

It seems correct to me, but please Anna answer

Thanks
--
Sanjeet Bhatti

--
ciao
Luca

www.lucadelu.org

3

On Wed, May 9, 2018 at 3:30 AM, Luca Delucchi <lucadeluge@gmail.com> wrote:

On 8 May 2018 at 07:39, Sanjeet <edu.sanjeet@gmail.com> wrote:

I made some minor changes in 'utils.py' in lib/python/script and
'test_utils.py' in its testsuite.
https://github.com/sanjeetbhatti/FullSupportPython3/blob/master/patch.diff

The Python 3's bytes cannot contain ASCII literal characters,
therefore I can't seem to test it as such, I encoded it first before
checking for an assertion, which I am not sure is a correct way to do
so as we are ensuring that we do not pass any bytes with ASCII literal
characters.

could be this a problem with languages using strange accent mark?
We had and have problem with some languages in translations

Same thing is in test_start_command_functions.py.
Moreover, I generalized the encode and decode functions by using
string conversion for int and float values.

Please let me know if this is the correct way to go about it.

It seems correct to me, but please Anna answer

Hi Anna and Luca,

I have been working with scripts and testing them with python3 and I
made some changes in 2 different patches:
1. patch.diff -> has changes for unicode/bytes in utils and its
test_utils.py along with test_start_command_functions.py (depending on
the changes in utils)
2. raster.diff -> has changes for raster.py

Please take a look here:
https://github.com/sanjeetbhatti/FullSupportPython3

For other tests, I have created a google sheet:
https://docs.google.com/spreadsheets/d/1eAIJtegNWSbBscFChhQzMTl0vmbGPjmrW_013hSw9bI

I'm working on few other testsuites and hoping to have a couple of
patches submitted by tomorrow.

Thanks
--
Sanjeet Bhatti

4

On 13 May 2018 at 08:12, Sanjeet <edu.sanjeet@gmail.com> wrote:

On Wed, May 9, 2018 at 3:30 AM, Luca Delucchi <lucadeluge@gmail.com> wrote:

On 8 May 2018 at 07:39, Sanjeet <edu.sanjeet@gmail.com> wrote:

I made some minor changes in 'utils.py' in lib/python/script and
'test_utils.py' in its testsuite.
https://github.com/sanjeetbhatti/FullSupportPython3/blob/master/patch.diff

The Python 3's bytes cannot contain ASCII literal characters,
therefore I can't seem to test it as such, I encoded it first before
checking for an assertion, which I am not sure is a correct way to do
so as we are ensuring that we do not pass any bytes with ASCII literal
characters.

could be this a problem with languages using strange accent mark?
We had and have problem with some languages in translations

Same thing is in test_start_command_functions.py.
Moreover, I generalized the encode and decode functions by using
string conversion for int and float values.

Please let me know if this is the correct way to go about it.

It seems correct to me, but please Anna answer

Hi Anna and Luca,

Hi Sanjeet

I have been working with scripts and testing them with python3 and I
made some changes in 2 different patches:
1. patch.diff -> has changes for unicode/bytes in utils and its
test_utils.py along with test_start_command_functions.py (depending on
the changes in utils)
2. raster.diff -> has changes for raster.py

What other devs think about the patches?

They seem ok for me, I tested and they work

Please take a look here:
https://github.com/sanjeetbhatti/FullSupportPython3

For other tests, I have created a google sheet:
https://docs.google.com/spreadsheets/d/1eAIJtegNWSbBscFChhQzMTl0vmbGPjmrW_013hSw9bI

A comment about results, for me the test_doctest.py is working fine
with Python 3.5.3

100%
100%
Loading floating point data with 8 bytes ... (6x4x3)
100%
Loading floating point data with 8 bytes ... (6x4x3)
100%
.
----------------------------------------------------------------------
Ran 1 test in 0.785s
OK

I'm working on few other testsuites and hoping to have a couple of
patches submitted by tomorrow.

Thanks
--
Sanjeet Bhatti

--
ciao
Luca

www.lucadelu.org

5

On Tue, May 15, 2018 at 7:47 PM, Luca Delucchi <lucadeluge@gmail.com> wrote:

On 13 May 2018 at 08:12, Sanjeet <edu.sanjeet@gmail.com> wrote:
Hi Sanjeet
What other devs think about the patches?

They seem ok for me, I tested and they work

Please take a look here:
https://github.com/sanjeetbhatti/FullSupportPython3

For other tests, I have created a google sheet:
https://docs.google.com/spreadsheets/d/1eAIJtegNWSbBscFChhQzMTl0vmbGPjmrW_013hSw9bI

Hi Luca,

Sorry about the access, I thought I already had given access, but it
is provided now. :slight_smile:

A comment about results, for me the test_doctest.py is working fine
with Python 3.5.3

100%
100%
Loading floating point data with 8 bytes ... (6x4x3)
100%
Loading floating point data with 8 bytes ... (6x4x3)
100%
.
----------------------------------------------------------------------
Ran 1 test in 0.785s
OK

I will test it again on the Python 3.6.5 as well as on the version
that you mentioned (3.5.3).

Thanks
--
Sanjeet Bhatti

6

Hi Sanjeet and others,

sorry for late answer, I hope to be more responsive now.

I created a ticket where we can discuss changes for Python Scripting
Library [1]. I uploaded your patch, so please upload your future
patches there, you can upload final patches to your github (I suggest
to structure your repo in some way and use more descriptive names than
patch.diff).
Look at my patch, hopefully it makes sense.

We discussed further changes in this library regarding unicode/bytes
handling with Vaclav and collected ideas here [2]. Basically, we need
to adjust many functions to accept and return unicode ('str' in
Python3), otherwise the functions won't behave as expected when used
by Python 3 users. For Popen wrapper functions (read_command,
run_command, ...) we suggest to use new encoding parameter (similar to
Popen in Python3.6) allowing to switch the behavior for fine control,
with using unicode by default (to work as expected by Python 3 users).
There will be problems because it's not garbage in garbage out as it
used to be, so something encoded on Windows won't be read correctly on
Linux. However this is our current idea, we can discuss it further.

So, Sanjeet, let me know if it makes sense to you and try to implement
the changes where necessary (starting with core.py). It would be good
to write more tests during the process.

Anna

[1] https://trac.osgeo.org/grass/ticket/3565
[2] https://trac.osgeo.org/grass/wiki/Python3Support#PythonScriptingLibrary

On Wed, May 16, 2018 at 1:36 AM, Sanjeet <edu.sanjeet@gmail.com> wrote:

On Tue, May 15, 2018 at 7:47 PM, Luca Delucchi <lucadeluge@gmail.com> wrote:

On 13 May 2018 at 08:12, Sanjeet <edu.sanjeet@gmail.com> wrote:
Hi Sanjeet
What other devs think about the patches?

They seem ok for me, I tested and they work

Please take a look here:
https://github.com/sanjeetbhatti/FullSupportPython3

For other tests, I have created a google sheet:
https://docs.google.com/spreadsheets/d/1eAIJtegNWSbBscFChhQzMTl0vmbGPjmrW_013hSw9bI

Hi Luca,

Sorry about the access, I thought I already had given access, but it
is provided now. :slight_smile:

A comment about results, for me the test_doctest.py is working fine
with Python 3.5.3

100%
100%
Loading floating point data with 8 bytes ... (6x4x3)
100%
Loading floating point data with 8 bytes ... (6x4x3)
100%
.
----------------------------------------------------------------------
Ran 1 test in 0.785s
OK

I will test it again on the Python 3.6.5 as well as on the version
that you mentioned (3.5.3).

Thanks
--
Sanjeet Bhatti

7

On Mon, May 21, 2018 at 8:03 PM, Anna Petrášová <kratochanna@gmail.com> wrote:

Hi Sanjeet and others,

Hi Anna,

We discussed further changes in this library regarding unicode/bytes
handling with Vaclav and collected ideas here [2]. Basically, we need
to adjust many functions to accept and return unicode ('str' in
Python3), otherwise the functions won't behave as expected when used
by Python 3 users. For Popen wrapper functions (read_command,
run_command, ...) we suggest to use new encoding parameter (similar to
Popen in Python3.6) allowing to switch the behavior for fine control,
with using unicode by default (to work as expected by Python 3 users).
There will be problems because it's not garbage in garbage out as it
used to be, so something encoded on Windows won't be read correctly on
Linux. However this is our current idea, we can discuss it further.

So, Sanjeet, let me know if it makes sense to you and try to implement
the changes where necessary (starting with core.py). It would be good
to write more tests during the process.

I uploaded a small patch (on the ticket[1]) last night regarding these
changes in the core.py.
Let me know if I am doing it correctly.

Thanks
--
Sanjeet Bhatti

[1] https://trac.osgeo.org/grass/ticket/3565